2024-09-16T08:36:19.419 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T08:36:19.443 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T08:36:19.529 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977 branch: wip-guits-main-2024-09-13-1248 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: gabrioux@ibm.com first_in_suite: false job_id: '7906977' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/reef ' name: gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-13-1248 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: mon pg warn min per osd: 0 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 8293d73f8690540e843a81caec373f9cc29cf705 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8293d73f8690540e843a81caec373f9cc29cf705 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-guits-main-2024-09-13-1248 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 6714 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-13-1248 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8293d73f8690540e843a81caec373f9cc29cf705 targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJe+qY51gA/exjDaCVy5lZqKAKgihoKi3ZlIZj2JCX8D2c/LqQSo6UV84a8mDD+ILzoGgyLn8tC+CnyDUAicFM= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBONyPITvzvXHRIT5tuwzWAvoP1m2cbrDEYWgDGh6DD13SVjcwYp4FbqTC9VZiaG01jSKFiRuaTDmBFP8yOgvcJE= tasks: - install: branch: reef exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:reef roleless: true - print: '**** done end installing reef cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/gabrioux/src/git.ceph.com_ceph-c_wip-guits-main-2024-09-13-1248/qa/suites/orch/cephadm/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_07:44:22 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-09-16T08:36:19.530 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa; will attempt to use it 2024-09-16T08:36:19.531 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks 2024-09-16T08:36:19.531 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T08:36:19.532 INFO:teuthology.task.internal:Checking packages... 2024-09-16T08:36:19.553 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8293d73f8690540e843a81caec373f9cc29cf705' 2024-09-16T08:36:19.553 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:36:19.553 INFO:teuthology.packaging:ref: None 2024-09-16T08:36:19.553 INFO:teuthology.packaging:tag: None 2024-09-16T08:36:19.553 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-13-1248 2024-09-16T08:36:19.553 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:36:19.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-13-1248 2024-09-16T08:36:19.698 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4917.g8293d73f 2024-09-16T08:36:19.700 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T08:36:19.707 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T08:36:19.707 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T08:36:19.716 INFO:teuthology.task.internal:Saving configuration 2024-09-16T08:36:19.736 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T08:36:19.743 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T08:36:19.767 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 08:31:30.571596', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPJe+qY51gA/exjDaCVy5lZqKAKgihoKi3ZlIZj2JCX8D2c/LqQSo6UV84a8mDD+ILzoGgyLn8tC+CnyDUAicFM='} 2024-09-16T08:36:19.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 08:31:30.570063', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBONyPITvzvXHRIT5tuwzWAvoP1m2cbrDEYWgDGh6DD13SVjcwYp4FbqTC9VZiaG01jSKFiRuaTDmBFP8yOgvcJE='} 2024-09-16T08:36:19.789 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T08:36:19.796 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-09-16T08:36:19.797 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-09-16T08:36:19.797 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T08:36:19.862 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9b2a3e7be0>, signals=[15]) 2024-09-16T08:36:19.862 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T08:36:19.870 INFO:teuthology.task.internal:Opening connections... 2024-09-16T08:36:19.870 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:36:19.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:36:19.949 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-09-16T08:36:19.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:36:20.027 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T08:36:20.035 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-16T08:36:20.054 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-16T08:36:20.054 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-16T08:36:20.111 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T08:36:20.112 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T08:36:20.113 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-16T08:36:20.132 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-09-16T08:36:20.151 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-09-16T08:36:20.151 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-09-16T08:36:20.209 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T08:36:20.210 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-09-16T08:36:20.210 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T08:36:20.210 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T08:36:20.210 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T08:36:20.210 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-09-16T08:36:20.232 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T08:36:20.242 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T08:36:20.249 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T08:36:20.249 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-16T08:36:20.251 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-09-16T08:36:20.268 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T08:36:20.275 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T08:36:20.276 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-16T08:36:20.309 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-09-16T08:36:20.393 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T08:36:20.449 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T08:36:20.466 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T08:36:20.466 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T08:36:20.466 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T08:36:20.467 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T08:36:20.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:36:20.485 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-16T08:36:20.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:36:20.486 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-16T08:36:20.543 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-503.el9.x86_64 2024-09-16T08:36:20.543 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-503.el9.x86_64 2024-09-16T08:36:20.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-09-16T08:36:20.545 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-503.el9.x86_64 2024-09-16T08:36:20.545 INFO:teuthology.task.kernel:Running kernel on smithi097: 5.14.0-503.el9.x86_64 2024-09-16T08:36:20.545 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum install -y kernel 2024-09-16T08:36:22.934 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-16T08:36:24.294 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.2 MB 00:01 2024-09-16T08:36:27.809 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 20 MB 00:03 2024-09-16T08:36:28.770 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 20 MB 00:03 2024-09-16T08:36:32.269 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-16T08:36:33.237 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-16T08:36:34.702 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 19 kB 00:01 2024-09-16T08:36:35.129 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 19 kB 00:00 2024-09-16T08:36:35.457 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-16T08:36:36.083 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-16T08:36:40.937 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-16T08:36:41.730 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T08:36:42.969 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T08:36:42.969 INFO:teuthology.orchestra.run.smithi097.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T08:36:43.052 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:36:43.061 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:36:43.061 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-09-16T08:36:43.061 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:36:43.061 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout:Install 4 Packages 2024-09-16T08:36:43.062 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:36:43.063 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 79 M 2024-09-16T08:36:43.063 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 126 M 2024-09-16T08:36:43.063 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:36:43.172 INFO:teuthology.orchestra.run.smithi097.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 257 kB/s | 23 kB 00:00 2024-09-16T08:36:43.836 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T08:36:43.837 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T08:36:43.930 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:36:43.945 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-16T08:36:43.946 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:36:43.947 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 79 M 2024-09-16T08:36:43.947 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 126 M 2024-09-16T08:36:43.947 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:36:44.090 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 189 kB/s | 23 kB 00:00 2024-09-16T08:36:44.857 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-16T08:36:45.407 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 24 MB/s | 35 MB 00:01 2024-09-16T08:36:46.105 INFO:teuthology.orchestra.run.smithi097.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.2 MB/s | 16 MB 00:03 2024-09-16T08:36:46.143 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 14 MB/s | 29 MB 00:02 2024-09-16T08:36:46.143 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:36:46.143 INFO:teuthology.orchestra.run.smithi031.stdout:Total 36 MB/s | 79 MB 00:02 2024-09-16T08:36:46.847 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:36:47.229 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:36:47.229 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:36:47.255 INFO:teuthology.orchestra.run.smithi097.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.0 MB/s | 29 MB 00:04 2024-09-16T08:36:48.247 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:36:48.247 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:36:48.647 INFO:teuthology.orchestra.run.smithi097.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.2 MB/s | 35 MB 00:05 2024-09-16T08:36:48.648 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:36:48.648 INFO:teuthology.orchestra.run.smithi097.stdout:Total 14 MB/s | 79 MB 00:05 2024-09-16T08:36:49.334 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:36:49.669 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:36:49.669 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:36:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:36:50.630 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:36:50.836 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:36:51.505 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:36:51.593 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:36:53.064 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:36:53.880 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:36:53.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:36:54.383 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:36:54.505 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:36:56.828 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:36:56.951 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:36:59.637 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:36:59.681 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:01.800 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:37:01.968 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:04.819 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:06.829 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:44.131 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:44.172 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:44.522 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:44.523 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:37:44.523 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:37:44.523 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:37:44.792 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:44.832 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:45.153 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T08:37:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T08:37:45.154 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T08:37:45.348 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:45.348 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:37:45.348 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:37:45.348 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.349 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.349 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.349 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.349 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:37:45.349 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:37:45.696 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:37:45.788 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.788 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:37:45.789 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T08:37:45.789 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T08:37:45.789 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T08:37:45.789 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-16T08:37:45.789 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T08:37:45.789 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:37:45.848 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.848 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:37:45.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-09-16T08:37:45.879 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-09-16T08:37:45.879 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-09-16T08:37:45.879 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-09-16T08:37:45.879 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 82.el9 2024-09-16T08:37:45.879 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-09-16T08:37:45.879 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T08:37:45.879 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8274670 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-09-16T08:37:45.880 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:37:45.881 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T08:37:45.882 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T08:37:45.882 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-505.el9.x86_64 2024-09-16T08:37:45.882 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T08:37:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T08:37:45.948 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:37:45.948 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:37:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T08:37:45.949 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:37:45.949 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:37:46.302 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:37:46.383 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:37:46.383 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:37:46.383 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T08:37:46.384 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T08:37:46.384 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T08:37:46.384 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-16T08:37:46.384 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi097.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T08:37:46.384 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T08:37:46.443 INFO:teuthology.orchestra.run.smithi097.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T08:37:46.443 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T08:37:46.444 DEBUG:teuthology.orchestra.run.smithi097:> sudo rpm -qi grub2-tools 2024-09-16T08:37:46.481 INFO:teuthology.orchestra.run.smithi097.stdout:Name : grub2-tools 2024-09-16T08:37:46.481 INFO:teuthology.orchestra.run.smithi097.stdout:Epoch : 1 2024-09-16T08:37:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:Version : 2.06 2024-09-16T08:37:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:Release : 82.el9 2024-09-16T08:37:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:Architecture: x86_64 2024-09-16T08:37:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T08:37:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:Group : Unspecified 2024-09-16T08:37:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:Size : 8274670 2024-09-16T08:37:46.482 INFO:teuthology.orchestra.run.smithi097.stdout:License : GPLv3+ 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Packager : builder@centos.org 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Vendor : CentOS 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Summary : Support tools for GRUB. 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout:Description : 2024-09-16T08:37:46.483 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:37:46.484 INFO:teuthology.orchestra.run.smithi097.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T08:37:46.484 INFO:teuthology.orchestra.run.smithi097.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T08:37:46.484 INFO:teuthology.orchestra.run.smithi097.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T08:37:46.484 INFO:teuthology.orchestra.run.smithi097.stdout:hardware devices. 2024-09-16T08:37:46.484 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:37:46.484 INFO:teuthology.orchestra.run.smithi097.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T08:37:46.485 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T08:37:46.485 INFO:teuthology.task.kernel:Updating grub on smithi097 to boot 5.14.0-505.el9.x86_64 2024-09-16T08:37:46.485 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T08:37:46.498 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-16T08:37:47.113 INFO:teuthology.orchestra.run.smithi097.stderr:Generating grub configuration file ... 2024-09-16T08:37:47.840 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T08:37:47.861 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-16T08:37:47.863 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-16T08:37:47.878 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.AOEKmq4fmC 2024-09-16T08:37:47.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AOEKmq4fmC 2024-09-16T08:37:47.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.AOEKmq4fmC 2024-09-16T08:37:48.097 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.AOEKmq4fmC is 6KB 2024-09-16T08:37:48.145 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.AOEKmq4fmC 2024-09-16T08:37:48.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T08:37:48.222 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T08:37:48.222 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T08:37:48.222 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T08:37:48.222 INFO:teuthology.orchestra.run.smithi031.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T08:37:48.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T08:37:48.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-09-16T08:37:48.384 INFO:teuthology.orchestra.run.smithi097.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T08:37:48.422 INFO:teuthology.orchestra.run.smithi097.stderr:done 2024-09-16T08:37:48.424 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-09-16T08:37:48.439 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.dA6Upqbi4s 2024-09-16T08:37:48.439 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dA6Upqbi4s 2024-09-16T08:37:48.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.dA6Upqbi4s 2024-09-16T08:37:48.651 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.dA6Upqbi4s is 6KB 2024-09-16T08:37:48.700 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.dA6Upqbi4s 2024-09-16T08:37:48.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T08:37:48.778 INFO:teuthology.orchestra.run.smithi097.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T08:37:48.778 INFO:teuthology.orchestra.run.smithi097.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T08:37:48.778 INFO:teuthology.orchestra.run.smithi097.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T08:37:48.778 INFO:teuthology.orchestra.run.smithi097.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T08:37:48.779 DEBUG:teuthology.orchestra.run.smithi097:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T08:37:48.927 DEBUG:teuthology.orchestra.run.smithi097:> sudo shutdown -r now 2024-09-16T08:38:18.405 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-16T08:38:18.406 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T08:38:18.407 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:38:18.929 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-16T08:38:18.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-16T08:38:18.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:38:36.861 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-09-16T08:38:37.373 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.97 2024-09-16T08:38:45.869 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T08:38:45.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:38:46.374 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-16T08:38:46.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:39:45.932 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T08:39:46.377 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T08:39:57.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T08:39:57.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:39:58.277 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-16T08:39:58.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-16T08:39:58.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:39:58.622 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T08:39:58.623 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T08:39:58.623 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-16T08:39:58.640 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-505.el9.x86_64 2024-09-16T08:39:58.640 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T08:39:58.641 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T08:39:58.641 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T08:39:58.806 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-16T08:39:59.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-16T08:39:59.156 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T08:39:59.156 DEBUG:teuthology.orchestra.run.smithi097:> uname -r 2024-09-16T08:39:59.172 INFO:teuthology.orchestra.run.smithi097.stdout:5.14.0-505.el9.x86_64 2024-09-16T08:39:59.173 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T08:39:59.173 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T08:39:59.173 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T08:39:59.642 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-16T08:39:59.642 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T08:39:59.717 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-16T08:39:59.740 DEBUG:teuthology.parallel:result is None 2024-09-16T08:40:00.174 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-16T08:40:00.174 DEBUG:teuthology.orchestra.run.smithi097:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T08:40:00.266 INFO:teuthology.orchestra.run.smithi097.stdout:ttyS1 2024-09-16T08:40:00.288 DEBUG:teuthology.parallel:result is None 2024-09-16T08:40:00.288 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T08:40:00.295 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T08:40:00.295 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T08:40:00.298 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T08:40:00.316 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T08:40:00.322 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T08:40:00.331 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T08:40:00.332 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T08:40:00.357 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T08:40:00.389 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T08:40:00.396 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T08:40:00.396 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T08:40:00.422 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T08:40:00.461 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:40:00.461 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:40:00.473 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:40:00.474 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T08:40:00.475 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T08:40:00.482 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T08:40:00.483 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T08:40:00.519 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T08:40:00.547 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T08:40:00.557 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T08:40:00.557 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T08:40:00.589 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T08:40:00.605 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:40:00.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:40:00.735 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:40:00.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:40:00.861 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:40:00.861 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T08:40:00.928 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:40:00.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T08:40:01.032 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:40:01.117 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T08:40:01.160 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:40:01.160 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T08:40:01.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-16T08:40:01.231 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-09-16T08:40:01.277 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:40:01.325 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T08:40:01.800 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T08:40:01.807 INFO:teuthology.task.internal:Starting timer... 2024-09-16T08:40:01.807 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T08:40:01.817 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T08:40:01.827 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-16T08:40:01.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-09-16T08:40:01.893 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-09-16T08:40:01.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-09-16T08:40:01.953 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-09-16T08:40:01.955 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T08:40:01.955 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-09-16T08:40:01.981 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-09-16T08:40:01.982 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-09-16T08:40:02.020 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-09-16T08:40:02.021 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive'} 2024-09-16T08:40:02.021 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T08:40:02.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:40:02.051 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T08:40:02.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:40:02.088 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T08:40:02.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-09-16T08:40:02.126 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-09-16T08:40:02.163 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T08:40:02.173 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T08:40:02.175 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T08:40:02.286 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T08:40:02.296 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T08:40:02.297 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi097.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T08:43:51.147 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com')] 2024-09-16T08:43:51.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T08:43:51.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:43:51.231 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-16T08:43:51.313 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-16T08:43:51.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-16T08:43:51.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T08:43:51.393 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-09-16T08:43:51.472 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-09-16T08:43:51.472 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T08:43:51.488 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T08:43:51.488 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T08:43:51.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:43:51.491 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T08:43:51.491 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T08:43:51.521 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T08:43:51.536 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T08:43:51.541 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T08:43:51.556 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T08:43:51.563 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-09-16T08:43:51.575 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-09-16T08:43:51.579 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-09-16T08:43:51.590 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T08:43:51.591 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-09-16T08:43:51.605 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T08:43:51.605 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T08:43:51.619 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T08:43:51.664 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:43:51.668 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:43:51.668 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-16T08:43:51.668 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.668 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.669 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.669 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.672 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-09-16T08:43:51.675 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T08:43:51.675 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-09-16T08:43:51.675 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.676 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.676 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.676 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T08:43:51.677 INFO:teuthology.run_tasks:Running task install... 2024-09-16T08:43:51.688 DEBUG:teuthology.task.install:project ceph 2024-09-16T08:43:51.688 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'}} 2024-09-16T08:43:51.688 DEBUG:teuthology.task.install:config {'branch': 'reef', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'} 2024-09-16T08:43:51.689 INFO:teuthology.task.install:Using flavor: default 2024-09-16T08:43:51.695 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T08:43:51.695 INFO:teuthology.task.install:extra packages: [] 2024-09-16T08:43:51.696 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'tag': None, 'wait_for_package': False} 2024-09-16T08:43:51.696 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:43:51.696 INFO:teuthology.packaging:ref: None 2024-09-16T08:43:51.697 INFO:teuthology.packaging:tag: None 2024-09-16T08:43:51.697 INFO:teuthology.packaging:branch: reef 2024-09-16T08:43:51.697 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:43:51.697 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T08:43:51.700 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'reef', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'tag': None, 'wait_for_package': False} 2024-09-16T08:43:51.701 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:43:51.701 INFO:teuthology.packaging:ref: None 2024-09-16T08:43:51.701 INFO:teuthology.packaging:tag: None 2024-09-16T08:43:51.701 INFO:teuthology.packaging:branch: reef 2024-09-16T08:43:51.701 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:43:51.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T08:43:51.949 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/ 2024-09-16T08:43:51.950 INFO:teuthology.task.install.rpm:Package version is 18.2.4-853.gae7f3733 2024-09-16T08:43:51.977 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/ 2024-09-16T08:43:51.977 INFO:teuthology.task.install.rpm:Package version is 18.2.4-853.gae7f3733 2024-09-16T08:43:52.162 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T08:43:52.163 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:43:52.163 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T08:43:52.165 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T08:43:52.165 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:43:52.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T08:43:52.192 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T08:43:52.193 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:43:52.193 INFO:teuthology.packaging:ref: None 2024-09-16T08:43:52.193 INFO:teuthology.packaging:tag: None 2024-09-16T08:43:52.193 INFO:teuthology.packaging:branch: reef 2024-09-16T08:43:52.193 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:43:52.193 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T08:43:52.194 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T08:43:52.194 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:43:52.194 INFO:teuthology.packaging:ref: None 2024-09-16T08:43:52.194 INFO:teuthology.packaging:tag: None 2024-09-16T08:43:52.194 INFO:teuthology.packaging:branch: reef 2024-09-16T08:43:52.195 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:43:52.195 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T08:43:52.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T08:43:52.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T08:43:52.332 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T08:43:52.341 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T08:43:52.397 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-09-16T08:43:52.398 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-09-16T08:43:52.407 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-09-16T08:43:52.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-09-16T08:43:52.637 INFO:teuthology.orchestra.run.smithi097.stdout:45 files removed 2024-09-16T08:43:52.656 INFO:teuthology.orchestra.run.smithi031.stdout:45 files removed 2024-09-16T08:43:52.663 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw 2024-09-16T08:43:52.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-09-16T08:43:53.313 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 211 kB/s | 77 kB 00:00 2024-09-16T08:43:53.330 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 224 kB/s | 77 kB 00:00 2024-09-16T08:43:53.598 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 52 kB/s | 11 kB 00:00 2024-09-16T08:43:53.614 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 45 kB/s | 11 kB 00:00 2024-09-16T08:43:53.890 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 7.4 kB/s | 1.9 kB 00:00 2024-09-16T08:43:53.898 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 7.6 kB/s | 1.9 kB 00:00 2024-09-16T08:43:54.414 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T08:43:56.282 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-16T08:43:59.265 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.2 MB 00:05 2024-09-16T08:44:00.692 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-16T08:44:01.415 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 19 MB/s | 20 MB 00:01 2024-09-16T08:44:02.134 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 101 kB/s | 19 kB 00:00 2024-09-16T08:44:03.096 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-16T08:44:06.475 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.5 MB 00:01 2024-09-16T08:44:08.251 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-16T08:44:08.702 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-16T08:44:09.155 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-16T08:44:11.165 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:44:11.172 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:44:11.172 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:11.172 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:44:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:18.2.4-853.gae7f3733.el9 ceph 7.7 M 2024-09-16T08:44:11.172 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 3.3 M 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 3.0 M 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:18.2.4-853.gae7f3733.el9 ceph 5.1 M 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:18.2.4-853.gae7f3733.el9 ceph 18 M 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:18.2.4-853.gae7f3733.el9 ceph 25 k 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T08:44:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 711 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 477 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 4.5 M 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T08:44:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T08:44:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:18.2.4-853.gae7f3733.el9 ceph 45 k 2024-09-16T08:44:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:18.2.4-853.gae7f3733.el9 ceph 129 k 2024-09-16T08:44:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:18.2.4-853.gae7f3733.el9 ceph 162 k 2024-09-16T08:44:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T08:44:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:18.2.4-853.gae7f3733.el9 ceph 321 k 2024-09-16T08:44:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:18.2.4-853.gae7f3733.el9 ceph 300 k 2024-09-16T08:44:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:18.2.4-853.gae7f3733.el9 ceph 99 k 2024-09-16T08:44:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T08:44:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T08:44:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:11.177 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:44:11.177 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:11.177 INFO:teuthology.orchestra.run.smithi031.stdout:Install 28 Packages 2024-09-16T08:44:11.177 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-09-16T08:44:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:11.178 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 53 M 2024-09-16T08:44:11.178 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:44:11.862 INFO:teuthology.orchestra.run.smithi031.stdout:(1/30): ceph-base-18.2.4-853.gae7f3733.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-09-16T08:44:11.928 INFO:teuthology.orchestra.run.smithi031.stdout:(2/30): ceph-selinux-18.2.4-853.gae7f3733.el9.x 379 kB/s | 25 kB 00:00 2024-09-16T08:44:12.037 INFO:teuthology.orchestra.run.smithi031.stdout:(3/30): ceph-radosgw-18.2.4-853.gae7f3733.el9.x 12 MB/s | 7.7 MB 00:00 2024-09-16T08:44:12.079 INFO:teuthology.orchestra.run.smithi031.stdout:(4/30): libcephfs2-18.2.4-853.gae7f3733.el9.x86 4.6 MB/s | 711 kB 00:00 2024-09-16T08:44:12.105 INFO:teuthology.orchestra.run.smithi031.stdout:(5/30): libradosstriper1-18.2.4-853.gae7f3733.e 7.0 MB/s | 477 kB 00:00 2024-09-16T08:44:12.172 INFO:teuthology.orchestra.run.smithi031.stdout:(6/30): python3-ceph-argparse-18.2.4-853.gae7f3 677 kB/s | 45 kB 00:00 2024-09-16T08:44:12.297 INFO:teuthology.orchestra.run.smithi031.stdout:(7/30): librgw2-18.2.4-853.gae7f3733.el9.x86_64 21 MB/s | 4.5 MB 00:00 2024-09-16T08:44:12.322 INFO:teuthology.orchestra.run.smithi031.stdout:(8/30): python3-ceph-common-18.2.4-853.gae7f373 861 kB/s | 129 kB 00:00 2024-09-16T08:44:12.347 INFO:teuthology.orchestra.run.smithi031.stdout:(9/30): python3-cephfs-18.2.4-853.gae7f3733.el9 3.1 MB/s | 162 kB 00:00 2024-09-16T08:44:12.381 INFO:teuthology.orchestra.run.smithi031.stdout:(10/30): python3-rados-18.2.4-853.gae7f3733.el9 5.3 MB/s | 321 kB 00:00 2024-09-16T08:44:12.406 INFO:teuthology.orchestra.run.smithi031.stdout:(11/30): python3-rbd-18.2.4-853.gae7f3733.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-16T08:44:12.431 INFO:teuthology.orchestra.run.smithi031.stdout:(12/30): python3-rgw-18.2.4-853.gae7f3733.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-16T08:44:12.615 INFO:teuthology.orchestra.run.smithi031.stdout:(13/30): ceph-common-18.2.4-853.gae7f3733.el9.x 15 MB/s | 18 MB 00:01 2024-09-16T08:44:12.765 INFO:teuthology.orchestra.run.smithi031.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 298 kB/s | 107 kB 00:00 2024-09-16T08:44:12.790 INFO:teuthology.orchestra.run.smithi031.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 258 kB/s | 45 kB 00:00 2024-09-16T08:44:12.816 INFO:teuthology.orchestra.run.smithi031.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 417 kB/s | 160 kB 00:00 2024-09-16T08:44:12.866 INFO:teuthology.orchestra.run.smithi031.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 844 kB/s | 42 kB 00:00 2024-09-16T08:44:12.908 INFO:teuthology.orchestra.run.smithi031.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-16T08:44:12.942 INFO:teuthology.orchestra.run.smithi031.stdout:(19/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-16T08:44:12.984 INFO:teuthology.orchestra.run.smithi031.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 946 kB/s | 111 kB 00:00 2024-09-16T08:44:13.025 INFO:teuthology.orchestra.run.smithi031.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 654 kB/s | 26 kB 00:00 2024-09-16T08:44:13.051 INFO:teuthology.orchestra.run.smithi031.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T08:44:13.076 INFO:teuthology.orchestra.run.smithi031.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.8 MB/s | 308 kB 00:00 2024-09-16T08:44:13.101 INFO:teuthology.orchestra.run.smithi031.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T08:44:13.135 INFO:teuthology.orchestra.run.smithi031.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T08:44:13.168 INFO:teuthology.orchestra.run.smithi031.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-16T08:44:13.227 INFO:teuthology.orchestra.run.smithi031.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-16T08:44:13.311 INFO:teuthology.orchestra.run.smithi031.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-16T08:44:13.402 INFO:teuthology.orchestra.run.smithi031.stdout:(29/30): librados2-18.2.4-853.gae7f3733.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-16T08:44:13.469 INFO:teuthology.orchestra.run.smithi031.stdout:(30/30): librbd1-18.2.4-853.gae7f3733.el9.x86_6 12 MB/s | 3.0 MB 00:00 2024-09-16T08:44:13.474 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:44:13.474 INFO:teuthology.orchestra.run.smithi031.stdout:Total 23 MB/s | 53 MB 00:02 2024-09-16T08:44:13.547 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:44:13.585 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:44:13.585 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:44:14.241 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:44:14.241 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:44:14.752 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 9.4 kB/s | 1.7 kB 00:00 2024-09-16T08:44:15.638 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:44:15.735 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T08:44:15.775 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T08:44:15.826 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T08:44:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T08:44:16.257 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:18.2.4-853.gae7f3733.el9.x86_64 5/32 2024-09-16T08:44:16.350 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:18.2.4-853.gae7f3733.el9.x86_64 5/32 2024-09-16T08:44:16.446 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 6/32 2024-09-16T08:44:16.475 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 7/32 2024-09-16T08:44:16.537 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 7/32 2024-09-16T08:44:16.566 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T08:44:16.611 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:18.2.4-853.gae7f3733.el9.x 9/32 2024-09-16T08:44:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 10/32 2024-09-16T08:44:16.710 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 11/32 2024-09-16T08:44:16.768 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 11/32 2024-09-16T08:44:16.877 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T08:44:16.917 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T08:44:16.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T08:44:17.007 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T08:44:17.034 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T08:44:17.071 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T08:44:17.104 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T08:44:17.119 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:44:17.134 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:17.134 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:18.2.4-853.gae7f3733.el9 ceph 7.7 M 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 3.3 M 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 3.0 M 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:18.2.4-853.gae7f3733.el9 ceph 5.1 M 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:18.2.4-853.gae7f3733.el9 ceph 18 M 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:18.2.4-853.gae7f3733.el9 ceph 25 k 2024-09-16T08:44:17.135 INFO:teuthology.orchestra.run.smithi097.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 711 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 477 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:18.2.4-853.gae7f3733.el9 ceph 4.5 M 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T08:44:17.136 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:18.2.4-853.gae7f3733.el9 ceph 45 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:18.2.4-853.gae7f3733.el9 ceph 129 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:18.2.4-853.gae7f3733.el9 ceph 162 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:18.2.4-853.gae7f3733.el9 ceph 321 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:18.2.4-853.gae7f3733.el9 ceph 300 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:18.2.4-853.gae7f3733.el9 ceph 99 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:17.137 INFO:teuthology.orchestra.run.smithi097.stdout:Install 28 Packages 2024-09-16T08:44:17.138 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 2 Packages 2024-09-16T08:44:17.138 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:17.138 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 53 M 2024-09-16T08:44:17.138 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:44:17.419 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T08:44:17.466 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 20/32 2024-09-16T08:44:17.557 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 20/32 2024-09-16T08:44:17.587 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 21/32 2024-09-16T08:44:17.652 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T08:44:17.687 INFO:teuthology.orchestra.run.smithi097.stdout:(1/30): ceph-base-18.2.4-853.gae7f3733.el9.x86_ 11 MB/s | 5.1 MB 00:00 2024-09-16T08:44:17.729 INFO:teuthology.orchestra.run.smithi097.stdout:(2/30): ceph-selinux-18.2.4-853.gae7f3733.el9.x 606 kB/s | 25 kB 00:00 2024-09-16T08:44:17.846 INFO:teuthology.orchestra.run.smithi097.stdout:(3/30): libcephfs2-18.2.4-853.gae7f3733.el9.x86 6.0 MB/s | 711 kB 00:00 2024-09-16T08:44:18.061 INFO:teuthology.orchestra.run.smithi097.stdout:(4/30): ceph-radosgw-18.2.4-853.gae7f3733.el9.x 9.2 MB/s | 7.7 MB 00:00 2024-09-16T08:44:18.085 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T08:44:18.097 INFO:teuthology.orchestra.run.smithi097.stdout:(5/30): libradosstriper1-18.2.4-853.gae7f3733.e 1.9 MB/s | 477 kB 00:00 2024-09-16T08:44:18.118 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 24/32 2024-09-16T08:44:18.147 INFO:teuthology.orchestra.run.smithi097.stdout:(6/30): python3-ceph-argparse-18.2.4-853.gae7f3 897 kB/s | 45 kB 00:00 2024-09-16T08:44:18.190 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 24/32 2024-09-16T08:44:18.269 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 25/32 2024-09-16T08:44:18.304 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:18.2.4-853.gae7f3733.el9.x86 26/32 2024-09-16T08:44:18.372 INFO:teuthology.orchestra.run.smithi097.stdout:(7/30): ceph-common-18.2.4-853.gae7f3733.el9.x8 16 MB/s | 18 MB 00:01 2024-09-16T08:44:18.398 INFO:teuthology.orchestra.run.smithi097.stdout:(8/30): python3-ceph-common-18.2.4-853.gae7f373 517 kB/s | 129 kB 00:00 2024-09-16T08:44:18.423 INFO:teuthology.orchestra.run.smithi097.stdout:(9/30): python3-cephfs-18.2.4-853.gae7f3733.el9 3.2 MB/s | 162 kB 00:00 2024-09-16T08:44:18.448 INFO:teuthology.orchestra.run.smithi097.stdout:(10/30): python3-rados-18.2.4-853.gae7f3733.el9 6.2 MB/s | 321 kB 00:00 2024-09-16T08:44:18.473 INFO:teuthology.orchestra.run.smithi097.stdout:(11/30): python3-rbd-18.2.4-853.gae7f3733.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-16T08:44:18.499 INFO:teuthology.orchestra.run.smithi097.stdout:(12/30): python3-rgw-18.2.4-853.gae7f3733.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-16T08:44:18.574 INFO:teuthology.orchestra.run.smithi097.stdout:(13/30): librgw2-18.2.4-853.gae7f3733.el9.x86_6 8.7 MB/s | 4.5 MB 00:00 2024-09-16T08:44:18.683 INFO:teuthology.orchestra.run.smithi097.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 511 kB/s | 107 kB 00:00 2024-09-16T08:44:18.700 INFO:teuthology.orchestra.run.smithi097.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-09-16T08:44:18.725 INFO:teuthology.orchestra.run.smithi097.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 710 kB/s | 160 kB 00:00 2024-09-16T08:44:18.767 INFO:teuthology.orchestra.run.smithi097.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-16T08:44:18.817 INFO:teuthology.orchestra.run.smithi097.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.2 MB/s | 111 kB 00:00 2024-09-16T08:44:18.842 INFO:teuthology.orchestra.run.smithi097.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-16T08:44:18.867 INFO:teuthology.orchestra.run.smithi097.stdout:(20/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-16T08:44:18.901 INFO:teuthology.orchestra.run.smithi097.stdout:(21/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 3.6 MB/s | 308 kB 00:00 2024-09-16T08:44:18.927 INFO:teuthology.orchestra.run.smithi097.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 3.0 MB/s | 49 kB 00:00 2024-09-16T08:44:18.952 INFO:teuthology.orchestra.run.smithi097.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-16T08:44:18.969 INFO:teuthology.orchestra.run.smithi097.stdout:(24/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 278 kB/s | 26 kB 00:00 2024-09-16T08:44:19.002 INFO:teuthology.orchestra.run.smithi097.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T08:44:19.044 INFO:teuthology.orchestra.run.smithi097.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-16T08:44:19.111 INFO:teuthology.orchestra.run.smithi097.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T08:44:19.195 INFO:teuthology.orchestra.run.smithi097.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-16T08:44:19.270 INFO:teuthology.orchestra.run.smithi097.stdout:(29/30): librados2-18.2.4-853.gae7f3733.el9.x86 15 MB/s | 3.3 MB 00:00 2024-09-16T08:44:19.320 INFO:teuthology.orchestra.run.smithi097.stdout:(30/30): librbd1-18.2.4-853.gae7f3733.el9.x86_6 14 MB/s | 3.0 MB 00:00 2024-09-16T08:44:19.324 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:44:19.325 INFO:teuthology.orchestra.run.smithi097.stdout:Total 24 MB/s | 53 MB 00:02 2024-09-16T08:44:19.393 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:44:19.427 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:44:19.427 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:44:20.054 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:44:20.055 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:44:20.514 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 27/32 2024-09-16T08:44:20.550 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 27/32 2024-09-16T08:44:21.109 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 27/32 2024-09-16T08:44:21.144 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 28/32 2024-09-16T08:44:21.243 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 28/32 2024-09-16T08:44:21.243 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T08:44:21.243 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T08:44:21.243 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:21.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 29/32 2024-09-16T08:44:21.422 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:44:21.496 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-16T08:44:21.525 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-16T08:44:21.575 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-16T08:44:21.939 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-16T08:44:21.991 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:18.2.4-853.gae7f3733.el9.x86_64 5/32 2024-09-16T08:44:22.082 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:18.2.4-853.gae7f3733.el9.x86_64 5/32 2024-09-16T08:44:22.174 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 6/32 2024-09-16T08:44:22.192 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 7/32 2024-09-16T08:44:22.255 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 7/32 2024-09-16T08:44:22.284 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-16T08:44:22.327 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:18.2.4-853.gae7f3733.el9.x 9/32 2024-09-16T08:44:22.393 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 10/32 2024-09-16T08:44:22.420 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 11/32 2024-09-16T08:44:22.485 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 11/32 2024-09-16T08:44:22.592 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-16T08:44:22.627 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-16T08:44:22.668 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-16T08:44:22.701 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-16T08:44:22.728 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-16T08:44:22.764 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-16T08:44:22.802 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-16T08:44:23.094 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-16T08:44:23.143 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 20/32 2024-09-16T08:44:23.232 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 20/32 2024-09-16T08:44:23.265 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 21/32 2024-09-16T08:44:23.300 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-16T08:44:23.703 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-16T08:44:23.729 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 24/32 2024-09-16T08:44:23.800 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 24/32 2024-09-16T08:44:23.875 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 25/32 2024-09-16T08:44:23.906 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:18.2.4-853.gae7f3733.el9.x86 26/32 2024-09-16T08:44:25.994 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 27/32 2024-09-16T08:44:26.046 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 27/32 2024-09-16T08:44:26.599 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 27/32 2024-09-16T08:44:26.631 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 28/32 2024-09-16T08:44:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 28/32 2024-09-16T08:44:26.720 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T08:44:26.720 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T08:44:26.720 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:26.749 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 29/32 2024-09-16T08:44:31.695 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 29/32 2024-09-16T08:44:31.695 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-09-16T08:44:31.695 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-09-16T08:44:31.695 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-09-16T08:44:31.695 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-09-16T08:44:31.696 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-09-16T08:44:31.696 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-09-16T08:44:31.696 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-09-16T08:44:31.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:32.466 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 30/32 2024-09-16T08:44:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 30/32 2024-09-16T08:44:32.494 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:32.494 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T08:44:32.494 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:44:32.494 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:44:32.494 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:32.496 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T08:44:32.585 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T08:44:32.586 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T08:44:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T08:44:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 1/32 2024-09-16T08:44:33.775 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 2/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 3/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 4/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 5/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 6/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 7/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:18.2.4-853.gae7f3733.el9.x 8/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:18.2.4-853.gae7f3733.el9.x86 9/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 10/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 11/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 12/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 13/32 2024-09-16T08:44:33.776 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T08:44:33.777 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T08:44:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T08:44:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T08:44:33.778 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T08:44:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:18.2.4-853.gae7f3733.el9.x86_64 29/32 2024-09-16T08:44:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T08:44:33.779 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 31/32 2024-09-16T08:44:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T08:44:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:34.693 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-09-16T08:44:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.693 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T08:44:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T08:44:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T08:44:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:34.696 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:44:35.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-09-16T08:44:35.655 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:44:35.801 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:18.2.4-853.gae7f3733.el9 ceph 40 M 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:18.2.4-853.gae7f3733.el9 ceph 166 k 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:35.802 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:44:35.803 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:35.803 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-09-16T08:44:35.803 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:35.803 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 40 M 2024-09-16T08:44:35.803 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 164 M 2024-09-16T08:44:35.803 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:44:36.234 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-16T08:44:36.267 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-16T08:44:36.317 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): libcephsqlite-18.2.4-853.gae7f3733.el9.x 711 kB/s | 166 kB 00:00 2024-09-16T08:44:36.578 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 29/32 2024-09-16T08:44:36.578 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-09-16T08:44:36.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-09-16T08:44:36.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-09-16T08:44:36.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-09-16T08:44:36.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-09-16T08:44:36.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-09-16T08:44:36.579 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-09-16T08:44:36.579 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:37.310 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 30/32 2024-09-16T08:44:37.335 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 30/32 2024-09-16T08:44:37.335 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:37.335 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T08:44:37.335 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:44:37.335 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T08:44:37.335 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:37.337 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T08:44:37.420 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-16T08:44:37.420 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T08:44:37.426 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-18.2.4-853.gae7f3733.el9.x86_6 30 MB/s | 40 MB 00:01 2024-09-16T08:44:37.428 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:44:37.428 INFO:teuthology.orchestra.run.smithi031.stdout:Total 25 MB/s | 40 MB 00:01 2024-09-16T08:44:37.457 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:44:37.479 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:44:37.479 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:44:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:44:37.829 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:44:38.330 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:44:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T08:44:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T08:44:38.494 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 3/4 2024-09-16T08:44:38.586 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T08:44:38.586 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 1/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 2/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 3/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 4/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 5/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 6/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 7/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:18.2.4-853.gae7f3733.el9.x 8/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:18.2.4-853.gae7f3733.el9.x86 9/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 10/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 11/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 12/32 2024-09-16T08:44:38.587 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 13/32 2024-09-16T08:44:38.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-16T08:44:38.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-16T08:44:38.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-16T08:44:38.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-16T08:44:38.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-16T08:44:38.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-16T08:44:38.588 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:18.2.4-853.gae7f3733.el9.x86_64 29/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-16T08:44:38.589 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 31/32 2024-09-16T08:44:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-16T08:44:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:39.421 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-09-16T08:44:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.421 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T08:44:39.422 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.423 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T08:44:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T08:44:39.424 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:39.424 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:44:39.746 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-test 2024-09-16T08:44:40.314 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:26 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:44:40.450 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:44:40.450 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:18.2.4-853.gae7f3733.el9 ceph 40 M 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:18.2.4-853.gae7f3733.el9 ceph 166 k 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:40.451 INFO:teuthology.orchestra.run.smithi097.stdout:Install 4 Packages 2024-09-16T08:44:40.452 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:40.452 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 40 M 2024-09-16T08:44:40.452 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 164 M 2024-09-16T08:44:40.452 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:44:40.928 INFO:teuthology.orchestra.run.smithi097.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-16T08:44:40.953 INFO:teuthology.orchestra.run.smithi097.stdout:(2/4): libcephsqlite-18.2.4-853.gae7f3733.el9.x 749 kB/s | 166 kB 00:00 2024-09-16T08:44:40.979 INFO:teuthology.orchestra.run.smithi097.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T08:44:42.020 INFO:teuthology.orchestra.run.smithi097.stdout:(4/4): ceph-test-18.2.4-853.gae7f3733.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-16T08:44:42.022 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:44:42.023 INFO:teuthology.orchestra.run.smithi097.stdout:Total 26 MB/s | 40 MB 00:01 2024-09-16T08:44:42.050 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:44:42.070 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:44:42.071 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:44:42.386 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 3/4 2024-09-16T08:44:42.409 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:44:42.409 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:44:42.461 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 4/4 2024-09-16T08:44:42.891 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:44:42.974 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T08:44:43.019 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T08:44:43.047 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 3/4 2024-09-16T08:44:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 4/4 2024-09-16T08:44:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 1/4 2024-09-16T08:44:44.298 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 2/4 2024-09-16T08:44:44.299 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T08:44:44.640 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:44.641 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:44:44.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-09-16T08:44:45.528 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:37 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:44:45.669 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:18.2.4-853.gae7f3733.el9 ceph 6.5 k 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:18.2.4-853.gae7f3733.el9 ceph 2.1 M 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:18.2.4-853.gae7f3733.el9 ceph 1.5 M 2024-09-16T08:44:45.672 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 246 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:18.2.4-853.gae7f3733.el9 ceph 4.7 M 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:18.2.4-853.gae7f3733.el9 ceph 17 M 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T08:44:45.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T08:44:45.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T08:44:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T08:44:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:45.675 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:44:45.675 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:45.675 INFO:teuthology.orchestra.run.smithi031.stdout:Install 33 Packages 2024-09-16T08:44:45.675 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:45.676 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 29 M 2024-09-16T08:44:45.676 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 104 M 2024-09-16T08:44:45.676 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:44:46.125 INFO:teuthology.orchestra.run.smithi031.stdout:(1/33): ceph-18.2.4-853.gae7f3733.el9.x86_64.rp 49 kB/s | 6.5 kB 00:00 2024-09-16T08:44:46.284 INFO:teuthology.orchestra.run.smithi031.stdout:(2/33): ceph-mgr-18.2.4-853.gae7f3733.el9.x86_6 5.0 MB/s | 1.5 MB 00:00 2024-09-16T08:44:46.334 INFO:teuthology.orchestra.run.smithi031.stdout:(3/33): ceph-mds-18.2.4-853.gae7f3733.el9.x86_6 6.1 MB/s | 2.1 MB 00:00 2024-09-16T08:44:46.484 INFO:teuthology.orchestra.run.smithi031.stdout:(4/33): ceph-mon-18.2.4-853.gae7f3733.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-16T08:44:46.509 INFO:teuthology.orchestra.run.smithi031.stdout:(5/33): ceph-mgr-modules-core-18.2.4-853.gae7f3 1.4 MB/s | 246 kB 00:00 2024-09-16T08:44:46.618 INFO:teuthology.orchestra.run.smithi031.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-09-16T08:44:46.685 INFO:teuthology.orchestra.run.smithi031.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-16T08:44:46.727 INFO:teuthology.orchestra.run.smithi031.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 5.6 MB/s | 1.2 MB 00:00 2024-09-16T08:44:46.739 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 3/4 2024-09-16T08:44:46.752 INFO:teuthology.orchestra.run.smithi031.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-16T08:44:46.812 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 4/4 2024-09-16T08:44:46.920 INFO:teuthology.orchestra.run.smithi031.stdout:(10/33): ceph-osd-18.2.4-853.gae7f3733.el9.x86_ 26 MB/s | 17 MB 00:00 2024-09-16T08:44:46.945 INFO:teuthology.orchestra.run.smithi031.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 574 kB/s | 125 kB 00:00 2024-09-16T08:44:46.970 INFO:teuthology.orchestra.run.smithi031.stdout:(12/33): python3-urllib3-1.26.5-6.el9.noarch.rp 986 kB/s | 215 kB 00:00 2024-09-16T08:44:47.045 INFO:teuthology.orchestra.run.smithi031.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-09-16T08:44:47.079 INFO:teuthology.orchestra.run.smithi031.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-16T08:44:47.104 INFO:teuthology.orchestra.run.smithi031.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 382 kB/s | 51 kB 00:00 2024-09-16T08:44:47.129 INFO:teuthology.orchestra.run.smithi031.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 496 kB/s | 42 kB 00:00 2024-09-16T08:44:47.180 INFO:teuthology.orchestra.run.smithi031.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-16T08:44:47.205 INFO:teuthology.orchestra.run.smithi031.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 468 kB/s | 11 kB 00:00 2024-09-16T08:44:47.230 INFO:teuthology.orchestra.run.smithi031.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-16T08:44:47.256 INFO:teuthology.orchestra.run.smithi031.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-16T08:44:47.281 INFO:teuthology.orchestra.run.smithi031.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-09-16T08:44:47.306 INFO:teuthology.orchestra.run.smithi031.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-16T08:44:47.331 INFO:teuthology.orchestra.run.smithi031.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-16T08:44:47.357 INFO:teuthology.orchestra.run.smithi031.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-16T08:44:47.390 INFO:teuthology.orchestra.run.smithi031.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-16T08:44:47.424 INFO:teuthology.orchestra.run.smithi031.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-16T08:44:47.449 INFO:teuthology.orchestra.run.smithi031.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-09-16T08:44:47.474 INFO:teuthology.orchestra.run.smithi031.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T08:44:47.500 INFO:teuthology.orchestra.run.smithi031.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-16T08:44:47.525 INFO:teuthology.orchestra.run.smithi031.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-16T08:44:47.550 INFO:teuthology.orchestra.run.smithi031.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T08:44:47.575 INFO:teuthology.orchestra.run.smithi031.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-09-16T08:44:47.601 INFO:teuthology.orchestra.run.smithi031.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-16T08:44:47.606 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:44:47.606 INFO:teuthology.orchestra.run.smithi031.stdout:Total 15 MB/s | 29 MB 00:01 2024-09-16T08:44:47.733 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:44:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:44:47.774 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:44:48.186 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:44:48.186 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:44:48.776 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 4/4 2024-09-16T08:44:48.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 1/4 2024-09-16T08:44:48.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 2/4 2024-09-16T08:44:48.776 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:49.107 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:44:49.389 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph 2024-09-16T08:44:49.422 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:44:49.479 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T08:44:49.511 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T08:44:49.544 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T08:44:49.640 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T08:44:49.734 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T08:44:49.789 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T08:44:49.824 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T08:44:49.853 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T08:44:49.913 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T08:44:49.946 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T08:44:49.956 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:35 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:44:49.995 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T08:44:50.047 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T08:44:50.081 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T08:44:50.102 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:44:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:50.104 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:44:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:50.104 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:18.2.4-853.gae7f3733.el9 ceph 6.5 k 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:18.2.4-853.gae7f3733.el9 ceph 2.1 M 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:18.2.4-853.gae7f3733.el9 ceph 1.5 M 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 246 k 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:18.2.4-853.gae7f3733.el9 ceph 4.7 M 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:18.2.4-853.gae7f3733.el9 ceph 17 M 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T08:44:50.105 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T08:44:50.106 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:50.107 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:44:50.108 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:44:50.108 INFO:teuthology.orchestra.run.smithi097.stdout:Install 33 Packages 2024-09-16T08:44:50.108 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:50.108 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 29 M 2024-09-16T08:44:50.108 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 104 M 2024-09-16T08:44:50.108 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:44:50.120 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T08:44:50.208 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T08:44:50.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T08:44:50.458 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T08:44:50.518 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T08:44:50.555 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T08:44:50.614 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T08:44:50.677 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T08:44:50.773 INFO:teuthology.orchestra.run.smithi097.stdout:(1/33): ceph-18.2.4-853.gae7f3733.el9.x86_64.rp 49 kB/s | 6.5 kB 00:00 2024-09-16T08:44:50.881 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T08:44:50.934 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T08:44:50.948 INFO:teuthology.orchestra.run.smithi097.stdout:(2/33): ceph-mds-18.2.4-853.gae7f3733.el9.x86_6 6.8 MB/s | 2.1 MB 00:00 2024-09-16T08:44:50.990 INFO:teuthology.orchestra.run.smithi097.stdout:(3/33): ceph-mgr-18.2.4-853.gae7f3733.el9.x86_6 4.2 MB/s | 1.5 MB 00:00 2024-09-16T08:44:51.015 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T08:44:51.090 INFO:teuthology.orchestra.run.smithi097.stdout:(4/33): ceph-mgr-modules-core-18.2.4-853.gae7f3 2.4 MB/s | 246 kB 00:00 2024-09-16T08:44:51.114 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T08:44:51.150 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T08:44:51.174 INFO:teuthology.orchestra.run.smithi097.stdout:(5/33): ceph-mon-18.2.4-853.gae7f3733.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-16T08:44:51.271 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T08:44:51.341 INFO:teuthology.orchestra.run.smithi097.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-16T08:44:51.444 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-853.gae7f3733.el9.n 28/33 2024-09-16T08:44:51.477 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 29/33 2024-09-16T08:44:51.505 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 29/33 2024-09-16T08:44:51.505 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:51.505 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T08:44:51.505 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:44:51.505 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:44:51.505 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:51.533 INFO:teuthology.orchestra.run.smithi097.stdout:(7/33): ceph-osd-18.2.4-853.gae7f3733.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-16T08:44:51.558 INFO:teuthology.orchestra.run.smithi097.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 491 kB/s | 106 kB 00:00 2024-09-16T08:44:51.617 INFO:teuthology.orchestra.run.smithi097.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-09-16T08:44:51.650 INFO:teuthology.orchestra.run.smithi097.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 2.6 MB/s | 1.2 MB 00:00 2024-09-16T08:44:51.675 INFO:teuthology.orchestra.run.smithi097.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-16T08:44:51.726 INFO:teuthology.orchestra.run.smithi097.stdout:(12/33): python3-pycparser-2.20-6.el9.noarch.rp 702 kB/s | 135 kB 00:00 2024-09-16T08:44:51.776 INFO:teuthology.orchestra.run.smithi097.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 347 kB/s | 35 kB 00:00 2024-09-16T08:44:51.801 INFO:teuthology.orchestra.run.smithi097.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-16T08:44:51.835 INFO:teuthology.orchestra.run.smithi097.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-16T08:44:51.860 INFO:teuthology.orchestra.run.smithi097.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-16T08:44:51.894 INFO:teuthology.orchestra.run.smithi097.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 471 kB/s | 43 kB 00:00 2024-09-16T08:44:51.927 INFO:teuthology.orchestra.run.smithi097.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 433 kB/s | 11 kB 00:00 2024-09-16T08:44:51.953 INFO:teuthology.orchestra.run.smithi097.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-16T08:44:51.978 INFO:teuthology.orchestra.run.smithi097.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 364 kB/s | 18 kB 00:00 2024-09-16T08:44:52.003 INFO:teuthology.orchestra.run.smithi097.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-16T08:44:52.028 INFO:teuthology.orchestra.run.smithi097.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-16T08:44:52.045 INFO:teuthology.orchestra.run.smithi097.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 469 kB/s | 20 kB 00:00 2024-09-16T08:44:52.071 INFO:teuthology.orchestra.run.smithi097.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-09-16T08:44:52.096 INFO:teuthology.orchestra.run.smithi097.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-16T08:44:52.121 INFO:teuthology.orchestra.run.smithi097.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-16T08:44:52.163 INFO:teuthology.orchestra.run.smithi097.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 245 kB/s | 16 kB 00:00 2024-09-16T08:44:52.188 INFO:teuthology.orchestra.run.smithi097.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-16T08:44:52.206 INFO:teuthology.orchestra.run.smithi097.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 850 kB/s | 36 kB 00:00 2024-09-16T08:44:52.231 INFO:teuthology.orchestra.run.smithi097.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-16T08:44:52.248 INFO:teuthology.orchestra.run.smithi097.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-16T08:44:52.273 INFO:teuthology.orchestra.run.smithi097.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-16T08:44:52.298 INFO:teuthology.orchestra.run.smithi097.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-16T08:44:52.302 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:44:52.303 INFO:teuthology.orchestra.run.smithi097.stdout:Total 13 MB/s | 29 MB 00:02 2024-09-16T08:44:52.424 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:44:52.460 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:44:52.460 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:44:52.872 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:44:52.872 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:44:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 30/33 2024-09-16T08:44:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 30/33 2024-09-16T08:44:53.059 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:53.059 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T08:44:53.059 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:44:53.059 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:44:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:53.662 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 31/33 2024-09-16T08:44:53.688 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 31/33 2024-09-16T08:44:53.688 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:53.688 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T08:44:53.688 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:44:53.688 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:44:53.688 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:53.921 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 32/33 2024-09-16T08:44:53.946 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 32/33 2024-09-16T08:44:53.947 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:53.947 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T08:44:53.947 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:44:53.947 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:44:53.947 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:54.013 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:18.2.4-853.gae7f3733.el9.x86_64 33/33 2024-09-16T08:44:54.140 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:44:54.202 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-16T08:44:54.233 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-16T08:44:54.275 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-16T08:44:54.365 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-16T08:44:54.444 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-16T08:44:54.493 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-16T08:44:54.533 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-16T08:44:54.562 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-16T08:44:54.597 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-16T08:44:54.630 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-16T08:44:54.671 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-16T08:44:54.705 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-16T08:44:54.740 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-16T08:44:54.778 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-16T08:44:54.852 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-16T08:44:55.005 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-16T08:44:55.086 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-16T08:44:55.139 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-16T08:44:55.178 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-16T08:44:55.235 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-16T08:44:55.290 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-16T08:44:55.474 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-16T08:44:55.524 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-16T08:44:55.555 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:18.2.4-853.gae7f3733.el9.x86_64 33/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:18.2.4-853.gae7f3733.el9.x86_64 1/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 2/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 3/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 4/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 5/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-853.gae7f3733.el9.n 6/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T08:44:55.556 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T08:44:55.557 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T08:44:55.558 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T08:44:55.592 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-16T08:44:55.682 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-16T08:44:55.723 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-16T08:44:55.838 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-16T08:44:55.989 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-853.gae7f3733.el9.n 28/33 2024-09-16T08:44:56.017 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 29/33 2024-09-16T08:44:56.043 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 29/33 2024-09-16T08:44:56.043 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:56.043 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T08:44:56.043 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:44:56.044 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T08:44:56.044 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T08:44:56.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T08:44:56.433 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T08:44:56.434 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T08:44:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T08:44:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T08:44:56.436 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:56.437 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:44:56.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-09-16T08:44:57.336 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:49 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:44:57.429 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:44:57.473 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:44:57.474 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:44:57.474 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:44:57.488 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 30/33 2024-09-16T08:44:57.518 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 30/33 2024-09-16T08:44:57.519 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:57.519 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T08:44:57.519 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:44:57.519 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T08:44:57.519 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:57.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-09-16T08:44:57.964 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 31/33 2024-09-16T08:44:57.988 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 31/33 2024-09-16T08:44:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T08:44:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:44:57.989 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T08:44:57.989 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:58.112 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:44:58.190 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 32/33 2024-09-16T08:44:58.214 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 32/33 2024-09-16T08:44:58.214 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:44:58.214 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T08:44:58.214 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:44:58.214 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T08:44:58.214 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:44:58.249 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 226 k 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:44:58.250 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:44:58.251 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T08:44:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:58.251 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 226 k 2024-09-16T08:44:58.251 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 217 k 2024-09-16T08:44:58.251 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:44:58.272 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:18.2.4-853.gae7f3733.el9.x86_64 33/33 2024-09-16T08:44:58.470 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-18.2.4-853.gae7f3733.el9.noarch.rpm 1.0 MB/s | 226 kB 00:00 2024-09-16T08:44:58.471 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:44:58.471 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-16T08:44:58.471 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:44:58.477 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:44:58.477 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:44:58.479 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:44:58.480 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:44:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:44:59.025 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:44:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:44:59.461 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:44:59.719 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:44:59.720 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:59.720 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:44:59.720 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:44:59.720 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:44:59.720 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:44:59.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T08:44:59.935 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-2:18.2.4-853.gae7f3733.el9.x86_64 33/33 2024-09-16T08:44:59.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:18.2.4-853.gae7f3733.el9.x86_64 1/33 2024-09-16T08:44:59.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 2/33 2024-09-16T08:44:59.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 3/33 2024-09-16T08:44:59.935 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 4/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 5/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-853.gae7f3733.el9.n 6/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-16T08:44:59.936 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-16T08:44:59.937 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-16T08:44:59.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-16T08:44:59.938 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-16T08:45:00.503 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-853.gae7f3733.el9 ceph 143 k 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:00.643 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:00.644 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T08:45:00.644 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:00.644 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 143 k 2024-09-16T08:45:00.644 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 395 k 2024-09-16T08:45:00.644 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:00.842 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-18.2.4-853.gae7f373 722 kB/s | 143 kB 00:00 2024-09-16T08:45:00.842 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:00.843 INFO:teuthology.orchestra.run.smithi031.stdout:Total 716 kB/s | 143 kB 00:00 2024-09-16T08:45:00.843 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:00.851 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:00.851 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:00.911 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:00.911 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:00.960 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-16T08:45:00.960 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:00.960 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:00.960 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:00.960 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:00.960 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:00.960 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T08:45:00.961 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T08:45:00.962 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T08:45:00.963 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T08:45:00.963 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T08:45:00.963 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:00.963 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:01.064 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:01.126 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.e 1/1 2024-09-16T08:45:01.152 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.e 1/1 2024-09-16T08:45:01.153 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:45:01.153 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T08:45:01.153 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:01.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-base 2024-09-16T08:45:01.937 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:47 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:02.026 INFO:teuthology.orchestra.run.smithi097.stdout:Package ceph-base-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:02.067 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:02.068 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:02.068 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:02.100 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.e 1/1 2024-09-16T08:45:02.100 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:02.100 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:02.100 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:02.100 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:02.100 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:02.142 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install cephadm 2024-09-16T08:45:02.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-09-16T08:45:02.696 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:48 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:02.827 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 226 k 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-09-16T08:45:02.828 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:02.829 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 226 k 2024-09-16T08:45:02.829 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 217 k 2024-09-16T08:45:02.829 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:02.897 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:02.991 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:03.035 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:03.036 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:03.036 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:03.066 INFO:teuthology.orchestra.run.smithi097.stdout:cephadm-18.2.4-853.gae7f3733.el9.noarch.rpm 952 kB/s | 226 kB 00:00 2024-09-16T08:45:03.067 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:03.067 INFO:teuthology.orchestra.run.smithi097.stdout:Total 946 kB/s | 226 kB 00:00 2024-09-16T08:45:03.067 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:03.072 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:03.072 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:03.075 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:03.075 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:03.101 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T08:45:03.144 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:03.595 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:45:03.687 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:03.694 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:45:03.824 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:03.824 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:03.824 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 3.5 M 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 24 k 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 16 k 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:03.825 INFO:teuthology.orchestra.run.smithi031.stdout:Install 5 Packages 2024-09-16T08:45:03.826 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:03.826 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.8 M 2024-09-16T08:45:03.826 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 87 M 2024-09-16T08:45:03.826 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:04.052 INFO:teuthology.orchestra.run.smithi031.stdout:(1/5): ceph-prometheus-alerts-18.2.4-853.gae7f3 101 kB/s | 16 kB 00:00 2024-09-16T08:45:04.060 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:45:04.078 INFO:teuthology.orchestra.run.smithi031.stdout:(2/5): ceph-grafana-dashboards-18.2.4-853.gae7f 130 kB/s | 24 kB 00:00 2024-09-16T08:45:04.128 INFO:teuthology.orchestra.run.smithi031.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 409 kB/s | 31 kB 00:00 2024-09-16T08:45:04.162 INFO:teuthology.orchestra.run.smithi031.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-16T08:45:04.270 INFO:teuthology.orchestra.run.smithi031.stdout:(5/5): ceph-mgr-dashboard-18.2.4-853.gae7f3733. 9.3 MB/s | 3.5 MB 00:00 2024-09-16T08:45:04.274 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:04.274 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.4 MB/s | 3.8 MB 00:00 2024-09-16T08:45:04.274 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:04.281 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:04.282 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:04.309 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/1 2024-09-16T08:45:04.309 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:04.309 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:04.309 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:04.309 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:04.309 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:04.364 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:04.364 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:04.527 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T08:45:04.625 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:04.711 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-16T08:45:04.750 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-16T08:45:04.779 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-853.gae7f3733.el9.no 3/5 2024-09-16T08:45:05.105 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:51 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:05.240 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-853.gae7f3733.el9 ceph 143 k 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:05.241 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-09-16T08:45:05.242 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:05.242 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 143 k 2024-09-16T08:45:05.242 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 395 k 2024-09-16T08:45:05.242 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:05.464 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-immutable-object-cache-18.2.4-853.gae7f373 642 kB/s | 143 kB 00:00 2024-09-16T08:45:05.465 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:05.465 INFO:teuthology.orchestra.run.smithi097.stdout:Total 637 kB/s | 143 kB 00:00 2024-09-16T08:45:05.465 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:05.472 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:05.473 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:05.530 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:05.531 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:05.676 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:05.732 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.e 1/1 2024-09-16T08:45:05.757 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.e 1/1 2024-09-16T08:45:05.757 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:45:05.757 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T08:45:05.757 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:05.899 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-853.gae7f3733.el9.n 4/5 2024-09-16T08:45:05.932 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 5/5 2024-09-16T08:45:06.696 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.e 1/1 2024-09-16T08:45:06.696 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:06.697 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:06.697 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:06.697 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:06.697 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:06.909 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr 2024-09-16T08:45:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 5/5 2024-09-16T08:45:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-853.gae7f3733.el9.n 1/5 2024-09-16T08:45:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 2/5 2024-09-16T08:45:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-853.gae7f3733.el9.no 3/5 2024-09-16T08:45:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-16T08:45:07.293 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:07.294 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:07.501 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:07.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T08:45:07.593 INFO:teuthology.orchestra.run.smithi097.stdout:Package ceph-mgr-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:07.635 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:07.636 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:07.636 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:07.708 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T08:45:08.097 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:00 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:08.237 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 7.4 M 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T08:45:08.238 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout:====================================================================================== 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-09-16T08:45:08.239 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:08.240 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-09-16T08:45:08.240 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 212 M 2024-09-16T08:45:08.240 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:08.271 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:54 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:08.403 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:08.404 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:45:08.404 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:08.404 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 3.5 M 2024-09-16T08:45:08.404 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:45:08.404 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 24 k 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 16 k 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout:Install 5 Packages 2024-09-16T08:45:08.405 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:08.406 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 3.8 M 2024-09-16T08:45:08.406 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 87 M 2024-09-16T08:45:08.406 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:08.585 INFO:teuthology.orchestra.run.smithi097.stdout:(1/5): ceph-grafana-dashboards-18.2.4-853.gae7f 164 kB/s | 24 kB 00:00 2024-09-16T08:45:08.610 INFO:teuthology.orchestra.run.smithi097.stdout:(2/5): ceph-prometheus-alerts-18.2.4-853.gae7f3 94 kB/s | 16 kB 00:00 2024-09-16T08:45:08.652 INFO:teuthology.orchestra.run.smithi097.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 459 kB/s | 31 kB 00:00 2024-09-16T08:45:08.694 INFO:teuthology.orchestra.run.smithi097.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-16T08:45:08.927 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.4-85 16 MB/s | 7.4 MB 00:00 2024-09-16T08:45:08.961 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 377 kB/s | 191 kB 00:00 2024-09-16T08:45:08.978 INFO:teuthology.orchestra.run.smithi097.stdout:(5/5): ceph-mgr-dashboard-18.2.4-853.gae7f3733. 6.5 MB/s | 3.5 MB 00:00 2024-09-16T08:45:08.981 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:08.981 INFO:teuthology.orchestra.run.smithi097.stdout:Total 6.6 MB/s | 3.8 MB 00:00 2024-09-16T08:45:08.982 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:08.988 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:08.989 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:09.069 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-09-16T08:45:09.084 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:09.084 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:09.119 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-16T08:45:09.145 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 242 kB/s | 18 kB 00:00 2024-09-16T08:45:09.237 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-16T08:45:09.329 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 8.1 MB/s | 3.0 MB 00:00 2024-09-16T08:45:09.340 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:09.433 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-16T08:45:09.437 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-16T08:45:09.465 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-16T08:45:09.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-853.gae7f3733.el9.no 3/5 2024-09-16T08:45:09.513 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 13 MB/s | 4.9 MB 00:00 2024-09-16T08:45:09.538 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-09-16T08:45:09.605 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-09-16T08:45:09.989 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-09-16T08:45:09.989 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:09.990 INFO:teuthology.orchestra.run.smithi031.stdout:Total 22 MB/s | 38 MB 00:01 2024-09-16T08:45:10.355 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:10.536 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-853.gae7f3733.el9.n 4/5 2024-09-16T08:45:10.573 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 5/5 2024-09-16T08:45:10.904 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:10.904 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:11.626 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 5/5 2024-09-16T08:45:11.626 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-853.gae7f3733.el9.n 1/5 2024-09-16T08:45:11.626 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 2/5 2024-09-16T08:45:11.626 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-853.gae7f3733.el9.no 3/5 2024-09-16T08:45:11.626 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-16T08:45:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:11.761 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T08:45:11.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:45:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-16T08:45:11.895 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:11.896 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:11.910 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:45:11.947 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T08:45:12.113 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T08:45:12.138 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T08:45:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T08:45:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T08:45:12.671 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:12.803 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:12.804 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-16T08:45:12.804 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:45:12.804 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-16T08:45:12.804 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:12.804 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 7.4 M 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T08:45:12.805 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T08:45:12.806 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T08:45:12.806 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:12.806 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:12.806 INFO:teuthology.orchestra.run.smithi097.stdout:====================================================================================== 2024-09-16T08:45:12.806 INFO:teuthology.orchestra.run.smithi097.stdout:Install 12 Packages 2024-09-16T08:45:12.806 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:12.806 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 38 M 2024-09-16T08:45:12.807 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 212 M 2024-09-16T08:45:12.807 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:12.949 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T08:45:13.078 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T08:45:13.435 INFO:teuthology.orchestra.run.smithi097.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 876 kB/s | 191 kB 00:00 2024-09-16T08:45:13.494 INFO:teuthology.orchestra.run.smithi097.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-16T08:45:13.678 INFO:teuthology.orchestra.run.smithi097.stdout:(3/12): ceph-mgr-diskprediction-local-18.2.4-85 16 MB/s | 7.4 MB 00:00 2024-09-16T08:45:14.028 INFO:teuthology.orchestra.run.smithi097.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 55 kB/s | 33 kB 00:00 2024-09-16T08:45:14.195 INFO:teuthology.orchestra.run.smithi097.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 36 kB/s | 18 kB 00:00 2024-09-16T08:45:14.396 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:45:14.587 INFO:teuthology.orchestra.run.smithi097.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-16T08:45:14.687 INFO:teuthology.orchestra.run.smithi097.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-16T08:45:15.287 INFO:teuthology.orchestra.run.smithi097.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.5 MB/s | 4.9 MB 00:01 2024-09-16T08:45:15.379 INFO:teuthology.orchestra.run.smithi097.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 309 kB/s | 245 kB 00:00 2024-09-16T08:45:15.486 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T08:45:15.518 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f37 12/12 2024-09-16T08:45:15.829 INFO:teuthology.orchestra.run.smithi097.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-16T08:45:16.113 INFO:teuthology.orchestra.run.smithi097.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 304 kB/s | 250 kB 00:00 2024-09-16T08:45:16.788 INFO:teuthology.orchestra.run.smithi097.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-09-16T08:45:16.789 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:16.790 INFO:teuthology.orchestra.run.smithi097.stdout:Total 9.6 MB/s | 38 MB 00:03 2024-09-16T08:45:17.149 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:17.176 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:17.176 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:17.676 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:17.677 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f37 12/12 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f37 1/12 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T08:45:18.089 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T08:45:18.090 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T08:45:18.090 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T08:45:18.090 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:45:18.090 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T08:45:18.436 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T08:45:18.534 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T08:45:18.535 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T08:45:18.536 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:18.536 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:18.553 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:45:18.656 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:45:18.704 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T08:45:18.888 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T08:45:18.920 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T08:45:18.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-09-16T08:45:18.996 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T08:45:19.532 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:11 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:19.660 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T08:45:19.671 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:19.672 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:19.672 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 49 k 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T08:45:19.673 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-09-16T08:45:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:19.675 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-09-16T08:45:19.675 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-09-16T08:45:19.675 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:19.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T08:45:20.159 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 206 kB/s | 19 kB 00:00 2024-09-16T08:45:20.176 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 233 kB/s | 26 kB 00:00 2024-09-16T08:45:20.234 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): ceph-mgr-rook-18.2.4-853.gae7f3733.el9. 293 kB/s | 49 kB 00:00 2024-09-16T08:45:20.285 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-16T08:45:20.310 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-16T08:45:20.326 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.3 MB/s | 54 kB 00:00 2024-09-16T08:45:20.360 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 643 kB/s | 32 kB 00:00 2024-09-16T08:45:20.377 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-16T08:45:20.402 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.6 MB/s | 279 kB 00:00 2024-09-16T08:45:20.437 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-16T08:45:20.461 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.8 MB/s | 90 kB 00:00 2024-09-16T08:45:20.486 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 707 kB/s | 59 kB 00:00 2024-09-16T08:45:20.528 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-16T08:45:20.532 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:20.532 INFO:teuthology.orchestra.run.smithi031.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-09-16T08:45:20.604 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:20.622 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:20.622 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:20.719 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:20.721 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:21.036 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:45:21.238 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T08:45:21.465 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T08:45:21.529 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T08:45:21.564 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T08:45:21.615 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T08:45:21.730 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T08:45:21.857 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T08:45:21.909 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T08:45:22.066 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T08:45:22.095 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f37 12/12 2024-09-16T08:45:22.686 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T08:45:22.729 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T08:45:22.763 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T08:45:22.812 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T08:45:22.841 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 13/13 2024-09-16T08:45:23.658 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 13/13 2024-09-16T08:45:23.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 1/13 2024-09-16T08:45:23.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T08:45:23.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T08:45:23.658 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T08:45:23.659 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T08:45:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T08:45:23.992 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:23.992 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T08:45:23.993 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T08:45:23.994 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:23.994 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:24.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T08:45:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f37 12/12 2024-09-16T08:45:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f37 1/12 2024-09-16T08:45:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T08:45:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T08:45:24.748 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T08:45:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T08:45:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T08:45:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T08:45:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T08:45:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T08:45:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T08:45:24.749 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T08:45:24.830 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:16 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:24.969 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:24.970 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:45:24.970 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:24.970 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 139 k 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-09-16T08:45:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:24.972 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-09-16T08:45:24.972 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-09-16T08:45:24.972 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:25.141 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T08:45:25.141 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:25.141 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:25.141 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:25.141 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T08:45:25.141 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:25.142 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:25.423 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-16T08:45:25.456 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): ceph-mgr-cephadm-18.2.4-853.gae7f3733.el 671 kB/s | 139 kB 00:00 2024-09-16T08:45:25.498 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-16T08:45:25.524 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 3.2 MB/s | 74 kB 00:00 2024-09-16T08:45:25.550 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 640 kB/s | 58 kB 00:00 2024-09-16T08:45:25.551 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-rook 2024-09-16T08:45:25.666 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-16T08:45:25.669 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:25.669 INFO:teuthology.orchestra.run.smithi031.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-16T08:45:25.744 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:25.756 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:25.863 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:25.863 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:26.124 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:12 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:26.190 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:26.257 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:26.258 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 49 k 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T08:45:26.259 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout:Install 13 Packages 2024-09-16T08:45:26.260 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:26.261 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 2.2 M 2024-09-16T08:45:26.261 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 26 M 2024-09-16T08:45:26.261 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:26.365 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T08:45:26.412 INFO:teuthology.orchestra.run.smithi097.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 265 kB/s | 26 kB 00:00 2024-09-16T08:45:26.419 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T08:45:26.437 INFO:teuthology.orchestra.run.smithi097.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 157 kB/s | 19 kB 00:00 2024-09-16T08:45:26.496 INFO:teuthology.orchestra.run.smithi097.stdout:(3/13): ceph-mgr-rook-18.2.4-853.gae7f3733.el9. 273 kB/s | 49 kB 00:00 2024-09-16T08:45:26.538 INFO:teuthology.orchestra.run.smithi097.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-16T08:45:26.571 INFO:teuthology.orchestra.run.smithi097.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-16T08:45:26.597 INFO:teuthology.orchestra.run.smithi097.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 918 kB/s | 54 kB 00:00 2024-09-16T08:45:26.639 INFO:teuthology.orchestra.run.smithi097.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-16T08:45:26.664 INFO:teuthology.orchestra.run.smithi097.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 210 kB/s | 14 kB 00:00 2024-09-16T08:45:26.698 INFO:teuthology.orchestra.run.smithi097.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 3.8 MB/s | 220 kB 00:00 2024-09-16T08:45:26.725 INFO:teuthology.orchestra.run.smithi097.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-16T08:45:26.757 INFO:teuthology.orchestra.run.smithi097.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-16T08:45:26.807 INFO:teuthology.orchestra.run.smithi097.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-09-16T08:45:26.840 INFO:teuthology.orchestra.run.smithi097.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-09-16T08:45:26.843 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:26.843 INFO:teuthology.orchestra.run.smithi097.stdout:Total 3.8 MB/s | 2.2 MB 00:00 2024-09-16T08:45:26.913 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:26.928 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:26.928 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:26.976 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T08:45:27.018 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:27.019 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:27.066 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T08:45:27.135 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T08:45:27.171 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 6/6 2024-09-16T08:45:27.527 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:27.728 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T08:45:27.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T08:45:27.839 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T08:45:27.876 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T08:45:27.917 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T08:45:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 6/6 2024-09-16T08:45:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/6 2024-09-16T08:45:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T08:45:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T08:45:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T08:45:27.969 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T08:45:28.019 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T08:45:28.120 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T08:45:28.177 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T08:45:28.250 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T08:45:28.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:28.250 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:28.250 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:28.250 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T08:45:28.250 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T08:45:28.250 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T08:45:28.251 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T08:45:28.251 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T08:45:28.251 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:28.251 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:28.506 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-09-16T08:45:28.887 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T08:45:28.942 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T08:45:28.976 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T08:45:29.033 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T08:45:29.053 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 13/13 2024-09-16T08:45:29.098 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:21 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:29.241 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:18.2.4-853.gae7f3733.el9 ceph 852 k 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:29.242 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-09-16T08:45:29.243 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:29.243 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 932 k 2024-09-16T08:45:29.243 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.7 M 2024-09-16T08:45:29.243 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:29.617 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): ceph-fuse-18.2.4-853.gae7f3733.el9.x86_6 3.3 MB/s | 852 kB 00:00 2024-09-16T08:45:29.792 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 186 kB/s | 80 kB 00:00 2024-09-16T08:45:29.793 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:29.793 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.6 MB/s | 932 kB 00:00 2024-09-16T08:45:29.805 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:29.829 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:29.830 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:29.904 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 13/13 2024-09-16T08:45:29.904 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 1/13 2024-09-16T08:45:29.904 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T08:45:29.904 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T08:45:29.904 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T08:45:29.905 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T08:45:29.917 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:29.917 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:30.113 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:30.225 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T08:45:30.226 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T08:45:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T08:45:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T08:45:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T08:45:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T08:45:30.227 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:30.227 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:30.253 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T08:45:30.328 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 2/2 2024-09-16T08:45:30.521 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T08:45:30.995 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 2/2 2024-09-16T08:45:30.995 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/2 2024-09-16T08:45:31.085 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:17 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:31.217 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:31.218 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:31.218 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:45:31.218 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:31.218 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:18.2.4-853.gae7f3733.el9 ceph-noarch 139 k 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T08:45:31.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:31.220 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:31.220 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:31.220 INFO:teuthology.orchestra.run.smithi097.stdout:Install 6 Packages 2024-09-16T08:45:31.220 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:31.221 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 7.0 M 2024-09-16T08:45:31.221 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 33 M 2024-09-16T08:45:31.221 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T08:45:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:31.286 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T08:45:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:31.286 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:31.501 INFO:teuthology.orchestra.run.smithi097.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.9 MB/s | 248 kB 00:00 2024-09-16T08:45:31.552 INFO:teuthology.orchestra.run.smithi097.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-16T08:45:31.577 INFO:teuthology.orchestra.run.smithi097.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-16T08:45:31.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-09-16T08:45:31.602 INFO:teuthology.orchestra.run.smithi097.stdout:(4/6): ceph-mgr-cephadm-18.2.4-853.gae7f3733.el 614 kB/s | 139 kB 00:00 2024-09-16T08:45:31.628 INFO:teuthology.orchestra.run.smithi097.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-16T08:45:31.728 INFO:teuthology.orchestra.run.smithi097.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-16T08:45:31.730 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:31.730 INFO:teuthology.orchestra.run.smithi097.stdout:Total 14 MB/s | 7.0 MB 00:00 2024-09-16T08:45:31.801 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:31.813 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:31.813 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:31.911 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:31.914 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:32.177 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:32.219 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:32.315 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:18.2.4-853.gae7f3733.el9 ceph 127 k 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T08:45:32.316 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:32.317 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-09-16T08:45:32.317 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 456 k 2024-09-16T08:45:32.317 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:32.378 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T08:45:32.438 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T08:45:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-18.2.4-853.gae7f3733.el9.x86_64. 646 kB/s | 127 kB 00:00 2024-09-16T08:45:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:Total 640 kB/s | 127 kB 00:00 2024-09-16T08:45:32.514 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:32.520 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:32.520 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:32.587 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:32.587 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:32.742 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:32.846 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:32.961 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T08:45:33.049 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T08:45:33.112 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T08:45:33.142 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 6/6 2024-09-16T08:45:33.264 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:33.507 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:33.507 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:33.508 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:33.508 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:33.508 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:33.508 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:33.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-09-16T08:45:33.888 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 6/6 2024-09-16T08:45:33.888 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 1/6 2024-09-16T08:45:33.888 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T08:45:33.888 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T08:45:33.888 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T08:45:33.888 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T08:45:34.160 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T08:45:34.160 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:34.160 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:34.160 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:18.2.4-853.gae7f3733.el9.noarch 2024-09-16T08:45:34.161 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T08:45:34.161 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T08:45:34.161 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T08:45:34.161 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T08:45:34.161 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T08:45:34.161 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:34.161 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:34.362 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:26 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:34.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-fuse 2024-09-16T08:45:34.465 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:34.510 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:34.511 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:34.512 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:34.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-09-16T08:45:34.994 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:20 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:35.127 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:35.127 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:35.127 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-09-16T08:45:35.127 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:18.2.4-853.gae7f3733.el9 ceph 852 k 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout:Install 2 Packages 2024-09-16T08:45:35.128 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:35.129 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 932 k 2024-09-16T08:45:35.129 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 2.7 M 2024-09-16T08:45:35.129 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:35.164 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:35.272 INFO:teuthology.orchestra.run.smithi097.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 661 kB/s | 80 kB 00:00 2024-09-16T08:45:35.302 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:35.303 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T08:45:35.303 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:35.303 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:18.2.4-853.gae7f3733.el9 ceph 31 k 2024-09-16T08:45:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T08:45:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 31 k 2024-09-16T08:45:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 139 k 2024-09-16T08:45:35.304 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:35.438 INFO:teuthology.orchestra.run.smithi097.stdout:(2/2): ceph-fuse-18.2.4-853.gae7f3733.el9.x86_6 2.9 MB/s | 852 kB 00:00 2024-09-16T08:45:35.439 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:35.439 INFO:teuthology.orchestra.run.smithi097.stdout:Total 2.9 MB/s | 932 kB 00:00 2024-09-16T08:45:35.449 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-18.2.4-853.gae7f3733.el9.x86_64 214 kB/s | 31 kB 00:00 2024-09-16T08:45:35.450 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:35.450 INFO:teuthology.orchestra.run.smithi031.stdout:Total 212 kB/s | 31 kB 00:00 2024-09-16T08:45:35.451 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:35.451 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:35.454 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:35.454 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:35.473 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:35.473 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:35.473 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:35.473 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:35.552 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:35.552 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:35.561 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:35.674 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:35.737 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:35.878 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T08:45:35.945 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 2/2 2024-09-16T08:45:36.117 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:36.347 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:36.348 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:36.348 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:36.348 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:36.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-09-16T08:45:36.618 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 2/2 2024-09-16T08:45:36.618 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/2 2024-09-16T08:45:36.893 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T08:45:36.893 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:36.893 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:36.894 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T08:45:36.894 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:36.894 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:37.118 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:37.174 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librados-devel 2024-09-16T08:45:37.211 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:37.256 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:37.257 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:37.258 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:37.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-09-16T08:45:37.764 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:23 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:37.901 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:37.902 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-09-16T08:45:37.902 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:37.903 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:18.2.4-853.gae7f3733.el9 ceph 127 k 2024-09-16T08:45:37.903 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:37.903 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-09-16T08:45:37.904 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 127 k 2024-09-16T08:45:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 456 k 2024-09-16T08:45:37.904 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:37.927 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:38.022 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:38.067 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:38.068 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:38.068 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:38.085 INFO:teuthology.orchestra.run.smithi097.stdout:librados-devel-18.2.4-853.gae7f3733.el9.x86_64. 698 kB/s | 127 kB 00:00 2024-09-16T08:45:38.086 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:38.086 INFO:teuthology.orchestra.run.smithi097.stdout:Total 691 kB/s | 127 kB 00:00 2024-09-16T08:45:38.086 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:38.092 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:38.092 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:38.134 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-09-16T08:45:38.154 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:38.155 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:38.311 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:38.399 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:38.727 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:38.820 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:38.842 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:38.865 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:38.866 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:38.933 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-09-16T08:45:39.101 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:39.101 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:39.101 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:39.101 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:39.101 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:39.101 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:39.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install libcephfs2 2024-09-16T08:45:39.526 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:39.619 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:39.665 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:39.666 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:39.666 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:39.731 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-09-16T08:45:39.937 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:25 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:40.028 INFO:teuthology.orchestra.run.smithi097.stdout:Package libcephfs2-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:40.072 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:40.073 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:40.073 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:40.157 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install libcephfs-devel 2024-09-16T08:45:40.324 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:40.417 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:40.463 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:40.464 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:40.464 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:40.527 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-09-16T08:45:40.723 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:26 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:40.856 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:18.2.4-853.gae7f3733.el9 ceph 31 k 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 31 k 2024-09-16T08:45:40.857 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 139 k 2024-09-16T08:45:40.858 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:41.005 INFO:teuthology.orchestra.run.smithi097.stdout:libcephfs-devel-18.2.4-853.gae7f3733.el9.x86_64 210 kB/s | 31 kB 00:00 2024-09-16T08:45:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:Total 208 kB/s | 31 kB 00:00 2024-09-16T08:45:41.006 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:41.010 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:41.019 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:41.101 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:41.117 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:41.210 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:41.218 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:41.255 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:41.256 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-09-16T08:45:41.256 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:41.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-09-16T08:45:41.585 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:41.825 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:41.825 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:41.826 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:41.826 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:41.942 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:42.031 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librados2 2024-09-16T08:45:42.082 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:42.082 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:18.2.4-853.gae7f3733.el9 ceph 87 k 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 87 k 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 227 k 2024-09-16T08:45:42.083 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:42.277 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-18.2.4-853.gae7f3733.el9.x86_64.rpm 447 kB/s | 87 kB 00:00 2024-09-16T08:45:42.278 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:42.278 INFO:teuthology.orchestra.run.smithi031.stdout:Total 444 kB/s | 87 kB 00:00 2024-09-16T08:45:42.278 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:42.285 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:42.286 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:42.352 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:42.352 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:42.501 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:42.597 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:28 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:42.605 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:42.687 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:42.729 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:42.730 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:42.730 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:42.802 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install librbd1 2024-09-16T08:45:42.995 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:43.264 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:43.264 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:43.265 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:43.368 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:43.458 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:43.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-09-16T08:45:43.501 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:43.502 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:43.502 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:43.580 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rados 2024-09-16T08:45:44.097 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:44.147 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:44.235 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rados-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:44.236 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:44.236 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:44.236 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:18.2.4-853.gae7f3733.el9 ceph 3.0 M 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.0 M 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 12 M 2024-09-16T08:45:44.237 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:44.278 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:44.279 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:44.279 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:44.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rgw 2024-09-16T08:45:44.571 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-18.2.4-853.gae7f3733.el9.x86_64.rpm 9.0 MB/s | 3.0 MB 00:00 2024-09-16T08:45:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.9 MB/s | 3.0 MB 00:00 2024-09-16T08:45:44.572 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:44.582 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:44.666 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:44.666 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:44.922 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:45.013 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rgw-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:45.056 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:45.056 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:45.057 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:45.107 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:45.138 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-cephfs 2024-09-16T08:45:45.185 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:45:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T08:45:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:45:45.211 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:45:45.211 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:45.703 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:45.793 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-cephfs-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:45.836 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:45.837 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:45.837 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:45.914 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install python3-rbd 2024-09-16T08:45:46.282 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:46.282 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:46.282 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:46.282 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:46.283 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:46.283 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:46.473 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:46.564 INFO:teuthology.orchestra.run.smithi097.stdout:Package python3-rbd-2:18.2.4-853.gae7f3733.el9.x86_64 is already installed. 2024-09-16T08:45:46.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-09-16T08:45:46.607 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:46.608 INFO:teuthology.orchestra.run.smithi097.stdout:Nothing to do. 2024-09-16T08:45:46.608 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:46.695 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-fuse 2024-09-16T08:45:47.164 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 08:44:08 AM UTC. 2024-09-16T08:45:47.262 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:47.303 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-09-16T08:45:47.304 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-09-16T08:45:47.304 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:47.304 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-09-16T08:45:47.304 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:18.2.4-853.gae7f3733.el9 ceph 172 k 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 172 k 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 494 k 2024-09-16T08:45:47.305 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-09-16T08:45:47.395 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:18.2.4-853.gae7f3733.el9 ceph 87 k 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:47.396 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 87 k 2024-09-16T08:45:47.397 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 227 k 2024-09-16T08:45:47.397 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:47.516 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-18.2.4-853.gae7f3733.el9.x86_64.rpm 818 kB/s | 172 kB 00:00 2024-09-16T08:45:47.517 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:47.517 INFO:teuthology.orchestra.run.smithi031.stdout:Total 811 kB/s | 172 kB 00:00 2024-09-16T08:45:47.517 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-09-16T08:45:47.525 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-09-16T08:45:47.525 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-09-16T08:45:47.592 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-09-16T08:45:47.592 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-09-16T08:45:47.593 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-fuse-18.2.4-853.gae7f3733.el9.x86_64.rpm 442 kB/s | 87 kB 00:00 2024-09-16T08:45:47.593 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:47.593 INFO:teuthology.orchestra.run.smithi097.stdout:Total 439 kB/s | 87 kB 00:00 2024-09-16T08:45:47.593 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:47.600 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:47.600 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:47.663 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:47.664 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:47.752 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-09-16T08:45:47.813 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:47.858 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:47.897 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:48.266 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:48.302 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:48.545 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:48.546 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-09-16T08:45:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:48.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:45:48.546 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-09-16T08:45:48.554 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:48.554 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:48.554 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:48.554 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:48.554 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:48.554 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:48.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-mirror 2024-09-16T08:45:48.788 DEBUG:teuthology.parallel:result is None 2024-09-16T08:45:49.347 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:49.483 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repo Size 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:18.2.4-853.gae7f3733.el9 ceph 3.0 M 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:49.484 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-09-16T08:45:49.485 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:49.485 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 3.0 M 2024-09-16T08:45:49.485 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 12 M 2024-09-16T08:45:49.485 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:49.821 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-mirror-18.2.4-853.gae7f3733.el9.x86_64.rpm 8.9 MB/s | 3.0 MB 00:00 2024-09-16T08:45:49.821 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:49.822 INFO:teuthology.orchestra.run.smithi097.stdout:Total 8.9 MB/s | 3.0 MB 00:00 2024-09-16T08:45:49.822 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:49.831 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:49.831 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:49.911 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:49.911 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:50.330 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:50.396 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:50.420 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T08:45:50.420 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T08:45:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:45:50.421 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T08:45:50.421 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:51.482 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:51.482 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:51.482 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:51.482 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:51.483 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:51.483 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:51.750 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install rbd-nbd 2024-09-16T08:45:52.336 INFO:teuthology.orchestra.run.smithi097.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 08:44:14 AM UTC. 2024-09-16T08:45:52.471 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:18.2.4-853.gae7f3733.el9 ceph 172 k 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2024-09-16T08:45:52.472 INFO:teuthology.orchestra.run.smithi097.stdout:Install 1 Package 2024-09-16T08:45:52.473 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:52.473 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 172 k 2024-09-16T08:45:52.473 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 494 k 2024-09-16T08:45:52.473 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-09-16T08:45:52.692 INFO:teuthology.orchestra.run.smithi097.stdout:rbd-nbd-18.2.4-853.gae7f3733.el9.x86_64.rpm 786 kB/s | 172 kB 00:00 2024-09-16T08:45:52.693 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:45:52.693 INFO:teuthology.orchestra.run.smithi097.stdout:Total 781 kB/s | 172 kB 00:00 2024-09-16T08:45:52.693 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-09-16T08:45:52.700 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-09-16T08:45:52.700 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-09-16T08:45:52.763 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-09-16T08:45:52.763 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-09-16T08:45:52.913 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-09-16T08:45:53.014 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:53.472 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 1/1 2024-09-16T08:45:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:53.722 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-09-16T08:45:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:18.2.4-853.gae7f3733.el9.x86_64 2024-09-16T08:45:53.722 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:45:53.722 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-09-16T08:45:53.988 DEBUG:teuthology.parallel:result is None 2024-09-16T08:45:53.989 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:45:53.989 INFO:teuthology.packaging:ref: None 2024-09-16T08:45:53.989 INFO:teuthology.packaging:tag: None 2024-09-16T08:45:53.989 INFO:teuthology.packaging:branch: reef 2024-09-16T08:45:53.989 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:45:53.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T08:45:54.227 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T08:45:54.253 INFO:teuthology.orchestra.run.smithi031.stdout:18.2.4-853.gae7f3733.el9 2024-09-16T08:45:54.254 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-853.gae7f3733.el9 2024-09-16T08:45:54.254 INFO:teuthology.task.install:The correct ceph version 18.2.4-853.gae7f3733 is installed. 2024-09-16T08:45:54.256 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T08:45:54.256 INFO:teuthology.packaging:ref: None 2024-09-16T08:45:54.256 INFO:teuthology.packaging:tag: None 2024-09-16T08:45:54.256 INFO:teuthology.packaging:branch: reef 2024-09-16T08:45:54.257 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:45:54.257 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-16T08:45:54.479 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T08:45:54.502 INFO:teuthology.orchestra.run.smithi097.stdout:18.2.4-853.gae7f3733.el9 2024-09-16T08:45:54.502 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-853.gae7f3733.el9 2024-09-16T08:45:54.502 INFO:teuthology.task.install:The correct ceph version 18.2.4-853.gae7f3733 is installed. 2024-09-16T08:45:54.504 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T08:45:54.504 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:45:54.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T08:45:54.537 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:45:54.537 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T08:45:54.569 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T08:45:54.569 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:45:54.569 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T08:45:54.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T08:45:54.678 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:45:54.678 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T08:45:54.704 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T08:45:54.770 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T08:45:54.770 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:45:54.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T08:45:54.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T08:45:54.870 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:45:54.870 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T08:45:54.895 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T08:45:54.960 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T08:45:54.961 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:45:54.961 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T08:45:54.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T08:45:55.053 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:45:55.053 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T08:45:55.077 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T08:45:55.142 INFO:teuthology.run_tasks:Running task print... 2024-09-16T08:45:55.151 INFO:teuthology.task.print:**** done install task... 2024-09-16T08:45:55.151 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T08:45:55.268 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'global': {'mon pg warn min per osd': 0}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}}, 'image': 'quay.ceph.io/ceph-ci/ceph:reef', 'roleless': True, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'} 2024-09-16T08:45:55.268 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:reef 2024-09-16T08:45:55.269 INFO:tasks.cephadm:Cluster fsid is 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:45:55.269 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T08:45:55.269 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-16T08:45:55.272 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi031': '172.21.15.31', 'mon.smithi097': '172.21.15.97'} 2024-09-16T08:45:55.272 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T08:45:55.273 DEBUG:teuthology.orchestra.run.smithi031:> sudo hostname $(hostname -s) 2024-09-16T08:45:55.299 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-09-16T08:45:55.325 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-09-16T08:45:55.325 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:45:55.465 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83046/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4917-g8293d73f', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4917.g8293d73f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-13 15:55:14.480698', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-13-1248', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-16T08:45:55.694 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 ae7f37330c1a72493684eb1fcaed6f2a01195bb6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-09-16T08:45:55.696 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm 2024-09-16T08:45:55.696 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm 2024-09-16T08:45:55.696 DEBUG:teuthology.orchestra.run.smithi031:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T08:45:56.022 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r--. 1 ubuntu ubuntu 217225 Sep 16 08:45 /home/ubuntu/cephtest/cephadm 2024-09-16T08:45:56.022 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/ae7f37330c1a72493684eb1fcaed6f2a01195bb6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T08:45:56.346 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 217225 Sep 16 08:45 /home/ubuntu/cephtest/cephadm 2024-09-16T08:45:56.347 DEBUG:teuthology.orchestra.run.smithi031:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T08:45:56.365 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T08:45:56.393 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:reef on all hosts... 2024-09-16T08:45:56.394 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-09-16T08:45:56.407 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef pull 2024-09-16T08:45:57.929 INFO:teuthology.orchestra.run.smithi031.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:45:58.041 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:46:52.393 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:46:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph_version": "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)", 2024-09-16T08:46:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "image_id": "fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95", 2024-09-16T08:46:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "repo_digests": [ 2024-09-16T08:46:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87" 2024-09-16T08:46:52.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-09-16T08:46:52.395 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:46:53.385 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-09-16T08:46:53.385 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)", 2024-09-16T08:46:53.385 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95", 2024-09-16T08:46:53.385 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-09-16T08:46:53.385 INFO:teuthology.orchestra.run.smithi097.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87" 2024-09-16T08:46:53.385 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-09-16T08:46:53.385 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-09-16T08:46:53.411 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph 2024-09-16T08:46:53.457 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-09-16T08:46:53.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /etc/ceph 2024-09-16T08:46:53.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-09-16T08:46:53.557 INFO:tasks.cephadm:Writing seed config... 2024-09-16T08:46:53.558 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-16T08:46:53.559 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T08:46:53.560 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T08:46:53.561 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:46:53.561 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T08:46:53.594 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1631ad7a-7408-11ef-bceb-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-16T08:46:53.595 DEBUG:teuthology.orchestra.run.smithi031:mon.smithi031> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service 2024-09-16T08:46:53.638 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T08:46:53.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef -v bootstrap --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.31 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:46:53.808 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-09-16T08:46:53.808 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:reef', '-v', 'bootstrap', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.31', '--skip-admin-label'] 2024-09-16T08:46:53.833 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:46:53.833 INFO:teuthology.orchestra.run.smithi031.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T08:46:53.834 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying podman|docker is present... 2024-09-16T08:46:53.856 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:46:53.856 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying lvm2 is present... 2024-09-16T08:46:53.856 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying time synchronization is in place... 2024-09-16T08:46:53.864 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T08:46:53.864 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T08:46:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T08:46:53.870 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T08:46:53.877 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-16T08:46:53.884 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-16T08:46:53.884 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-16T08:46:53.884 INFO:teuthology.orchestra.run.smithi031.stdout:Repeating the final host check... 2024-09-16T08:46:53.906 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout 5.2.2 2024-09-16T08:46:53.906 INFO:teuthology.orchestra.run.smithi031.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T08:46:53.906 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl is present 2024-09-16T08:46:53.906 INFO:teuthology.orchestra.run.smithi031.stdout:lvcreate is present 2024-09-16T08:46:53.913 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T08:46:53.913 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T08:46:53.919 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T08:46:53.919 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T08:46:53.926 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout enabled 2024-09-16T08:46:53.932 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout active 2024-09-16T08:46:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:Unit chronyd.service is enabled and running 2024-09-16T08:46:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:Host looks OK 2024-09-16T08:46:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:Cluster fsid: 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:46:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:Acquiring lock 139660378383696 on /run/cephadm/1631ad7a-7408-11ef-bceb-c7b262605968.lock 2024-09-16T08:46:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:Lock 139660378383696 acquired on /run/cephadm/1631ad7a-7408-11ef-bceb-c7b262605968.lock 2024-09-16T08:46:53.933 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 3300 ... 2024-09-16T08:46:53.934 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying IP 172.21.15.31 port 6789 ... 2024-09-16T08:46:53.934 INFO:teuthology.orchestra.run.smithi031.stdout:Base mon IP(s) is [172.21.15.31:3300, 172.21.15.31:6789], mon addrv is [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-16T08:46:53.937 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.31 metric 100 2024-09-16T08:46:53.937 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.31 metric 100 2024-09-16T08:46:53.939 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T08:46:53.939 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-16T08:46:53.940 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1788sec hoplimit 64 pref medium 2024-09-16T08:46:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T08:46:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T08:46:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T08:46:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout 3: ens1f0: mtu 1500 state UP qlen 1000 2024-09-16T08:46:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-16T08:46:53.942 INFO:teuthology.orchestra.run.smithi031.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T08:46:53.943 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-16T08:46:53.943 INFO:teuthology.orchestra.run.smithi031.stdout:Mon IP `172.21.15.31` is in CIDR network `172.21.0.0/20` 2024-09-16T08:46:53.943 INFO:teuthology.orchestra.run.smithi031.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T08:46:53.943 INFO:teuthology.orchestra.run.smithi031.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T08:46:53.944 INFO:teuthology.orchestra.run.smithi031.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:46:55.066 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stdout fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 2024-09-16T08:46:55.067 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:reef... 2024-09-16T08:46:55.067 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T08:46:55.067 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:7554792ee943fcbd9ac63c8a3057c4d86592e594de711470628678498ff1ba71 2024-09-16T08:46:55.067 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T08:46:55.067 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Copying config sha256:fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 2024-09-16T08:46:55.067 INFO:teuthology.orchestra.run.smithi031.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T08:46:55.980 INFO:teuthology.orchestra.run.smithi031.stdout:ceph: stdout ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable) 2024-09-16T08:46:55.980 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph version: ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable) 2024-09-16T08:46:55.980 INFO:teuthology.orchestra.run.smithi031.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T08:46:56.935 INFO:teuthology.orchestra.run.smithi031.stdout:stat: stdout 167 167 2024-09-16T08:46:56.936 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial keys... 2024-09-16T08:46:57.866 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQCB8OdmB4iQDxAAWZjYFQzcbDnjKFt+0sgfKA== 2024-09-16T08:46:58.764 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQCC8OdmJ7f9ChAAAnz/v0zgRIQTQqDDANtZVg== 2024-09-16T08:46:59.686 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph-authtool: stdout AQCD8Odm/beTBRAAzpejnmBEfmk59ie2UB5ezQ== 2024-09-16T08:46:59.688 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial monmap... 2024-09-16T08:47:00.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T08:47:00.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-16T08:47:00.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:00.561 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T08:47:00.562 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool for smithi031 [v2:172.21.15.31:3300,v1:172.21.15.31:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T08:47:00.562 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = pacific 2024-09-16T08:47:00.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: set fsid to 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:00.562 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T08:47:00.562 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:47:00.562 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mon... 2024-09-16T08:47:01.432 INFO:teuthology.orchestra.run.smithi031.stdout:create mon.smithi031 on 2024-09-16T08:47:01.656 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T08:47:01.804 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T08:47:01.994 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1631ad7a-7408-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1631ad7a-7408-11ef-bceb-c7b262605968.target. 2024-09-16T08:47:01.996 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1631ad7a-7408-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-1631ad7a-7408-11ef-bceb-c7b262605968.target. 2024-09-16T08:47:02.231 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031 2024-09-16T08:47:02.231 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service: Unit ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service not loaded. 2024-09-16T08:47:02.412 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1631ad7a-7408-11ef-bceb-c7b262605968.target.wants/ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service → /etc/systemd/system/ceph-1631ad7a-7408-11ef-bceb-c7b262605968@.service. 2024-09-16T08:47:02.473 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 systemd[1]: Starting Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:47:02.786 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 podman[27448]: 2024-09-16 08:47:02.559717471 +0000 UTC m=+0.016048951 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph:reef 2024-09-16T08:47:02.787 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 podman[27448]: 2024-09-16 08:47:02.672755173 +0000 UTC m=+0.129086630 container create f97153fdbf2c60cc019a439ff107029ab438336cb1170291aafaffcb0fcaa6da (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=reef-ae7f373, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:47:02.990 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:47:02.990 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T08:47:02.997 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:47:02.997 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T08:47:02.997 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T08:47:02.997 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T08:47:02.998 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon to start... 2024-09-16T08:47:02.998 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mon... 2024-09-16T08:47:03.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 podman[27448]: 2024-09-16 08:47:02.900545947 +0000 UTC m=+0.356877420 container init f97153fdbf2c60cc019a439ff107029ab438336cb1170291aafaffcb0fcaa6da (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:47:03.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 podman[27448]: 2024-09-16 08:47:02.905002031 +0000 UTC m=+0.361333491 container start f97153fdbf2c60cc019a439ff107029ab438336cb1170291aafaffcb0fcaa6da (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-16T08:47:03.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 ceph-mon[27475]: mkfs 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:03.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 ceph-mon[27475]: mon.smithi031 is new leader, mons smithi031 in quorum (ranks 0) 2024-09-16T08:47:03.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 bash[27448]: f97153fdbf2c60cc019a439ff107029ab438336cb1170291aafaffcb0fcaa6da 2024-09-16T08:47:03.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:02 smithi031 systemd[1]: Started Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T08:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout id: 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T08:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout services: 2024-09-16T08:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi031 (age 0.500884s) 2024-09-16T08:47:04.044 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout data: 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:mon is available 2024-09-16T08:47:04.045 INFO:teuthology.orchestra.run.smithi031.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T08:47:04.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:03 smithi031 ceph-mon[27475]: mon.smithi031 is new leader, mons smithi031 in quorum (ranks 0) 2024-09-16T08:47:04.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:03 smithi031 ceph-mon[27475]: monmap e1: 1 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:47:04.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:03 smithi031 ceph-mon[27475]: fsmap 2024-09-16T08:47:04.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:03 smithi031 ceph-mon[27475]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T08:47:04.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:03 smithi031 ceph-mon[27475]: mgrmap e1: no daemons active 2024-09-16T08:47:04.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:03 smithi031 ceph-mon[27475]: from='client.? 172.21.15.31:0/1441446901' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T08:47:05.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:04 smithi031 ceph-mon[27475]: from='client.? 172.21.15.31:0/2424444553' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T08:47:05.084 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:04 smithi031 ceph-mon[27475]: from='client.? 172.21.15.31:0/2424444553' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.31:3300,v1:172.21.15.31:6789] 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T08:47:05.086 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T08:47:05.087 INFO:teuthology.orchestra.run.smithi031.stdout:Generating new minimal ceph.conf... 2024-09-16T08:47:06.049 INFO:teuthology.orchestra.run.smithi031.stdout:Restarting the monitor... 2024-09-16T08:47:06.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:05 smithi031 ceph-mon[27475]: from='client.? 172.21.15.31:0/3195582949' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:47:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:06 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:47:06.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:06 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[27471]: 2024-09-16T08:47:06.341+0000 7f4e044f8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:47:06.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:06 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[27471]: 2024-09-16T08:47:06.341+0000 7f4e044f8640 -1 mon.smithi031@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T08:47:06.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:06 smithi031 podman[27720]: 2024-09-16 08:47:06.546146842 +0000 UTC m=+0.323223976 container died f97153fdbf2c60cc019a439ff107029ab438336cb1170291aafaffcb0fcaa6da (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T08:47:07.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:06 smithi031 podman[27720]: 2024-09-16 08:47:06.899066179 +0000 UTC m=+0.676143310 container cleanup f97153fdbf2c60cc019a439ff107029ab438336cb1170291aafaffcb0fcaa6da (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T08:47:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:06 smithi031 bash[27720]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031 2024-09-16T08:47:07.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 podman[27732]: 2024-09-16 08:47:07.20027039 +0000 UTC m=+0.645239370 container remove f97153fdbf2c60cc019a439ff107029ab438336cb1170291aafaffcb0fcaa6da (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=reef-ae7f373, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-16T08:47:07.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service: Deactivated successfully. 2024-09-16T08:47:07.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 systemd[1]: Stopped Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:47:07.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 systemd[1]: Starting Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 podman[27825]: 2024-09-16 08:47:07.567655609 +0000 UTC m=+0.092588026 container create 31533db037fd5c9ce47a7df4f0d299b33538bf4d078523f2824b3c5d8a917e52 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 podman[27825]: 2024-09-16 08:47:07.490834444 +0000 UTC m=+0.015766878 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph:reef 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 podman[27825]: 2024-09-16 08:47:07.769912332 +0000 UTC m=+0.294844751 container init 31533db037fd5c9ce47a7df4f0d299b33538bf4d078523f2824b3c5d8a917e52 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 podman[27825]: 2024-09-16 08:47:07.77328231 +0000 UTC m=+0.298214737 container start 31533db037fd5c9ce47a7df4f0d299b33538bf4d078523f2824b3c5d8a917e52 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable), process ceph-mon, pid 2 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: pidfile_write: ignore empty --pid-file 2024-09-16T08:47:07.854 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: load: jerasure load: lrc 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Git sha 0 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Compile date 2024-09-15 20:36:04 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: DB SUMMARY 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: DB Session ID: TROB5XIQ4NHO4PB42FPF 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi031/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi031/store.db: 000009.log size: 91100 ; 2024-09-16T08:47:07.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:47:07.856 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:47:07.856 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:47:07.857 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:47:07.857 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.env: 0x55d647cc5bc0 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.info_log: 0x55d649574dc0 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.statistics: (nil) 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.use_fsync: 0 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:47:07.858 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.db_log_dir: 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.wal_dir: 2024-09-16T08:47:07.859 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.write_buffer_manager: 0x55d649582320 2024-09-16T08:47:07.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.unordered_write: 0 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:47:07.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.row_cache: None 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.wal_filter: None 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.wal_compression: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:47:07.862 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:47:07.863 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_open_files: -1 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Compression algorithms supported: 2024-09-16T08:47:07.864 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kZSTD supported: 0 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000010 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:47:07.865 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.merge_operator: 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_filter: None 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d649574a20) 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_type: 0 2024-09-16T08:47:07.866 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_index_type: 0 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_shortening: 1 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: checksum: 4 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: no_block_cache: 0 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache: 0x55d649569350 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_options: 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: capacity : 536870912 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_shard_bits : 4 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: strict_capacity_limit : 0 2024-09-16T08:47:07.867 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_compressed: (nil) 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: persistent_cache: (nil) 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size: 4096 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size_deviation: 10 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_restart_interval: 16 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_block_restart_interval: 1 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_block_size: 4096 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: partition_filters: 0 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: use_delta_encoding: 1 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: filter_policy: bloomfilter 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: whole_key_filtering: 1 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: verify_compression: 0 2024-09-16T08:47:07.868 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: format_version: 5 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_index_compression: 1 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_align: 0 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_auto_readahead_size: 262144 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: prepopulate_block_cache: 0 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression: NoCompression 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:47:07.869 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.num_levels: 7 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:47:07.870 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:47:07.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:47:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:47:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:47:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.ttl: 2592000 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:47:07.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T08:47:07.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3312a79e-384d-4a0c-9aaf-cd5b2e0c684c 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476427799827, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476427801295, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 87135, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 282, "table_properties": {"data_size": 85190, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11738, "raw_average_key_size": 48, "raw_value_size": 78660, "raw_average_value_size": 326, "num_data_blocks": 11, "num_entries": 241, "num_filter_entries": 241, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726476427, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3312a79e-384d-4a0c-9aaf-cd5b2e0c684c", "db_session_id": "TROB5XIQ4NHO4PB42FPF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476427801429, "job": 1, "event": "recovery_finished"} 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi031/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d649594e00 2024-09-16T08:47:07.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: DB pointer 0x55d649688000 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** DB Stats ** 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:47:07.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L0 2/0 86.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 92.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Sum 2/0 86.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 92.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 92.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 92.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:47:07.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative compaction: 0.00 GB write, 18.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval compaction: 0.00 GB write, 18.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache BinnedLRUCache@0x55d649569350#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-16T08:47:07.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:47:07.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:47:07.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: starting mon.smithi031 rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi031 fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:07.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???) e1 preinit fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:07.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???).mds e1 new map 2024-09-16T08:47:07.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???).mds e1 print_map 2024-09-16T08:47:07.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: e1 2024-09-16T08:47:07.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: legacy client fscid: -1 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: No filesystems configured 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T08:47:07.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 bash[27825]: 31533db037fd5c9ce47a7df4f0d299b33538bf4d078523f2824b3c5d8a917e52 2024-09-16T08:47:07.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 systemd[1]: Started Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:47:07.883 INFO:teuthology.orchestra.run.smithi031.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T08:47:08.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mon.smithi031 is new leader, mons smithi031 in quorum (ranks 0) 2024-09-16T08:47:08.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: monmap e1: 1 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:47:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: fsmap 2024-09-16T08:47:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T08:47:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:07 smithi031 ceph-mon[27844]: mgrmap e1: no daemons active 2024-09-16T08:47:08.972 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T08:47:08.993 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:47:08.993 INFO:teuthology.orchestra.run.smithi031.stdout:Creating mgr... 2024-09-16T08:47:08.993 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T08:47:08.994 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T08:47:08.994 INFO:teuthology.orchestra.run.smithi031.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-16T08:47:09.244 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mgr.smithi031.dcmqpv 2024-09-16T08:47:09.245 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mgr.smithi031.dcmqpv.service: Unit ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mgr.smithi031.dcmqpv.service not loaded. 2024-09-16T08:47:09.400 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1631ad7a-7408-11ef-bceb-c7b262605968.target.wants/ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mgr.smithi031.dcmqpv.service → /etc/systemd/system/ceph-1631ad7a-7408-11ef-bceb-c7b262605968@.service. 2024-09-16T08:47:09.620 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:09 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1268920353' entity='client.admin' 2024-09-16T08:47:09.895 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:47:09.895 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T08:47:09.903 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:47:09.903 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T08:47:09.903 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T08:47:09.903 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T08:47:09.912 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:47:09.912 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T08:47:09.920 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:47:09.920 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T08:47:09.920 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T08:47:09.920 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-16T08:47:09.920 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr to start... 2024-09-16T08:47:09.920 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr... 2024-09-16T08:47:10.787 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:10 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1283403233' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T08:47:11.050 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "1631ad7a-7408-11ef-bceb-c7b262605968", 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-09-16T08:47:11.051 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:47:11.052 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:47:11.053 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:47:11.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:47:11.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:47:11.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:11.056 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:47:11.057 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:11.058 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:47:02.944653+0000", 2024-09-16T08:47:11.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:47:11.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:11.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:47:11.059 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:47:11.059 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (1/15)... 2024-09-16T08:47:13.732 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:13 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1086839308' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T08:47:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:47:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: mgrmap e2: smithi031.dcmqpv(active, starting, since 0.00282695s) 2024-09-16T08:47:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:47:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:47:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:47:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:47:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:47:15.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:47:15.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:47:15.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:15.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"}]: dispatch 2024-09-16T08:47:15.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:15.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:14 smithi031 ceph-mon[27844]: from='mgr.14100 172.21.15.31:0/745067713' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:17.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:16 smithi031 ceph-mon[27844]: mgrmap e3: smithi031.dcmqpv(active, since 1.0073s) 2024-09-16T08:47:17.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:17.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:47:17.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "1631ad7a-7408-11ef-bceb-c7b262605968", 2024-09-16T08:47:17.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:47:17.347 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:17.348 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T08:47:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:47:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:47:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:47:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:47:17.349 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:47:17.352 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:17.353 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:47:17.354 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:47:02.944653+0000", 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:47:17.355 INFO:teuthology.orchestra.run.smithi031.stdout:mgr not available, waiting (2/15)... 2024-09-16T08:47:18.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:17 smithi031 ceph-mon[27844]: mgrmap e4: smithi031.dcmqpv(active, since 2s) 2024-09-16T08:47:20.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:20 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/428938332' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T08:47:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:47:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsid": "1631ad7a-7408-11ef-bceb-c7b262605968", 2024-09-16T08:47:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T08:47:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T08:47:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T08:47:20.504 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 0 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "smithi031" 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-16T08:47:20.505 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T08:47:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T08:47:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T08:47:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T08:47:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T08:47:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T08:47:20.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T08:47:20.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T08:47:20.509 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:20.510 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ], 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T08:47:20.511 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T08:47:20.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T08:47:02.944653+0000", 2024-09-16T08:47:20.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T08:47:20.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout }, 2024-09-16T08:47:20.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T08:47:20.512 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:47:20.512 INFO:teuthology.orchestra.run.smithi031.stdout:mgr is available 2024-09-16T08:47:21.309 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:21 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3118916057' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T08:47:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T08:47:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout fsid = 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T08:47:21.619 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T08:47:21.620 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling cephadm module... 2024-09-16T08:47:22.580 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:22 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2668018695' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T08:47:23.571 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:23 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2668018695' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T08:47:23.571 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:23 smithi031 ceph-mon[27844]: mgrmap e5: smithi031.dcmqpv(active, since 7s) 2024-09-16T08:47:25.781 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:25 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/4083223059' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T08:47:27.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: Active manager daemon smithi031.dcmqpv restarted 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: mgrmap e6: smithi031.dcmqpv(active, starting, since 0.0033783s) 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:27.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:47:27.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:47:27.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:47:27.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:27 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"}]: dispatch 2024-09-16T08:47:28.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:28 smithi031 ceph-mon[27844]: Found migration_current of "None". Setting to last migration. 2024-09-16T08:47:28.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:28 smithi031 ceph-mon[27844]: mgrmap e7: smithi031.dcmqpv(active, since 1.00553s) 2024-09-16T08:47:28.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:28 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:28.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:28 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:30.169 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:47:30.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T08:47:30.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:47:30.170 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "smithi031.dcmqpv", 2024-09-16T08:47:30.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T08:47:30.171 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:47:30.171 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-16T08:47:30.171 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 5... 2024-09-16T08:47:30.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:30 smithi031 ceph-mon[27844]: mgrmap e8: smithi031.dcmqpv(active, since 2s) 2024-09-16T08:47:30.418 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:30 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:47:31.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:47:31.234 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-16T08:47:31.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T08:47:31.235 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:47:31.235 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 5 is available 2024-09-16T08:47:31.235 INFO:teuthology.orchestra.run.smithi031.stdout:Setting orchestrator backend to cephadm... 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:31 smithi031 ceph-mon[27844]: [16/Sep/2024:08:47:29] ENGINE Bus STARTING 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:31 smithi031 ceph-mon[27844]: [16/Sep/2024:08:47:29] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:31 smithi031 ceph-mon[27844]: [16/Sep/2024:08:47:29] ENGINE Error in HTTPServer.serve 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Traceback (most recent call last): 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:47:31.485 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._run(expiration_interval) 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self.do_handshake() 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T08:47:31.486 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:47:31.487 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:31 smithi031 ceph-mon[27844]: [16/Sep/2024:08:47:29] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:47:31.487 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:31 smithi031 ceph-mon[27844]: [16/Sep/2024:08:47:29] ENGINE Bus STARTED 2024-09-16T08:47:32.551 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:32 smithi031 ceph-mon[27844]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T08:47:32.551 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:32 smithi031 ceph-mon[27844]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T08:47:32.551 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:32 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:32.551 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:32 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:47:33.537 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T08:47:33.537 INFO:teuthology.orchestra.run.smithi031.stdout:Generating ssh key... 2024-09-16T08:47:33.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:33 smithi031 ceph-mon[27844]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:34.514 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:34 smithi031 ceph-mon[27844]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:34.514 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:34 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:34.514 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:34 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:35.345 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:35 smithi031 ceph-mon[27844]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:35.345 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:35 smithi031 ceph-mon[27844]: Generating ssh key... 2024-09-16T08:47:35.916 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC026jKJYfQXrUmWbfOms+khqE37ZCaNl8SUsUMfsJcFg0tkr34GxQpZMLJr+y+ifCyFSz1bybkGl5PuZLnB/FYC3DHWnUbDDIPZBaXCeFxUQNBIw+XVQJuC/ckJbk9EfjNEffIbPWcLquY2J1WvWFsgtHI5APut/oEbZ3VK6WyDPAA1G5U3o102ndE3gqkUiwuCZIZuRbVum1H/0kAPF5NvThjnY5JY3TevAPYQBK2cW1WHk+kosGKj+1zEmoDA+PMy0RiqbrUCG6gIehqcWJvL2x76pptwO8uhCF1QVTA5PeGJBkiiREjkN/NY6JixDCVJ7x4ggpoKmtulT+3bbLeBB8BccfHi0rs8TNYDvXQtWv4GaDdvnelK+oA4nK/2vA49CHPOvTteqSbujSaVNWDeGBxdb4Hh67xNZ/OMdkjDvQS8b+hynIwqGKKiPNh9nE390HU0wcBqs0h4PoIDv3AC+kQ6OX9wV+0yoYRMMItSOaMMPT8QPDN8F9P+V5mgwM= ceph-1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:47:35.916 INFO:teuthology.orchestra.run.smithi031.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T08:47:35.916 INFO:teuthology.orchestra.run.smithi031.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T08:47:35.922 INFO:teuthology.orchestra.run.smithi031.stdout:Adding host smithi031... 2024-09-16T08:47:36.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:36 smithi031 ceph-mon[27844]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:37.531 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:37 smithi031 ceph-mon[27844]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi031", "addr": "172.21.15.31", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:38.453 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:38 smithi031 ceph-mon[27844]: Deploying cephadm binary to smithi031 2024-09-16T08:47:40.456 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Added host 'smithi031' with addr '172.21.15.31' 2024-09-16T08:47:40.456 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying mon service with default placement... 2024-09-16T08:47:41.035 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:40 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:41.036 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:40 smithi031 ceph-mon[27844]: Added host smithi031 2024-09-16T08:47:41.036 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:40 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:47:41.974 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T08:47:41.974 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying mgr service with default placement... 2024-09-16T08:47:42.223 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:42 smithi031 ceph-mon[27844]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:42.223 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:42 smithi031 ceph-mon[27844]: Saving service mon spec with placement count:5 2024-09-16T08:47:42.223 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:42 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:43.499 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T08:47:43.499 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying crash service with default placement... 2024-09-16T08:47:43.791 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:43 smithi031 ceph-mon[27844]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:43.791 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:43 smithi031 ceph-mon[27844]: Saving service mgr spec with placement count:2 2024-09-16T08:47:43.791 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:43 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:43.791 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:43 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:43.791 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:43 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:44.664 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-16T08:47:44.665 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying ceph-exporter service with default placement... 2024-09-16T08:47:45.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:45 smithi031 ceph-mon[27844]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:45.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:45 smithi031 ceph-mon[27844]: Saving service crash spec with placement * 2024-09-16T08:47:45.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:45 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:46.189 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-09-16T08:47:46.189 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying prometheus service with default placement... 2024-09-16T08:47:46.454 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:46 smithi031 ceph-mon[27844]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:46.454 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:46 smithi031 ceph-mon[27844]: Saving service ceph-exporter spec with placement * 2024-09-16T08:47:46.454 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:46 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:46.454 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:46 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:46.454 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:46 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:47.413 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-16T08:47:47.414 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying grafana service with default placement... 2024-09-16T08:47:47.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:47 smithi031 ceph-mon[27844]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:47.947 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:47 smithi031 ceph-mon[27844]: Saving service prometheus spec with placement count:1 2024-09-16T08:47:47.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:47 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:47.948 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:47 smithi031 ceph-mon[27844]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:47:48.838 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-16T08:47:48.838 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying node-exporter service with default placement... 2024-09-16T08:47:49.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:49 smithi031 ceph-mon[27844]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:49.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:49 smithi031 ceph-mon[27844]: Saving service grafana spec with placement count:1 2024-09-16T08:47:49.086 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:49 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:50.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:50 smithi031 ceph-mon[27844]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:47:50.253 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:50 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:50.254 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-16T08:47:50.255 INFO:teuthology.orchestra.run.smithi031.stdout:Deploying alertmanager service with default placement... 2024-09-16T08:47:51.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:51 smithi031 ceph-mon[27844]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:51 smithi031 ceph-mon[27844]: Saving service node-exporter spec with placement * 2024-09-16T08:47:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:51 smithi031 ceph-mon[27844]: from='mgr.14118 172.21.15.31:0/3587505850' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:47:51.418 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-16T08:47:52.310 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:52 smithi031 ceph-mon[27844]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:47:52.311 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:52 smithi031 ceph-mon[27844]: Saving service alertmanager spec with placement count:1 2024-09-16T08:47:52.311 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:52 smithi031 ceph-mon[27844]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:47:52.311 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:52 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/670293885' entity='client.admin' 2024-09-16T08:47:53.713 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling the dashboard module... 2024-09-16T08:47:54.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:54 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1005858874' entity='client.admin' 2024-09-16T08:47:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:54 smithi031 ceph-mon[27844]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:47:55.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:55 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3837741937' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T08:47:56.344 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:56 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3837741937' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T08:47:56.345 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:56 smithi031 ceph-mon[27844]: mgrmap e9: smithi031.dcmqpv(active, since 27s) 2024-09-16T08:47:56.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:47:56.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T08:47:56.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T08:47:56.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "active_name": "smithi031.dcmqpv", 2024-09-16T08:47:56.992 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T08:47:56.993 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:47:56.993 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for the mgr to restart... 2024-09-16T08:47:56.993 INFO:teuthology.orchestra.run.smithi031.stdout:Waiting for mgr epoch 9... 2024-09-16T08:47:57.240 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:47:57 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3779513003' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T08:48:00.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: Active manager daemon smithi031.dcmqpv restarted 2024-09-16T08:48:00.384 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: mgrmap e10: smithi031.dcmqpv(active, starting, since 0.00568484s) 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:48:00.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:00 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"}]: dispatch 2024-09-16T08:48:01.942 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout { 2024-09-16T08:48:01.942 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T08:48:01.942 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T08:48:01.942 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout } 2024-09-16T08:48:01.942 INFO:teuthology.orchestra.run.smithi031.stdout:mgr epoch 9 is available 2024-09-16T08:48:01.942 INFO:teuthology.orchestra.run.smithi031.stdout:Generating a dashboard self-signed certificate... 2024-09-16T08:48:02.189 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:02 smithi031 ceph-mon[27844]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T08:48:02.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:02 smithi031 ceph-mon[27844]: mgrmap e11: smithi031.dcmqpv(active, since 1.00578s) 2024-09-16T08:48:02.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:02 smithi031 ceph-mon[27844]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T08:48:02.190 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:02 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: [16/Sep/2024:08:48:02] ENGINE Bus STARTING 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: [16/Sep/2024:08:48:02] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: [16/Sep/2024:08:48:02] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: [16/Sep/2024:08:48:02] ENGINE Bus STARTED 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: [16/Sep/2024:08:48:02] ENGINE Error in HTTPServer.serve 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Traceback (most recent call last): 2024-09-16T08:48:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._run(expiration_interval) 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:48:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self.do_handshake() 2024-09-16T08:48:03.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:48:03.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T08:48:03.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:48:03.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:48:03.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: mgrmap e12: smithi031.dcmqpv(active, since 2s) 2024-09-16T08:48:03.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:03.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:03 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:04.088 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T08:48:04.088 INFO:teuthology.orchestra.run.smithi031.stdout:Creating initial admin user... 2024-09-16T08:48:05.644 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$8rILt7PpME8V1l2niP05D.9wYP8cf3jlYnQzLHd5qFxZN6FnrcfjC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726476485, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T08:48:05.644 INFO:teuthology.orchestra.run.smithi031.stdout:Fetching dashboard port number... 2024-09-16T08:48:06.210 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:06 smithi031 ceph-mon[27844]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:48:06.210 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:06 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:06.766 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T08:48:06.777 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T08:48:06.777 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout disabled 2024-09-16T08:48:06.785 INFO:teuthology.orchestra.run.smithi031.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T08:48:06.785 INFO:teuthology.orchestra.run.smithi031.stdout:systemctl: stdout inactive 2024-09-16T08:48:06.785 INFO:teuthology.orchestra.run.smithi031.stdout:firewalld.service is not enabled 2024-09-16T08:48:06.785 INFO:teuthology.orchestra.run.smithi031.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T08:48:06.786 INFO:teuthology.orchestra.run.smithi031.stdout:Ceph Dashboard is now available at: 2024-09-16T08:48:06.786 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:06.786 INFO:teuthology.orchestra.run.smithi031.stdout: URL: https://smithi031.front.sepia.ceph.com:8443/ 2024-09-16T08:48:06.786 INFO:teuthology.orchestra.run.smithi031.stdout: User: admin 2024-09-16T08:48:06.787 INFO:teuthology.orchestra.run.smithi031.stdout: Password: hp2lu395bw 2024-09-16T08:48:06.787 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:06.787 INFO:teuthology.orchestra.run.smithi031.stdout:Saving cluster configuration to /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config directory 2024-09-16T08:48:06.787 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling autotune for osd_memory_target 2024-09-16T08:48:07.083 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:07 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2008837894' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T08:48:08.156 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:08 smithi031 ceph-mon[27844]: mgrmap e13: smithi031.dcmqpv(active, since 6s) 2024-09-16T08:48:09.245 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T08:48:09.245 INFO:teuthology.orchestra.run.smithi031.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T08:48:09.245 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.245 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T08:48:09.245 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: ceph telemetry on 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout:For more information see: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:09.246 INFO:teuthology.orchestra.run.smithi031.stdout:Bootstrap complete. 2024-09-16T08:48:09.278 INFO:tasks.cephadm:Fetching config... 2024-09-16T08:48:09.278 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:48:09.279 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T08:48:09.295 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T08:48:09.295 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:48:09.295 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T08:48:09.352 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T08:48:09.352 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:48:09.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/keyring of=/dev/stdout 2024-09-16T08:48:09.419 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T08:48:09.419 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:48:09.419 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T08:48:09.475 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T08:48:09.476 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC026jKJYfQXrUmWbfOms+khqE37ZCaNl8SUsUMfsJcFg0tkr34GxQpZMLJr+y+ifCyFSz1bybkGl5PuZLnB/FYC3DHWnUbDDIPZBaXCeFxUQNBIw+XVQJuC/ckJbk9EfjNEffIbPWcLquY2J1WvWFsgtHI5APut/oEbZ3VK6WyDPAA1G5U3o102ndE3gqkUiwuCZIZuRbVum1H/0kAPF5NvThjnY5JY3TevAPYQBK2cW1WHk+kosGKj+1zEmoDA+PMy0RiqbrUCG6gIehqcWJvL2x76pptwO8uhCF1QVTA5PeGJBkiiREjkN/NY6JixDCVJ7x4ggpoKmtulT+3bbLeBB8BccfHi0rs8TNYDvXQtWv4GaDdvnelK+oA4nK/2vA49CHPOvTteqSbujSaVNWDeGBxdb4Hh67xNZ/OMdkjDvQS8b+hynIwqGKKiPNh9nE390HU0wcBqs0h4PoIDv3AC+kQ6OX9wV+0yoYRMMItSOaMMPT8QPDN8F9P+V5mgwM= ceph-1631ad7a-7408-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T08:48:09.550 INFO:teuthology.orchestra.run.smithi031.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC026jKJYfQXrUmWbfOms+khqE37ZCaNl8SUsUMfsJcFg0tkr34GxQpZMLJr+y+ifCyFSz1bybkGl5PuZLnB/FYC3DHWnUbDDIPZBaXCeFxUQNBIw+XVQJuC/ckJbk9EfjNEffIbPWcLquY2J1WvWFsgtHI5APut/oEbZ3VK6WyDPAA1G5U3o102ndE3gqkUiwuCZIZuRbVum1H/0kAPF5NvThjnY5JY3TevAPYQBK2cW1WHk+kosGKj+1zEmoDA+PMy0RiqbrUCG6gIehqcWJvL2x76pptwO8uhCF1QVTA5PeGJBkiiREjkN/NY6JixDCVJ7x4ggpoKmtulT+3bbLeBB8BccfHi0rs8TNYDvXQtWv4GaDdvnelK+oA4nK/2vA49CHPOvTteqSbujSaVNWDeGBxdb4Hh67xNZ/OMdkjDvQS8b+hynIwqGKKiPNh9nE390HU0wcBqs0h4PoIDv3AC+kQ6OX9wV+0yoYRMMItSOaMMPT8QPDN8F9P+V5mgwM= ceph-1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:48:09.562 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC026jKJYfQXrUmWbfOms+khqE37ZCaNl8SUsUMfsJcFg0tkr34GxQpZMLJr+y+ifCyFSz1bybkGl5PuZLnB/FYC3DHWnUbDDIPZBaXCeFxUQNBIw+XVQJuC/ckJbk9EfjNEffIbPWcLquY2J1WvWFsgtHI5APut/oEbZ3VK6WyDPAA1G5U3o102ndE3gqkUiwuCZIZuRbVum1H/0kAPF5NvThjnY5JY3TevAPYQBK2cW1WHk+kosGKj+1zEmoDA+PMy0RiqbrUCG6gIehqcWJvL2x76pptwO8uhCF1QVTA5PeGJBkiiREjkN/NY6JixDCVJ7x4ggpoKmtulT+3bbLeBB8BccfHi0rs8TNYDvXQtWv4GaDdvnelK+oA4nK/2vA49CHPOvTteqSbujSaVNWDeGBxdb4Hh67xNZ/OMdkjDvQS8b+hynIwqGKKiPNh9nE390HU0wcBqs0h4PoIDv3AC+kQ6OX9wV+0yoYRMMItSOaMMPT8QPDN8F9P+V5mgwM= ceph-1631ad7a-7408-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T08:48:09.621 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC026jKJYfQXrUmWbfOms+khqE37ZCaNl8SUsUMfsJcFg0tkr34GxQpZMLJr+y+ifCyFSz1bybkGl5PuZLnB/FYC3DHWnUbDDIPZBaXCeFxUQNBIw+XVQJuC/ckJbk9EfjNEffIbPWcLquY2J1WvWFsgtHI5APut/oEbZ3VK6WyDPAA1G5U3o102ndE3gqkUiwuCZIZuRbVum1H/0kAPF5NvThjnY5JY3TevAPYQBK2cW1WHk+kosGKj+1zEmoDA+PMy0RiqbrUCG6gIehqcWJvL2x76pptwO8uhCF1QVTA5PeGJBkiiREjkN/NY6JixDCVJ7x4ggpoKmtulT+3bbLeBB8BccfHi0rs8TNYDvXQtWv4GaDdvnelK+oA4nK/2vA49CHPOvTteqSbujSaVNWDeGBxdb4Hh67xNZ/OMdkjDvQS8b+hynIwqGKKiPNh9nE390HU0wcBqs0h4PoIDv3AC+kQ6OX9wV+0yoYRMMItSOaMMPT8QPDN8F9P+V5mgwM= ceph-1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:48:09.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T08:48:09.745 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:09 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1923957277' entity='client.admin' 2024-09-16T08:48:09.938 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:48:13.860 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3322356888' entity='client.admin' 2024-09-16T08:48:13.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:13.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:13.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:48:13.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:13.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:48:13.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T08:48:13.861 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:13 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:48:13.882 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T08:48:13.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T08:48:14.929 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:48:14.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:14 smithi031 ceph-mon[27844]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-09-16T08:48:17.690 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2024-09-16T08:48:17.690 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:48:17.690 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2024-09-16T08:48:17.710 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:48:17.710 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:48:17.769 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2024-09-16T08:48:17.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch host add smithi097 2024-09-16T08:48:17.794 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:17.794 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:17.794 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:17.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:17.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:48:17.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T08:48:17.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:48:17.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: Deploying daemon crash.smithi031 on smithi031 2024-09-16T08:48:17.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='client.14187 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:48:17.795 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:17 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:18.624 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:48:21.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:21 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:21.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:21 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:21.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:21 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:21.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:21 smithi031 ceph-mon[27844]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:21.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:21 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:21.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:21 smithi031 ceph-mon[27844]: Deploying daemon node-exporter.smithi031 on smithi031 2024-09-16T08:48:21.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:21 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:22.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:22 smithi031 ceph-mon[27844]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:48:23.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:23 smithi031 ceph-mon[27844]: Deploying cephadm binary to smithi097 2024-09-16T08:48:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:23 smithi031 ceph-mon[27844]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:24.109 INFO:teuthology.orchestra.run.smithi031.stdout:Added host 'smithi097' with addr '172.21.15.97' 2024-09-16T08:48:24.678 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T08:48:24.924 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: Deploying daemon alertmanager.smithi031 on smithi031 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: Added host smithi097 2024-09-16T08:48:25.046 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:24 smithi031 ceph-mon[27844]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:26.334 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:26 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:27.469 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:27 smithi031 ceph-mon[27844]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:27.599 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:48:27.599 INFO:teuthology.orchestra.run.smithi031.stdout:[{"addr": "172.21.15.31", "hostname": "smithi031", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2024-09-16T08:48:29.097 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T08:48:29.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T08:48:29.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:29 smithi031 ceph-mon[27844]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:48:29.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:29 smithi031 ceph-mon[27844]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:29.504 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:48:31.315 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:31 smithi031 ceph-mon[27844]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:33.252 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:33 smithi031 ceph-mon[27844]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:34.184 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-09-16T08:48:34.451 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:34 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3067794228' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T08:48:35.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:35.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3067794228' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T08:48:35.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:48:35.264 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.265 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:35 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:35.287 INFO:tasks.cephadm:Adding mon.smithi031 on smithi031 2024-09-16T08:48:35.287 INFO:tasks.cephadm:Adding mon.smithi097 on smithi097 2024-09-16T08:48:35.287 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch apply mon '2;smithi031:172.21.15.31=smithi031;smithi097:172.21.15.97=smithi097' 2024-09-16T08:48:35.557 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:35.602 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:36.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:36 smithi031 ceph-mon[27844]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:48:36.472 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:36 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:48:36.472 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:36 smithi031 ceph-mon[27844]: Deploying daemon grafana.smithi031 on smithi031 2024-09-16T08:48:37.335 INFO:teuthology.orchestra.run.smithi097.stdout:Scheduled mon update... 2024-09-16T08:48:37.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:37 smithi031 ceph-mon[27844]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:37.902 DEBUG:teuthology.orchestra.run.smithi097:mon.smithi097> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi097.service 2024-09-16T08:48:37.904 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:48:37.904 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:48:38.126 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:38.170 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:38.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:38 smithi031 ceph-mon[27844]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi097:172.21.15.97=smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:48:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:38 smithi031 ceph-mon[27844]: Saving service mon spec with placement smithi031:172.21.15.31=smithi031;smithi097:172.21.15.97=smithi097;count:2 2024-09-16T08:48:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:38 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:38 smithi031 ceph-mon[27844]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:39.699 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:48:39.699 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:48:39.699 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:48:40.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:39 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/2373557747' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:48:41.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:40 smithi031 ceph-mon[27844]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:41.254 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:48:41.255 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:48:41.454 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:41.496 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:43.208 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:48:43.208 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:48:43.208 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:48:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:43 smithi031 ceph-mon[27844]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:44.474 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:44 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/770832355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:48:44.760 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:48:44.760 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:48:44.957 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:45.002 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:45.237 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:45 smithi031 ceph-mon[27844]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:46.674 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:48:46.674 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:48:46.675 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:48:47.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.306 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.307 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.307 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.307 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.307 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:47.307 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:47.307 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:46 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/179983357' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:48:48.246 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:48:48.246 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:48:48.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:47 smithi031 ceph-mon[27844]: Deploying daemon prometheus.smithi031 on smithi031 2024-09-16T08:48:48.438 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:48.493 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:48 smithi031 ceph-mon[27844]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:50.099 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:48:50.100 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:48:50.100 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:48:50.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:50 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/375195107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:48:51.523 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:51 smithi031 ceph-mon[27844]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:51.524 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:51 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:48:51.676 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:48:51.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:48:51.868 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:51.913 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:53.456 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:48:53.456 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:48:53.456 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:48:53.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:53 smithi031 ceph-mon[27844]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:54.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:54 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/4168559965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:48:54.970 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:48:54.970 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:48:55.165 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:55.210 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:55.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:55 smithi031 ceph-mon[27844]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:56.833 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:48:56.833 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:48:56.834 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:48:57.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:57 smithi031 ceph-mon[27844]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:48:57.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:57 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/1700442713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:48:58.399 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:48:58.400 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:48:58.596 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:58.641 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:48:59.472 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:48:59 smithi031 ceph-mon[27844]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:00.127 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:00.127 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:00.128 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:00 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/185738052' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:01.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:01 smithi031 ceph-mon[27844]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:01.754 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:01.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:01.959 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:02.004 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:03 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:03 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:03 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:03 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T08:49:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:03 smithi031 ceph-mon[27844]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:03.617 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:03.617 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:03.618 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:04.347 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:04 smithi031 ceph-mon[27844]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:49:04.347 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:04 smithi031 ceph-mon[27844]: mgrmap e14: smithi031.dcmqpv(active, since 62s) 2024-09-16T08:49:04.347 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:04 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/2373837153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:05.172 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:05.172 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:05.367 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:05.412 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:06.897 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:06.898 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:06.898 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:07.189 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:06 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/2320958270' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:08.299 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: Active manager daemon smithi031.dcmqpv restarted 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: mgrmap e15: smithi031.dcmqpv(active, starting, since 0.00293234s) 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:49:08.300 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:49:08.301 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:08.301 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:08.468 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:08.468 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:08.661 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:08.707 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:49:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"}]: dispatch 2024-09-16T08:49:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:09 smithi031 ceph-mon[27844]: mgrmap e16: smithi031.dcmqpv(active, since 1.0059s) 2024-09-16T08:49:10.282 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:10.283 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:10.283 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:11 smithi031 ceph-mon[27844]: mgrmap e17: smithi031.dcmqpv(active, since 2s) 2024-09-16T08:49:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:11 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:11 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/2446472085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:11.853 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:11.853 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:12.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:12 smithi031 ceph-mon[27844]: [16/Sep/2024:08:49:10] ENGINE Bus STARTING 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:12 smithi031 ceph-mon[27844]: [16/Sep/2024:08:49:10] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:12 smithi031 ceph-mon[27844]: [16/Sep/2024:08:49:10] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:12 smithi031 ceph-mon[27844]: [16/Sep/2024:08:49:10] ENGINE Bus STARTED 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:12 smithi031 ceph-mon[27844]: [16/Sep/2024:08:49:10] ENGINE Error in HTTPServer.serve 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Traceback (most recent call last): 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._run(expiration_interval) 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:49:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self.do_handshake() 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:49:12.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:12.385 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:12.431 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T08:49:13.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:13 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:13.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:13 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:13.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:13 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:13.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:13 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:13.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:13 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:49:15.183 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:15.183 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:15.184 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:49:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/3686843056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:49:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T08:49:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:16.718 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:16 smithi031 ceph-mon[27844]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:49:16.718 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:16 smithi031 ceph-mon[27844]: Deploying daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:49:16.758 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:16.758 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:16.962 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:18.749 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:18.749 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:18.749 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:19.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:18 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/2531045266' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:20.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:20 smithi031 ceph-mon[27844]: Deploying daemon crash.smithi097 on smithi097 2024-09-16T08:49:20.715 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:20.715 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:20.931 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:22.927 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:22.928 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:22.928 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:24.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:24.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:24.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:24.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:24.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: Deploying daemon node-exporter.smithi097 on smithi097 2024-09-16T08:49:24.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/3899013324' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:24.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:24.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:24.607 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:24.607 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:24.808 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:27.901 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:27.902 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:27.902 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: Deploying daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:49:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/1048356789' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:30.374 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:30.374 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:30.950 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:31 smithi031 ceph-mon[27844]: Deploying daemon mon.smithi097 on smithi097 2024-09-16T08:49:33.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:33 smithi031 ceph-mon[27844]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:33.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:33 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:33.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:33 smithi097 systemd[1]: Starting Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:49:34.366 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 podman[35446]: 2024-09-16 08:49:34.272963182 +0000 UTC m=+0.114254350 container create a26ddb2c052373921d65bcd9579d2820a30a2e29adb28b5547809bd0c96ab79d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=reef-ae7f373) 2024-09-16T08:49:34.366 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 podman[35446]: 2024-09-16 08:49:34.182154931 +0000 UTC m=+0.023446103 image pull fd1d6ca2b4a51b7dd04a60817dc43aeea69da272641c8151cf0fecceea98aa95 quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87 2024-09-16T08:49:34.382 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:34.382 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":1,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:47:00.018794Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T08:49:34.382 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 1 2024-09-16T08:49:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 podman[35446]: 2024-09-16 08:49:34.483984631 +0000 UTC m=+0.325275810 container init a26ddb2c052373921d65bcd9579d2820a30a2e29adb28b5547809bd0c96ab79d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=reef-ae7f373, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T08:49:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 podman[35446]: 2024-09-16 08:49:34.487356629 +0000 UTC m=+0.328647797 container start a26ddb2c052373921d65bcd9579d2820a30a2e29adb28b5547809bd0c96ab79d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:49:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:49:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable), process ceph-mon, pid 2 2024-09-16T08:49:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pidfile_write: ignore empty --pid-file 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: load: jerasure load: lrc 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Git sha 0 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Compile date 2024-09-15 20:36:04 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: DB SUMMARY 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: DB Session ID: MDR14CRLDTWVT2VTWWUH 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi097/store.db dir, Total Num: 0, files: 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi097/store.db: 000004.log size: 511 ; 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:49:34.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.env: 0x56372998abc0 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.info_log: 0x56372bd16dc0 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:49:34.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.statistics: (nil) 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.use_fsync: 0 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:49:34.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.db_log_dir: 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.wal_dir: 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:49:34.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.write_buffer_manager: 0x56372bd24320 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:49:34.769 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.unordered_write: 0 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.row_cache: None 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.wal_filter: None 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.wal_compression: 0 2024-09-16T08:49:34.770 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:49:34.771 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:49:34.772 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_open_files: -1 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Compression algorithms supported: 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kZSTD supported: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:49:34.773 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi097/store.db/MANIFEST-000005 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.merge_operator: 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_filter: None 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:49:34.774 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56372bd16a20) 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: index_type: 0 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: data_block_index_type: 0 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: index_shortening: 1 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: checksum: 4 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: no_block_cache: 0 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache: 0x56372bd0b350 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:49:34.775 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache_options: 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: capacity : 536870912 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: num_shard_bits : 4 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: strict_capacity_limit : 0 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache_compressed: (nil) 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: persistent_cache: (nil) 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_size: 4096 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_size_deviation: 10 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_restart_interval: 16 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: index_block_restart_interval: 1 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: metadata_block_size: 4096 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: partition_filters: 0 2024-09-16T08:49:34.776 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: use_delta_encoding: 1 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: filter_policy: bloomfilter 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: whole_key_filtering: 1 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: verify_compression: 0 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: format_version: 5 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: enable_index_compression: 1 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_align: 0 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: max_auto_readahead_size: 262144 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: prepopulate_block_cache: 0 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:49:34.777 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression: NoCompression 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.num_levels: 7 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:49:34.778 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:49:34.779 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:49:34.780 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:49:34.781 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:49:34.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.ttl: 2592000 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:49:34.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi097/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 450393fc-fda0-48b4-8d44-55dd7896794d 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476574514011, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476574514406, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726476574, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "450393fc-fda0-48b4-8d44-55dd7896794d", "db_session_id": "MDR14CRLDTWVT2VTWWUH", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726476574514499, "job": 1, "event": "recovery_finished"} 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi097/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56372bd36e00 2024-09-16T08:49:34.784 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: DB pointer 0x56372be2a000 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** DB Stats ** 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** Compaction Stats [default] ** 2024-09-16T08:49:34.785 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** Compaction Stats [default] ** 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:49:34.786 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Block cache BinnedLRUCache@0x56372bd0b350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097 does not exist in monmap, will attempt to join an existing cluster 2024-09-16T08:49:34.787 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: using public_addr v2:172.21.15.97:0/0 -> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: starting mon.smithi097 rank -1 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi097 fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(???) e0 preinit fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).mds e1 new map 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).mds e1 print_map 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: e1 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: legacy client fscid: -1 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: No filesystems configured 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T08:49:34.788 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.789 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon crash.smithi031 on smithi031 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.14187 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon node-exporter.smithi031 on smithi031 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:49:34.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying cephadm binary to smithi097 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon alertmanager.smithi031 on smithi031 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Added host smithi097 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:49:34.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3067794228' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3067794228' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon grafana.smithi031 on smithi031 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.14195 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi031:172.21.15.31=smithi031;smithi097:172.21.15.97=smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Saving service mon spec with placement smithi031:172.21.15.31=smithi031;smithi097:172.21.15.97=smithi097;count:2 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/2373557747' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.793 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/770832355' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.794 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/179983357' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon prometheus.smithi031 on smithi031 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/375195107' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/4168559965' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/1700442713' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/185738052' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.795 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14162 172.21.15.31:0/3106697772' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mgrmap e14: smithi031.dcmqpv(active, since 62s) 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/2373837153' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/2320958270' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Active manager daemon smithi031.dcmqpv restarted 2024-09-16T08:49:34.796 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mgrmap e15: smithi031.dcmqpv(active, starting, since 0.00293234s) 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:49:34.797 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"}]: dispatch 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mgrmap e16: smithi031.dcmqpv(active, since 1.0059s) 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mgrmap e17: smithi031.dcmqpv(active, since 2s) 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/2446472085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: [16/Sep/2024:08:49:10] ENGINE Bus STARTING 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: [16/Sep/2024:08:49:10] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: [16/Sep/2024:08:49:10] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: [16/Sep/2024:08:49:10] ENGINE Bus STARTED 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: [16/Sep/2024:08:49:10] ENGINE Error in HTTPServer.serve 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Traceback (most recent call last): 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:49:34.798 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._run(expiration_interval) 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s = self.context.wrap_socket( 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: return self.sslsocket_class._create( 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self.do_handshake() 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:49:34.799 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._sslobj.do_handshake() 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi031", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:49:34.800 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/3686843056' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.801 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/2531045266' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-16T08:49:34.802 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon crash.smithi097 on smithi097 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon node-exporter.smithi097 on smithi097 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/3899013324' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.803 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:49:34.804 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/1048356789' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:34.805 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: Deploying daemon mon.smithi097 on smithi097 2024-09-16T08:49:34.806 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:34.806 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:34.806 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 ceph-mon[35478]: mon.smithi097@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-09-16T08:49:34.806 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 bash[35446]: a26ddb2c052373921d65bcd9579d2820a30a2e29adb28b5547809bd0c96ab79d 2024-09-16T08:49:34.806 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:34 smithi097 systemd[1]: Started Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:49:36.268 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-16T08:49:36.268 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T08:49:36.474 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:49:39.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:49:39.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: mon.smithi031 calling monitor election 2024-09-16T08:49:39.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:49:39.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.790 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: mon.smithi097 calling monitor election 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: mon.smithi031 is new leader, mons smithi031,smithi097 in quorum (ranks 0,1) 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: monmap e2: 2 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],smithi097=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: fsmap 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T08:49:39.791 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: mgrmap e17: smithi031.dcmqpv(active, since 31s) 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: overall HEALTH_OK 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.792 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: mon.smithi031 calling monitor election 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: mon.smithi097 calling monitor election 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: mon.smithi031 is new leader, mons smithi031,smithi097 in quorum (ranks 0,1) 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: monmap e2: 2 mons at {smithi031=[v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0],smithi097=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: fsmap 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: osdmap e5: 0 total, 0 up, 0 in 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: mgrmap e17: smithi031.dcmqpv(active, since 31s) 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: overall HEALTH_OK 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:49:39.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:49:39.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:49:39.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/3638897437' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:49:39.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:39.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:40.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:40 smithi031 ceph-mon[27844]: mgrmap e18: smithi031.dcmqpv(active, since 31s), standbys: smithi097.mpashf 2024-09-16T08:49:40.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:40 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"}]: dispatch 2024-09-16T08:49:40.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:40 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:49:40.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:40 smithi031 ceph-mon[27844]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:40.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:40 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:41.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:40 smithi097 ceph-mon[35478]: mgrmap e18: smithi031.dcmqpv(active, since 31s), standbys: smithi097.mpashf 2024-09-16T08:49:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:40 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"}]: dispatch 2024-09-16T08:49:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:40 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:49:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:40 smithi097 ceph-mon[35478]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:40 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:49:41.189 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:49:41.189 INFO:teuthology.orchestra.run.smithi097.stdout:{"epoch":2,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","modified":"2024-09-16T08:49:34.544973Z","created":"2024-09-16T08:47:00.018794Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi031","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi097","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T08:49:41.189 INFO:teuthology.orchestra.run.smithi097.stderr:dumped monmap epoch 2 2024-09-16T08:49:41.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:41 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/1073438797' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:41.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:41.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:41.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:41.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:49:42.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:41 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/1073438797' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T08:49:42.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:42.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:42.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:42.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:49:42.194 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T08:49:42.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T08:49:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:49:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:49:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:43.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:43.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: Reconfiguring mon.smithi031 (unknown last config time)... 2024-09-16T08:49:43.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:49:43.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:49:43.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:43.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:43 smithi097 ceph-mon[35478]: Reconfiguring daemon mon.smithi031 on smithi031 2024-09-16T08:49:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:49:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:49:43.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:43.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:49:43.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:43.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:43.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: Reconfiguring mon.smithi031 (unknown last config time)... 2024-09-16T08:49:43.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:49:43.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:49:43.376 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:43.376 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:43 smithi031 ceph-mon[27844]: Reconfiguring daemon mon.smithi031 on smithi031 2024-09-16T08:49:43.459 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:49:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:49:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:44.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:44.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:44.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:44.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:49:44.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:49:44.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:45.301 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:45 smithi031 ceph-mon[27844]: Reconfiguring mgr.smithi031.dcmqpv (unknown last config time)... 2024-09-16T08:49:45.301 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:45 smithi031 ceph-mon[27844]: Reconfiguring daemon mgr.smithi031.dcmqpv on smithi031 2024-09-16T08:49:45.301 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:45 smithi031 ceph-mon[27844]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:45.375 INFO:teuthology.orchestra.run.smithi031.stdout:# minimal ceph.conf for 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:49:45.375 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-16T08:49:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:49:45.376 INFO:teuthology.orchestra.run.smithi031.stdout: mon_host = [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2024-09-16T08:49:45.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:45 smithi097 ceph-mon[35478]: Reconfiguring mgr.smithi031.dcmqpv (unknown last config time)... 2024-09-16T08:49:45.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:45 smithi097 ceph-mon[35478]: Reconfiguring daemon mgr.smithi031.dcmqpv on smithi031 2024-09-16T08:49:45.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:45 smithi097 ceph-mon[35478]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:46.154 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:46 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1374407508' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:46.179 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T08:49:46.179 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:49:46.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T08:49:46.208 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:49:46.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:49:46.275 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:49:46.275 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T08:49:46.308 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:49:46.308 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:49:46.373 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:46 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1374407508' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:46.378 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-16T08:49:46.378 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-16T08:49:46.380 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-16T08:49:46.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set client debug_client 20 2024-09-16T08:49:46.423 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-16T08:49:46.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set client debug_ms 1 2024-09-16T08:49:46.466 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-16T08:49:46.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-16T08:49:46.510 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-16T08:49:46.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-16T08:49:46.553 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-16T08:49:46.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-16T08:49:46.597 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-16T08:49:46.598 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-16T08:49:46.642 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-16T08:49:46.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-16T08:49:46.689 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-16T08:49:46.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-16T08:49:46.735 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-16T08:49:46.735 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-16T08:49:46.781 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-16T08:49:46.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-16T08:49:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:46.827 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-16T08:49:46.828 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-16T08:49:46.843 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:46.855 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-16T08:49:46.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-16T08:49:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:46.876 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-16T08:49:46.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-16T08:49:46.920 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-16T08:49:46.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-16T08:49:46.969 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-16T08:49:46.970 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-16T08:49:47.016 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-16T08:49:47.016 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-16T08:49:47.061 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-16T08:49:47.061 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-16T08:49:47.111 DEBUG:tasks.cephadm:set 18 configs 2024-09-16T08:49:47.111 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi031') 2024-09-16T08:49:47.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:47 smithi031 ceph-mon[27844]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:47.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:47.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:47.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:49:47.236 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:47.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:47 smithi097 ceph-mon[35478]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:47.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:47.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:47.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:49:47.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:48 smithi031 ceph-mon[27844]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-09-16T08:49:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:48 smithi031 ceph-mon[27844]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-09-16T08:49:48.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:48 smithi097 ceph-mon[35478]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-09-16T08:49:48.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:48 smithi097 ceph-mon[35478]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-09-16T08:49:49.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:49 smithi031 ceph-mon[27844]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:49.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:49 smithi097 ceph-mon[35478]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:50.310 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:51.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:51 smithi097 ceph-mon[35478]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:51 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3069894217' entity='client.admin' 2024-09-16T08:49:51.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:51 smithi031 ceph-mon[27844]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:51 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3069894217' entity='client.admin' 2024-09-16T08:49:51.890 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi031') 2024-09-16T08:49:52.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:52 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2437066935' entity='client.admin' 2024-09-16T08:49:52.871 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:52 smithi031 ceph-mon[27844]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:52.884 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:53.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:52 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2437066935' entity='client.admin' 2024-09-16T08:49:53.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:52 smithi097 ceph-mon[35478]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:53.763 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:54.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:54.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:54.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:53 smithi031 ceph-mon[27844]: Reconfiguring crash.smithi031 (monmap changed)... 2024-09-16T08:49:54.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:49:54.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:54.120 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:53 smithi031 ceph-mon[27844]: Reconfiguring daemon crash.smithi031 on smithi031 2024-09-16T08:49:54.121 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:54.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:54.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:49:54.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:53 smithi097 ceph-mon[35478]: Reconfiguring crash.smithi031 (monmap changed)... 2024-09-16T08:49:54.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:49:54.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:49:54.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:53 smithi097 ceph-mon[35478]: Reconfiguring daemon crash.smithi031 on smithi031 2024-09-16T08:49:54.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:49:54.666 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:54.667 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:55.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:54 smithi097 ceph-mon[35478]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:55.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:54 smithi031 ceph-mon[27844]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:55.615 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:55.615 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:55.616 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:55.617 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:56.016 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi031') 2024-09-16T08:49:56.512 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:56.513 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:49:57.345 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:57 smithi031 ceph-mon[27844]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:57.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:57 smithi097 ceph-mon[35478]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:57.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.793+0000 7f107dffb640 1 -- 172.21.15.31:0/2633190270 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v13)=0 v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7f106c0200b0 con 0x7f10800ffca0 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 -- 172.21.15.31:0/2633190270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f107006cbd0 msgr2=0x7f107006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 --2- 172.21.15.31:0/2633190270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f107006cbd0 0x7f107006f090 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1074004ee0 tx=0x7f1074005d50 comp rx=0 tx=0).stop 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 -- 172.21.15.31:0/2633190270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10800ffca0 msgr2=0x7f10801958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 --2- 172.21.15.31:0/2633190270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10800ffca0 0x7f10801958c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f106c00d900 tx=0x7f106c00ddc0 comp rx=0 tx=0).stop 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 -- 172.21.15.31:0/2633190270 shutdown_connections 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 --2- 172.21.15.31:0/2633190270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f107006cbd0 0x7f107006f090 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 --2- 172.21.15.31:0/2633190270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10800ffca0 0x7f10801958c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:57.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 --2- 172.21.15.31:0/2633190270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10800feaa0 0x7f1080195380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:57.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 -- 172.21.15.31:0/2633190270 >> 172.21.15.31:0/2633190270 conn(0x7f10800fa250 msgr2=0x7f1080068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:49:57.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 -- 172.21.15.31:0/2633190270 shutdown_connections 2024-09-16T08:49:57.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:57.795+0000 7f1086e0e640 1 -- 172.21.15.31:0/2633190270 wait complete. 2024-09-16T08:49:59.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:58 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2633190270' entity='client.admin' 2024-09-16T08:49:59.096 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:49:58 smithi031 ceph-mon[27844]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:59.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:58 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2633190270' entity='client.admin' 2024-09-16T08:49:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:49:58 smithi097 ceph-mon[35478]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:49:59.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4262581497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 msgr2=0x7f71ec0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:49:59.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4262581497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec0fff90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f71e8009880 tx=0x7f71e802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4262581497 shutdown_connections 2024-09-16T08:49:59.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4262581497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec100d90 0x7f71ec101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4262581497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec0fff90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4262581497 >> 172.21.15.31:0/4262581497 conn(0x7f71ec0fb320 msgr2=0x7f71ec0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4262581497 shutdown_connections 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4262581497 wait complete. 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 Processor -- start 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.626+0000 7f71f3f9d640 1 -- start start 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f3f9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec195450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f3f9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec100d90 0x7f71ec195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f3f9d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec195ed0 con 0x7f71ec0ffb90 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f3f9d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec196040 con 0x7f71ec100d90 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1d39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec195450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1d39640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec195450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60672/0 (socket says 172.21.15.31:60672) 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1d39640 1 -- 172.21.15.31:0/4030059911 learned_addr learned my addr 172.21.15.31:0/4030059911 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:49:59.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1538640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec100d90 0x7f71ec195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1d39640 1 -- 172.21.15.31:0/4030059911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec100d90 msgr2=0x7f71ec195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1d39640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec100d90 0x7f71ec195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1d39640 1 -- 172.21.15.31:0/4030059911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e8009560 con 0x7f71ec0ffb90 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f1d39640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec195450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71e8004fa0 tx=0x7f71e8004d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71e2ffd640 1 -- 172.21.15.31:0/4030059911 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71e8005560 con 0x7f71ec0ffb90 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71e2ffd640 1 -- 172.21.15.31:0/4030059911 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f71e8003a70 con 0x7f71ec0ffb90 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec19aac0 con 0x7f71ec0ffb90 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.627+0000 7f71e2ffd640 1 -- 172.21.15.31:0/4030059911 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71e80319a0 con 0x7f71ec0ffb90 2024-09-16T08:49:59.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.628+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec19afe0 con 0x7f71ec0ffb90 2024-09-16T08:49:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.629+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ec0fff90 con 0x7f71ec0ffb90 2024-09-16T08:49:59.631 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.629+0000 7f71e2ffd640 1 -- 172.21.15.31:0/4030059911 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f71e8003be0 con 0x7f71ec0ffb90 2024-09-16T08:49:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.630+0000 7f71e2ffd640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f71d806cbd0 0x7f71d806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:49:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.630+0000 7f71e2ffd640 1 -- 172.21.15.31:0/4030059911 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f71e80ac2e0 con 0x7f71ec0ffb90 2024-09-16T08:49:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.630+0000 7f71f1538640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f71d806cbd0 0x7f71d806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:49:59.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.630+0000 7f71f1538640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f71d806cbd0 0x7f71d806f090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f71ec196970 tx=0x7f71dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:49:59.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.634+0000 7f71e2ffd640 1 -- 172.21.15.31:0/4030059911 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f71e8077790 con 0x7f71ec0ffb90 2024-09-16T08:49:59.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.806+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f71ec108500 con 0x7f71ec0ffb90 2024-09-16T08:49:59.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.811+0000 7f71e2ffd640 1 -- 172.21.15.31:0/4030059911 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) v1 ==== 120+0+0 (secure 0 0 0) 0x7f71e80abaa0 con 0x7f71ec0ffb90 2024-09-16T08:49:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f71d806cbd0 msgr2=0x7f71d806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:49:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f71d806cbd0 0x7f71d806f090 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f71ec196970 tx=0x7f71dc00a2d0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 msgr2=0x7f71ec195450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:49:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec195450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f71e8004fa0 tx=0x7f71e8004d80 comp rx=0 tx=0).stop 2024-09-16T08:49:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 shutdown_connections 2024-09-16T08:49:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f71d806cbd0 0x7f71d806f090 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f71ec100d90 0x7f71ec195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 --2- 172.21.15.31:0/4030059911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71ec0ffb90 0x7f71ec195450 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:49:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 >> 172.21.15.31:0/4030059911 conn(0x7f71ec0fb320 msgr2=0x7f71ec103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:49:59.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 shutdown_connections 2024-09-16T08:49:59.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:49:59.813+0000 7f71f3f9d640 1 -- 172.21.15.31:0/4030059911 wait complete. 2024-09-16T08:50:00.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 -- 172.21.15.31:0/3276844927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880fea80 msgr2=0x7f6e880fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/3276844927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880fea80 0x7f6e880fee80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009880 tx=0x7f6e7c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 -- 172.21.15.31:0/3276844927 shutdown_connections 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/3276844927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e880ffc80 0x7f6e88100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/3276844927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880fea80 0x7f6e880fee80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 -- 172.21.15.31:0/3276844927 >> 172.21.15.31:0/3276844927 conn(0x7f6e880fa230 msgr2=0x7f6e880fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 -- 172.21.15.31:0/3276844927 shutdown_connections 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.202+0000 7f6e8eee7640 1 -- 172.21.15.31:0/3276844927 wait complete. 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8eee7640 1 Processor -- start 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8eee7640 1 -- start start 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8eee7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e880fea80 0x7f6e88197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8eee7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880ffc80 0x7f6e88197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:00.210 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8eee7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e88198150 con 0x7f6e880ffc80 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8eee7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e881982c0 con 0x7f6e880fea80 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e87fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880ffc80 0x7f6e88197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e87fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880ffc80 0x7f6e88197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60688/0 (socket says 172.21.15.31:60688) 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e87fff640 1 -- 172.21.15.31:0/2776571621 learned_addr learned my addr 172.21.15.31:0/2776571621 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8cc83640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e880fea80 0x7f6e88197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e87fff640 1 -- 172.21.15.31:0/2776571621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e880fea80 msgr2=0x7f6e88197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e87fff640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e880fea80 0x7f6e88197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e87fff640 1 -- 172.21.15.31:0/2776571621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c009560 con 0x7f6e880ffc80 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e8cc83640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e880fea80 0x7f6e88197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:50:00.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.203+0000 7f6e87fff640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880ffc80 0x7f6e88197b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c840 tx=0x7f6e7400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.204+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e740041f0 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.204+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f6e74007e30 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.204+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8819cd60 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.204+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e740047a0 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.204+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8819d330 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.205+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e880fee80 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.205+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f6e740079b0 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.206+0000 7f6e85ffb640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e7806cbd0 0x7f6e7806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.206+0000 7f6e8cc83640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e7806cbd0 0x7f6e7806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.206+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f6e7408aea0 con 0x7f6e880ffc80 2024-09-16T08:50:00.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.206+0000 7f6e8cc83640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e7806cbd0 0x7f6e7806f090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c000c00 tx=0x7f6e7c004f40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:00.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.210+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6e740563d0 con 0x7f6e880ffc80 2024-09-16T08:50:00.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.385+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7f6e881073f0 con 0x7f6e880ffc80 2024-09-16T08:50:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.388+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f6e740563d0 con 0x7f6e880ffc80 2024-09-16T08:50:00.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.389+0000 7f6e85ffb640 1 -- 172.21.15.31:0/2776571621 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6e7408a660 con 0x7f6e880ffc80 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e7806cbd0 msgr2=0x7f6e7806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e7806cbd0 0x7f6e7806f090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c000c00 tx=0x7f6e7c004f40 comp rx=0 tx=0).stop 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880ffc80 msgr2=0x7f6e88197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880ffc80 0x7f6e88197b80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400c840 tx=0x7f6e7400cd00 comp rx=0 tx=0).stop 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 shutdown_connections 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e7806cbd0 0x7f6e7806f090 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e880ffc80 0x7f6e88197b80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 --2- 172.21.15.31:0/2776571621 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e880fea80 0x7f6e88197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 >> 172.21.15.31:0/2776571621 conn(0x7f6e880fa230 msgr2=0x7f6e88068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 shutdown_connections 2024-09-16T08:50:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.391+0000 7f6e8eee7640 1 -- 172.21.15.31:0/2776571621 wait complete. 2024-09-16T08:50:00.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 -- 172.21.15.31:0/3951037064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080feaa0 msgr2=0x7f34080feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3951037064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080feaa0 0x7f34080feea0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009880 tx=0x7f33f802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 -- 172.21.15.31:0/3951037064 shutdown_connections 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3951037064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34080ffca0 0x7f3408100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3951037064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080feaa0 0x7f34080feea0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 -- 172.21.15.31:0/3951037064 >> 172.21.15.31:0/3951037064 conn(0x7f34080fa250 msgr2=0x7f34080fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 -- 172.21.15.31:0/3951037064 shutdown_connections 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.661+0000 7f340e1ca640 1 -- 172.21.15.31:0/3951037064 wait complete. 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f340e1ca640 1 Processor -- start 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f340e1ca640 1 -- start start 2024-09-16T08:50:00.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f340e1ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34080feaa0 0x7f3408102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f340e1ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080ffca0 0x7f3408101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f340e1ca640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34081015c0 con 0x7f34080ffca0 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f340e1ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3408101730 con 0x7f34080feaa0 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f3406ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080ffca0 0x7f3408101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f3406ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080ffca0 0x7f3408101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60710/0 (socket says 172.21.15.31:60710) 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f3406ffd640 1 -- 172.21.15.31:0/3706163429 learned_addr learned my addr 172.21.15.31:0/3706163429 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.662+0000 7f34077fe640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34080feaa0 0x7f3408102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f3406ffd640 1 -- 172.21.15.31:0/3706163429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34080feaa0 msgr2=0x7f3408102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f3406ffd640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34080feaa0 0x7f3408102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f3406ffd640 1 -- 172.21.15.31:0/3706163429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f8009560 con 0x7f34080ffca0 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f3406ffd640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080ffca0 0x7f3408101080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d6f0 tx=0x7f33fc00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f3404ff9640 1 -- 172.21.15.31:0/3706163429 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc009a30 con 0x7f34080ffca0 2024-09-16T08:50:00.807 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f3404ff9640 1 -- 172.21.15.31:0/3706163429 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f33fc0054c0 con 0x7f34080ffca0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f3404ff9640 1 -- 172.21.15.31:0/3706163429 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc004dc0 con 0x7f34080ffca0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3408101a10 con 0x7f34080ffca0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.663+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340819d5e0 con 0x7f34080ffca0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.664+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34080feea0 con 0x7f34080ffca0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.665+0000 7f3404ff9640 1 -- 172.21.15.31:0/3706163429 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f33fc004f40 con 0x7f34080ffca0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.665+0000 7f3404ff9640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f33f006cb70 0x7f33f006f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.665+0000 7f34077fe640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f33f006cb70 0x7f33f006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.665+0000 7f3404ff9640 1 -- 172.21.15.31:0/3706163429 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f33fc08ac30 con 0x7f34080ffca0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.666+0000 7f34077fe640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f33f006cb70 0x7f33f006f030 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006010 tx=0x7f33f8000bc0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.670+0000 7f3404ff9640 1 -- 172.21.15.31:0/3706163429 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f33fc056190 con 0x7f34080ffca0 2024-09-16T08:50:00.852 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.852+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) v1 -- 0x7f340819d8f0 con 0x7f34080ffca0 2024-09-16T08:50:00.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.856+0000 7f3404ff9640 1 -- 172.21.15.31:0/3706163429 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v16)=0 v16) v1 ==== 119+0+0 (secure 0 0 0) 0x7f33fc0170b0 con 0x7f34080ffca0 2024-09-16T08:50:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.857+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f33f006cb70 msgr2=0x7f33f006f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.857+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f33f006cb70 0x7f33f006f030 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f33f8006010 tx=0x7f33f8000bc0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080ffca0 msgr2=0x7f3408101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:00.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080ffca0 0x7f3408101080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d6f0 tx=0x7f33fc00dbb0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 shutdown_connections 2024-09-16T08:50:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f33f006cb70 0x7f33f006f030 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f34080ffca0 0x7f3408101080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 --2- 172.21.15.31:0/3706163429 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f34080feaa0 0x7f3408102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 >> 172.21.15.31:0/3706163429 conn(0x7f34080fa250 msgr2=0x7f34080fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 shutdown_connections 2024-09-16T08:50:00.859 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:00.858+0000 7f340e1ca640 1 -- 172.21.15.31:0/3706163429 wait complete. 2024-09-16T08:50:01.047 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:00 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/4030059911' entity='client.admin' 2024-09-16T08:50:01.047 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:00 smithi031 ceph-mon[27844]: overall HEALTH_OK 2024-09-16T08:50:01.047 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:00 smithi031 ceph-mon[27844]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:01.047 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:00 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2776571621' entity='client.admin' 2024-09-16T08:50:01.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:00 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/4030059911' entity='client.admin' 2024-09-16T08:50:01.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:00 smithi097 ceph-mon[35478]: overall HEALTH_OK 2024-09-16T08:50:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:00 smithi097 ceph-mon[35478]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:00 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2776571621' entity='client.admin' 2024-09-16T08:50:01.673 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:50:02.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:01 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3706163429' entity='client.admin' 2024-09-16T08:50:02.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:01 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3706163429' entity='client.admin' 2024-09-16T08:50:03.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:02 smithi031 ceph-mon[27844]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:03.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:03.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:03.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:02 smithi097 ceph-mon[35478]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:04.030 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:03 smithi031 ceph-mon[27844]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-09-16T08:50:04.030 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:03 smithi031 ceph-mon[27844]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-09-16T08:50:04.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:03 smithi097 ceph-mon[35478]: Reconfiguring alertmanager.smithi031 (dependencies changed)... 2024-09-16T08:50:04.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:03 smithi097 ceph-mon[35478]: Reconfiguring daemon alertmanager.smithi031 on smithi031 2024-09-16T08:50:05.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:04 smithi031 ceph-mon[27844]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:50:05.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:04 smithi097 ceph-mon[35478]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:07.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:07 smithi031 ceph-mon[27844]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:07.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:07 smithi097 ceph-mon[35478]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:08.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:08.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:09.170 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:09 smithi031 ceph-mon[27844]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:09.194 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-16T08:50:09.194 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-16T08:50:09.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:09 smithi097 ceph-mon[35478]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:09.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/2478626608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0ffcf0 msgr2=0x7f7f9c100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/2478626608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0ffcf0 0x7f7f9c100170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7f90009880 tx=0x7f7f9002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/2478626608 shutdown_connections 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/2478626608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0ffcf0 0x7f7f9c100170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/2478626608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0feaf0 0x7f7f9c0feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/2478626608 >> 172.21.15.31:0/2478626608 conn(0x7f7f9c0fa280 msgr2=0x7f7f9c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/2478626608 shutdown_connections 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.330+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/2478626608 wait complete. 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7fa0ff7640 1 Processor -- start 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7fa0ff7640 1 -- start start 2024-09-16T08:50:09.519 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7fa0ff7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0feaf0 0x7f7f9c0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7fa0ff7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0ffcf0 0x7f7f9c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7fa0ff7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c19a8d0 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7fa0ff7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c19aa40 con 0x7f7f9c0ffcf0 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7f9a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0feaf0 0x7f7f9c0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7f9a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0feaf0 0x7f7f9c0687d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38168/0 (socket says 172.21.15.31:38168) 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.331+0000 7f7f9a59c640 1 -- 172.21.15.31:0/3161539085 learned_addr learned my addr 172.21.15.31:0/3161539085 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f99d9b640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0ffcf0 0x7f7f9c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f9a59c640 1 -- 172.21.15.31:0/3161539085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0ffcf0 msgr2=0x7f7f9c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:09.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f9a59c640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0ffcf0 0x7f7f9c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f9a59c640 1 -- 172.21.15.31:0/3161539085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009560 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f9a59c640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0feaf0 0x7f7f9c0687d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00d8d0 tx=0x7f7f8c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f8b7fe640 1 -- 172.21.15.31:0/3161539085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f8c009a30 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f8b7fe640 1 -- 172.21.15.31:0/3161539085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f7f8c009b90 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7f8b7fe640 1 -- 172.21.15.31:0/3161539085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f8c00f5f0 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c19ad20 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.332+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c19b210 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.333+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c18e550 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.334+0000 7f7f8b7fe640 1 -- 172.21.15.31:0/3161539085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f7f8c009d00 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.334+0000 7f7f8b7fe640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7f8406cbd0 0x7f7f8406f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.334+0000 7f7f8b7fe640 1 -- 172.21.15.31:0/3161539085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7f8c08aec0 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.334+0000 7f7f99d9b640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7f8406cbd0 0x7f7f8406f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.335+0000 7f7f99d9b640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7f8406cbd0 0x7f7f8406f090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c068550 tx=0x7f7f90031400 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.338+0000 7f7f8b7fe640 1 -- 172.21.15.31:0/3161539085 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7f8c0563f0 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.511+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) v1 -- 0x7f7f9c1075c0 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.515+0000 7f7f8b7fe640 1 -- 172.21.15.31:0/3161539085 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v17)=0 v17) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7f8c0170b0 con 0x7f7f9c0feaf0 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7f8406cbd0 msgr2=0x7f7f8406f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7f8406cbd0 0x7f7f8406f090 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c068550 tx=0x7f7f90031400 comp rx=0 tx=0).stop 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0feaf0 msgr2=0x7f7f9c0687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0feaf0 0x7f7f9c0687d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00d8d0 tx=0x7f7f8c00dd90 comp rx=0 tx=0).stop 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 shutdown_connections 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7f8406cbd0 0x7f7f8406f090 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f9c0ffcf0 0x7f7f9c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 --2- 172.21.15.31:0/3161539085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f9c0feaf0 0x7f7f9c0687d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:09.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 >> 172.21.15.31:0/3161539085 conn(0x7f7f9c0fa280 msgr2=0x7f7f9c1033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:09.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 shutdown_connections 2024-09-16T08:50:09.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:09.517+0000 7f7fa0ff7640 1 -- 172.21.15.31:0/3161539085 wait complete. 2024-09-16T08:50:10.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 -- 172.21.15.31:0/3454008870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 msgr2=0x7fb0e0100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:10.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3454008870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0100120 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0009880 tx=0x7fb0d002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 -- 172.21.15.31:0/3454008870 shutdown_connections 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3454008870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0100120 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3454008870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e00feaa0 0x7fb0e00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 -- 172.21.15.31:0/3454008870 >> 172.21.15.31:0/3454008870 conn(0x7fb0e00fa250 msgr2=0x7fb0e00fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 -- 172.21.15.31:0/3454008870 shutdown_connections 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.649+0000 7fb0e879a640 1 -- 172.21.15.31:0/3454008870 wait complete. 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e879a640 1 Processor -- start 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e879a640 1 -- start start 2024-09-16T08:50:10.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e879a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e00feaa0 0x7fb0e0102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e879a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e879a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e01015c0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e879a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e0101730 con 0x7fb0e00feaa0 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e5d35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e5d35640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:38196/0 (socket says 172.21.15.31:38196) 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e5d35640 1 -- 172.21.15.31:0/3751166043 learned_addr learned my addr 172.21.15.31:0/3751166043 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.650+0000 7fb0e6536640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e00feaa0 0x7fb0e0102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0e5d35640 1 -- 172.21.15.31:0/3751166043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e00feaa0 msgr2=0x7fb0e0102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0e5d35640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e00feaa0 0x7fb0e0102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0e5d35640 1 -- 172.21.15.31:0/3751166043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0009560 con 0x7fb0e00ffca0 2024-09-16T08:50:10.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0e5d35640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0101080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0006010 tx=0x7fb0d0004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:10.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0d77fe640 1 -- 172.21.15.31:0/3751166043 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d00054e0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e01019b0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0d77fe640 1 -- 172.21.15.31:0/3751166043 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7fb0d0005640 con 0x7fb0e00ffca0 2024-09-16T08:50:10.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0d77fe640 1 -- 172.21.15.31:0/3751166043 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0d0031c40 con 0x7fb0e00ffca0 2024-09-16T08:50:10.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.651+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e0101ea0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.652+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e00feea0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.653+0000 7fb0d77fe640 1 -- 172.21.15.31:0/3751166043 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7fb0d00035f0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.653+0000 7fb0d77fe640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb0cc06cbc0 0x7fb0cc06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:10.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.653+0000 7fb0d77fe640 1 -- 172.21.15.31:0/3751166043 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb0d00ac380 con 0x7fb0e00ffca0 2024-09-16T08:50:10.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.653+0000 7fb0e6536640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb0cc06cbc0 0x7fb0cc06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:10.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.654+0000 7fb0e6536640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb0cc06cbc0 0x7fb0cc06f080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0058c0 tx=0x7fb0dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:10.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.658+0000 7fb0d77fe640 1 -- 172.21.15.31:0/3751166043 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb0d00778e0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:10 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3161539085' entity='client.admin' 2024-09-16T08:50:10.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:10 smithi031 ceph-mon[27844]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:10.832 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.831+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7fb0e0107570 con 0x7fb0e00ffca0 2024-09-16T08:50:10.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.835+0000 7fb0d77fe640 1 -- 172.21.15.31:0/3751166043 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v18)=0 v18) v1 ==== 108+0+0 (secure 0 0 0) 0x7fb0d004a0b0 con 0x7fb0e00ffca0 2024-09-16T08:50:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.837+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb0cc06cbc0 msgr2=0x7fb0cc06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.837+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb0cc06cbc0 0x7fb0cc06f080 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0058c0 tx=0x7fb0dc00a2d0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.837+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 msgr2=0x7fb0e0101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:10.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.837+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0101080 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0006010 tx=0x7fb0d0004da0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.838+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 shutdown_connections 2024-09-16T08:50:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.838+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb0cc06cbc0 0x7fb0cc06f080 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.838+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0e00ffca0 0x7fb0e0101080 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.838+0000 7fb0e879a640 1 --2- 172.21.15.31:0/3751166043 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb0e00feaa0 0x7fb0e0102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.838+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 >> 172.21.15.31:0/3751166043 conn(0x7fb0e00fa250 msgr2=0x7fb0e0068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.838+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 shutdown_connections 2024-09-16T08:50:10.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:10.838+0000 7fb0e879a640 1 -- 172.21.15.31:0/3751166043 wait complete. 2024-09-16T08:50:11.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:10 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3161539085' entity='client.admin' 2024-09-16T08:50:11.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:10 smithi097 ceph-mon[35478]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 -- 172.21.15.31:0/3971501035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280feaf0 msgr2=0x7ffb280feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/3971501035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280feaf0 0x7ffb280feef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009880 tx=0x7ffb1802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 -- 172.21.15.31:0/3971501035 shutdown_connections 2024-09-16T08:50:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/3971501035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280ffcf0 0x7ffb28100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/3971501035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280feaf0 0x7ffb280feef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 -- 172.21.15.31:0/3971501035 >> 172.21.15.31:0/3971501035 conn(0x7ffb280fa280 msgr2=0x7ffb280fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 -- 172.21.15.31:0/3971501035 shutdown_connections 2024-09-16T08:50:11.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.045+0000 7ffb2cff6640 1 -- 172.21.15.31:0/3971501035 wait complete. 2024-09-16T08:50:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.046+0000 7ffb2cff6640 1 Processor -- start 2024-09-16T08:50:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.046+0000 7ffb2cff6640 1 -- start start 2024-09-16T08:50:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.046+0000 7ffb2cff6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280feaf0 0x7ffb281977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.046+0000 7ffb2cff6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280ffcf0 0x7ffb28197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.046+0000 7ffb2cff6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281982e0 con 0x7ffb280ffcf0 2024-09-16T08:50:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb2cff6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28198450 con 0x7ffb280feaf0 2024-09-16T08:50:11.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb2659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280feaf0 0x7ffb281977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb2659c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280feaf0 0x7ffb281977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:40008/0 (socket says 172.21.15.31:40008) 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb2659c640 1 -- 172.21.15.31:0/2275039419 learned_addr learned my addr 172.21.15.31:0/2275039419 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb2659c640 1 -- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280ffcf0 msgr2=0x7ffb28197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb25d9b640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280ffcf0 0x7ffb28197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb2659c640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280ffcf0 0x7ffb28197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb2659c640 1 -- 172.21.15.31:0/2275039419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb18009560 con 0x7ffb280feaf0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.047+0000 7ffb25d9b640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280ffcf0 0x7ffb28197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.048+0000 7ffb2659c640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280feaf0 0x7ffb281977d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ffb1802a9c0 tx=0x7ffb18004cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.048+0000 7ffb177fe640 1 -- 172.21.15.31:0/2275039419 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb180054a0 con 0x7ffb280feaf0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.048+0000 7ffb177fe640 1 -- 172.21.15.31:0/2275039419 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7ffb18005600 con 0x7ffb280feaf0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.048+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2819ce90 con 0x7ffb280feaf0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.048+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2819d300 con 0x7ffb280feaf0 2024-09-16T08:50:11.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.048+0000 7ffb177fe640 1 -- 172.21.15.31:0/2275039419 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb18031ae0 con 0x7ffb280feaf0 2024-09-16T08:50:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.049+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb281075a0 con 0x7ffb280feaf0 2024-09-16T08:50:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.049+0000 7ffb177fe640 1 -- 172.21.15.31:0/2275039419 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7ffb18031c40 con 0x7ffb280feaf0 2024-09-16T08:50:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.050+0000 7ffb177fe640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffb1006cbc0 0x7ffb1006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:11.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.050+0000 7ffb177fe640 1 -- 172.21.15.31:0/2275039419 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ffb180ac3e0 con 0x7ffb280feaf0 2024-09-16T08:50:11.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.050+0000 7ffb25d9b640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffb1006cbc0 0x7ffb1006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:11.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.050+0000 7ffb25d9b640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffb1006cbc0 0x7ffb1006f080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb28198cf0 tx=0x7ffb1c005e50 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:11.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.054+0000 7ffb177fe640 1 -- 172.21.15.31:0/2275039419 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ffb18077870 con 0x7ffb280feaf0 2024-09-16T08:50:11.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.230+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) v1 -- 0x7ffb28107780 con 0x7ffb280feaf0 2024-09-16T08:50:11.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.237+0000 7ffb177fe640 1 -- 172.21.15.31:0/2275039419 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) v1 ==== 120+0+0 (secure 0 0 0) 0x7ffb1807ae90 con 0x7ffb280feaf0 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffb1006cbc0 msgr2=0x7ffb1006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffb1006cbc0 0x7ffb1006f080 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb28198cf0 tx=0x7ffb1c005e50 comp rx=0 tx=0).stop 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280feaf0 msgr2=0x7ffb281977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280feaf0 0x7ffb281977d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ffb1802a9c0 tx=0x7ffb18004cd0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 shutdown_connections 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffb1006cbc0 0x7ffb1006f080 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb280ffcf0 0x7ffb28197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 --2- 172.21.15.31:0/2275039419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb280feaf0 0x7ffb281977d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:11.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 >> 172.21.15.31:0/2275039419 conn(0x7ffb280fa280 msgr2=0x7ffb280fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:11.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 shutdown_connections 2024-09-16T08:50:11.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:11.239+0000 7ffb2cff6640 1 -- 172.21.15.31:0/2275039419 wait complete. 2024-09-16T08:50:11.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:11 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3751166043' entity='client.admin' 2024-09-16T08:50:11.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:11 smithi031 ceph-mon[27844]: from='client.? ' entity='client.admin' 2024-09-16T08:50:12.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:11 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3751166043' entity='client.admin' 2024-09-16T08:50:12.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:11 smithi097 ceph-mon[35478]: from='client.? ' entity='client.admin' 2024-09-16T08:50:13.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:12 smithi031 ceph-mon[27844]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:13.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:12 smithi097 ceph-mon[35478]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 -- 172.21.15.31:0/1290327977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 msgr2=0x7f5e20102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/1290327977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e20102ad0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e10009880 tx=0x7f5e1002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 -- 172.21.15.31:0/1290327977 shutdown_connections 2024-09-16T08:50:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/1290327977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e20103010 0x7f5e20105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/1290327977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e20102ad0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 -- 172.21.15.31:0/1290327977 >> 172.21.15.31:0/1290327977 conn(0x7f5e200fa280 msgr2=0x7f5e200fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:13.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 -- 172.21.15.31:0/1290327977 shutdown_connections 2024-09-16T08:50:13.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.292+0000 7f5e25ec0640 1 -- 172.21.15.31:0/1290327977 wait complete. 2024-09-16T08:50:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e25ec0640 1 Processor -- start 2024-09-16T08:50:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e25ec0640 1 -- start start 2024-09-16T08:50:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e25ec0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e200ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e25ec0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e20103010 0x7f5e200fe690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e25ec0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200febd0 con 0x7f5e201006e0 2024-09-16T08:50:13.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e25ec0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200fed40 con 0x7f5e20103010 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e1f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e200ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e1f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e200ffff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44644/0 (socket says 172.21.15.31:44644) 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.293+0000 7f5e1f7fe640 1 -- 172.21.15.31:0/2715962284 learned_addr learned my addr 172.21.15.31:0/2715962284 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1effd640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e20103010 0x7f5e200fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1f7fe640 1 -- 172.21.15.31:0/2715962284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e20103010 msgr2=0x7f5e200fe690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1f7fe640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e20103010 0x7f5e200fe690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1f7fe640 1 -- 172.21.15.31:0/2715962284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e10009560 con 0x7f5e201006e0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1f7fe640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e200ffff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e10006010 tx=0x7f5e10004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1cff9640 1 -- 172.21.15.31:0/2715962284 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e100054e0 con 0x7f5e201006e0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1cff9640 1 -- 172.21.15.31:0/2715962284 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f5e10005640 con 0x7f5e201006e0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e200ff020 con 0x7f5e201006e0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e1cff9640 1 -- 172.21.15.31:0/2715962284 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e10031c40 con 0x7f5e201006e0 2024-09-16T08:50:13.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.294+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2019d630 con 0x7f5e201006e0 2024-09-16T08:50:13.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.295+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e20108680 con 0x7f5e201006e0 2024-09-16T08:50:13.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.296+0000 7f5e1cff9640 1 -- 172.21.15.31:0/2715962284 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f5e1003c470 con 0x7f5e201006e0 2024-09-16T08:50:13.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.296+0000 7f5e1cff9640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cbc0 0x7f5e0806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:13.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.297+0000 7f5e1cff9640 1 -- 172.21.15.31:0/2715962284 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f5e100ac310 con 0x7f5e201006e0 2024-09-16T08:50:13.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.297+0000 7f5e1effd640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cbc0 0x7f5e0806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:13.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.297+0000 7f5e1effd640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cbc0 0x7f5e0806f080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e14005fd0 tx=0x7f5e14005e20 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:13.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.301+0000 7f5e1cff9640 1 -- 172.21.15.31:0/2715962284 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5e10077840 con 0x7f5e201006e0 2024-09-16T08:50:13.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.474+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) v1 -- 0x7f5e201088c0 con 0x7f5e201006e0 2024-09-16T08:50:13.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.478+0000 7f5e1cff9640 1 -- 172.21.15.31:0/2715962284 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e100400c0 con 0x7f5e201006e0 2024-09-16T08:50:13.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cbc0 msgr2=0x7f5e0806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:13.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cbc0 0x7f5e0806f080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5e14005fd0 tx=0x7f5e14005e20 comp rx=0 tx=0).stop 2024-09-16T08:50:13.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 msgr2=0x7f5e200ffff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e200ffff0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5e10006010 tx=0x7f5e10004da0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 shutdown_connections 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cbc0 0x7f5e0806f080 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e20103010 0x7f5e200fe690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.479+0000 7f5e25ec0640 1 --2- 172.21.15.31:0/2715962284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e201006e0 0x7f5e200ffff0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.480+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 >> 172.21.15.31:0/2715962284 conn(0x7f5e200fa280 msgr2=0x7f5e200fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.480+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 shutdown_connections 2024-09-16T08:50:13.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:13.480+0000 7f5e25ec0640 1 -- 172.21.15.31:0/2715962284 wait complete. 2024-09-16T08:50:14.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:14 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2715962284' entity='client.admin' 2024-09-16T08:50:14.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:14 smithi097 ceph-mon[35478]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:14.779 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:14 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2715962284' entity='client.admin' 2024-09-16T08:50:14.779 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:14 smithi031 ceph-mon[27844]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 -- 172.21.15.31:0/2303322282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 msgr2=0x7f1b5c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 --2- 172.21.15.31:0/2303322282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c1053a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1b500098b0 tx=0x7f1b5002a4d0 comp rx=0 tx=0).stop 2024-09-16T08:50:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 -- 172.21.15.31:0/2303322282 shutdown_connections 2024-09-16T08:50:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 --2- 172.21.15.31:0/2303322282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c1053a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:16.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 --2- 172.21.15.31:0/2303322282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b5c100680 0x7f1b5c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 -- 172.21.15.31:0/2303322282 >> 172.21.15.31:0/2303322282 conn(0x7f1b5c0fa280 msgr2=0x7f1b5c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 -- 172.21.15.31:0/2303322282 shutdown_connections 2024-09-16T08:50:16.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.019+0000 7f1b6084e640 1 -- 172.21.15.31:0/2303322282 wait complete. 2024-09-16T08:50:16.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b6084e640 1 Processor -- start 2024-09-16T08:50:16.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b6084e640 1 -- start start 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b6084e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b5c100680 0x7f1b5c0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b6084e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c0fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b6084e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0feb70 con 0x7f1b5c102fb0 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b6084e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0fece0 con 0x7f1b5c100680 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b59d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c0fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b59d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c0fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44652/0 (socket says 172.21.15.31:44652) 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.020+0000 7f1b59d9b640 1 -- 172.21.15.31:0/1498837590 learned_addr learned my addr 172.21.15.31:0/1498837590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.021+0000 7f1b5a59c640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b5c100680 0x7f1b5c0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.021+0000 7f1b59d9b640 1 -- 172.21.15.31:0/1498837590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b5c100680 msgr2=0x7f1b5c0fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.021+0000 7f1b59d9b640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b5c100680 0x7f1b5c0fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.021+0000 7f1b59d9b640 1 -- 172.21.15.31:0/1498837590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50009560 con 0x7f1b5c102fb0 2024-09-16T08:50:16.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.021+0000 7f1b59d9b640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c0fe630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009880 tx=0x7f1b50004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:16.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.023+0000 7f1b4b7fe640 1 -- 172.21.15.31:0/1498837590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b50005460 con 0x7f1b5c102fb0 2024-09-16T08:50:16.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.023+0000 7f1b4b7fe640 1 -- 172.21.15.31:0/1498837590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f1b500055c0 con 0x7f1b5c102fb0 2024-09-16T08:50:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.024+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c0fef60 con 0x7f1b5c102fb0 2024-09-16T08:50:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.024+0000 7f1b4b7fe640 1 -- 172.21.15.31:0/1498837590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b50031a90 con 0x7f1b5c102fb0 2024-09-16T08:50:16.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.024+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c0ff390 con 0x7f1b5c102fb0 2024-09-16T08:50:16.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.026+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c108720 con 0x7f1b5c102fb0 2024-09-16T08:50:16.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.026+0000 7f1b4b7fe640 1 -- 172.21.15.31:0/1498837590 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f1b50031bf0 con 0x7f1b5c102fb0 2024-09-16T08:50:16.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.027+0000 7f1b4b7fe640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1b4406cc10 0x7f1b4406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:16.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.027+0000 7f1b4b7fe640 1 -- 172.21.15.31:0/1498837590 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f1b500ac1a0 con 0x7f1b5c102fb0 2024-09-16T08:50:16.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.027+0000 7f1b5a59c640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1b4406cc10 0x7f1b4406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:16.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.028+0000 7f1b5a59c640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1b4406cc10 0x7f1b4406f0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009990 tx=0x7f1b4c009110 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:16.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.031+0000 7f1b4b7fe640 1 -- 172.21.15.31:0/1498837590 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1b50077770 con 0x7f1b5c102fb0 2024-09-16T08:50:16.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.206+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f1b5c19d8b0 con 0x7f1b5c102fb0 2024-09-16T08:50:16.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.210+0000 7f1b4b7fe640 1 -- 172.21.15.31:0/1498837590 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f1b5007ad90 con 0x7f1b5c102fb0 2024-09-16T08:50:16.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1b4406cc10 msgr2=0x7f1b4406f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:16.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1b4406cc10 0x7f1b4406f0d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009990 tx=0x7f1b4c009110 comp rx=0 tx=0).stop 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 msgr2=0x7f1b5c0fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c0fe630 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009880 tx=0x7f1b50004d30 comp rx=0 tx=0).stop 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 shutdown_connections 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1b4406cc10 0x7f1b4406f0d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b5c102fb0 0x7f1b5c0fe630 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 --2- 172.21.15.31:0/1498837590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b5c100680 0x7f1b5c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.212+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 >> 172.21.15.31:0/1498837590 conn(0x7f1b5c0fa280 msgr2=0x7f1b5c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.213+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 shutdown_connections 2024-09-16T08:50:16.214 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:16.213+0000 7f1b6084e640 1 -- 172.21.15.31:0/1498837590 wait complete. 2024-09-16T08:50:17.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:17 smithi031 ceph-mon[27844]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:17.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:17 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1498837590' entity='client.admin' 2024-09-16T08:50:17.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:17 smithi097 ceph-mon[35478]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:17.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:17 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1498837590' entity='client.admin' 2024-09-16T08:50:18.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 -- 172.21.15.31:0/1351221649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 msgr2=0x7f0144102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:18.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 --2- 172.21.15.31:0/1351221649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f0144102a20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f012c0098b0 tx=0x7f012c02a4d0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 -- 172.21.15.31:0/1351221649 shutdown_connections 2024-09-16T08:50:18.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 --2- 172.21.15.31:0/1351221649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0144102f60 0x7f0144105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 --2- 172.21.15.31:0/1351221649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f0144102a20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 -- 172.21.15.31:0/1351221649 >> 172.21.15.31:0/1351221649 conn(0x7f01440fa250 msgr2=0x7f01440fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:18.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 -- 172.21.15.31:0/1351221649 shutdown_connections 2024-09-16T08:50:18.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.256+0000 7f014b01b640 1 -- 172.21.15.31:0/1351221649 wait complete. 2024-09-16T08:50:18.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f014b01b640 1 Processor -- start 2024-09-16T08:50:18.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f014b01b640 1 -- start start 2024-09-16T08:50:18.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f014b01b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f01441977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f014b01b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0144102f60 0x7f0144197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f014b01b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01441982f0 con 0x7f0144100630 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f014b01b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0144198460 con 0x7f0144102f60 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0148db7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f01441977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0148db7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f01441977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44656/0 (socket says 172.21.15.31:44656) 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0148db7640 1 -- 172.21.15.31:0/392416424 learned_addr learned my addr 172.21.15.31:0/392416424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0143fff640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0144102f60 0x7f0144197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0148db7640 1 -- 172.21.15.31:0/392416424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0144102f60 msgr2=0x7f0144197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0148db7640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0144102f60 0x7f0144197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0148db7640 1 -- 172.21.15.31:0/392416424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c009560 con 0x7f0144100630 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.257+0000 7f0143fff640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0144102f60 0x7f0144197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.258+0000 7f0148db7640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f01441977e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f012c005ec0 tx=0x7f012c004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:18.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.258+0000 7f0141ffb640 1 -- 172.21.15.31:0/392416424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f012c005520 con 0x7f0144100630 2024-09-16T08:50:18.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.258+0000 7f0141ffb640 1 -- 172.21.15.31:0/392416424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f012c005680 con 0x7f0144100630 2024-09-16T08:50:18.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.258+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014419cea0 con 0x7f0144100630 2024-09-16T08:50:18.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.258+0000 7f0141ffb640 1 -- 172.21.15.31:0/392416424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f012c031a80 con 0x7f0144100630 2024-09-16T08:50:18.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.258+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f014419d310 con 0x7f0144100630 2024-09-16T08:50:18.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.259+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01441086a0 con 0x7f0144100630 2024-09-16T08:50:18.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.260+0000 7f0141ffb640 1 -- 172.21.15.31:0/392416424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f012c031cb0 con 0x7f0144100630 2024-09-16T08:50:18.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.260+0000 7f0141ffb640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f013006cbc0 0x7f013006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:18.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.260+0000 7f0141ffb640 1 -- 172.21.15.31:0/392416424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f012c0ac3f0 con 0x7f0144100630 2024-09-16T08:50:18.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.260+0000 7f0143fff640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f013006cbc0 0x7f013006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:18.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.261+0000 7f0143fff640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f013006cbc0 0x7f013006f080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01340058c0 tx=0x7f0134005850 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:18.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.265+0000 7f0141ffb640 1 -- 172.21.15.31:0/392416424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f012c0778a0 con 0x7f0144100630 2024-09-16T08:50:18.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.437+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) v1 -- 0x7f0144108830 con 0x7f0144100630 2024-09-16T08:50:18.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.441+0000 7f0141ffb640 1 -- 172.21.15.31:0/392416424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v22)=0 v22) v1 ==== 117+0+0 (secure 0 0 0) 0x7f012c077020 con 0x7f0144100630 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f013006cbc0 msgr2=0x7f013006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f013006cbc0 0x7f013006f080 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01340058c0 tx=0x7f0134005850 comp rx=0 tx=0).stop 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 msgr2=0x7f01441977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f01441977e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f012c005ec0 tx=0x7f012c004cd0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 shutdown_connections 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f013006cbc0 0x7f013006f080 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0144102f60 0x7f0144197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 --2- 172.21.15.31:0/392416424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0144100630 0x7f01441977e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 >> 172.21.15.31:0/392416424 conn(0x7f01440fa250 msgr2=0x7f01440ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:18.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 shutdown_connections 2024-09-16T08:50:18.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:18.443+0000 7f014b01b640 1 -- 172.21.15.31:0/392416424 wait complete. 2024-09-16T08:50:19.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:19 smithi031 ceph-mon[27844]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:19.262 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:19 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/392416424' entity='client.admin' 2024-09-16T08:50:19.326 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:19 smithi097 ceph-mon[35478]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:19.326 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:19 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/392416424' entity='client.admin' 2024-09-16T08:50:20.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 -- 172.21.15.31:0/3105182616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 msgr2=0x7f6dd0102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:20.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 --2- 172.21.15.31:0/3105182616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd0102b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009880 tx=0x7f6dc002a4b0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 -- 172.21.15.31:0/3105182616 shutdown_connections 2024-09-16T08:50:20.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 --2- 172.21.15.31:0/3105182616 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dd0103070 0x7f6dd0105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 --2- 172.21.15.31:0/3105182616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd0102b30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 -- 172.21.15.31:0/3105182616 >> 172.21.15.31:0/3105182616 conn(0x7f6dd00fa280 msgr2=0x7f6dd00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 -- 172.21.15.31:0/3105182616 shutdown_connections 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.745+0000 7f6dd6357640 1 -- 172.21.15.31:0/3105182616 wait complete. 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dd6357640 1 Processor -- start 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dd6357640 1 -- start start 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dd6357640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd01955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dd6357640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dd0103070 0x7f6dd0195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dd6357640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd0196100 con 0x7f6dd0100740 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dd6357640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd0196270 con 0x7f6dd0103070 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd01955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd01955f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44668/0 (socket says 172.21.15.31:44668) 2024-09-16T08:50:20.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.746+0000 7f6dcffff640 1 -- 172.21.15.31:0/1654164086 learned_addr learned my addr 172.21.15.31:0/1654164086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcf7fe640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dd0103070 0x7f6dd0195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcffff640 1 -- 172.21.15.31:0/1654164086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dd0103070 msgr2=0x7f6dd0195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcffff640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dd0103070 0x7f6dd0195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcffff640 1 -- 172.21.15.31:0/1654164086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc0009560 con 0x7f6dd0100740 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcffff640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd01955f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00099a0 tx=0x7f6dc0004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcd7fa640 1 -- 172.21.15.31:0/1654164086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc0005390 con 0x7f6dd0100740 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcd7fa640 1 -- 172.21.15.31:0/1654164086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f6dc00054f0 con 0x7f6dd0100740 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd019acb0 con 0x7f6dd0100740 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dcd7fa640 1 -- 172.21.15.31:0/1654164086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dc0031a20 con 0x7f6dd0100740 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.747+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd019b1a0 con 0x7f6dd0100740 2024-09-16T08:50:20.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.749+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd01086d0 con 0x7f6dd0100740 2024-09-16T08:50:20.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.749+0000 7f6dcd7fa640 1 -- 172.21.15.31:0/1654164086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f6dc0031c90 con 0x7f6dd0100740 2024-09-16T08:50:20.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.750+0000 7f6dcd7fa640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6db806cbc0 0x7f6db806f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:20.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.750+0000 7f6dcd7fa640 1 -- 172.21.15.31:0/1654164086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f6dc00ac0f0 con 0x7f6dd0100740 2024-09-16T08:50:20.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.750+0000 7f6dcf7fe640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6db806cbc0 0x7f6db806f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:20.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.750+0000 7f6dcf7fe640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6db806cbc0 0x7f6db806f080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0196b10 tx=0x7f6dc400b390 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:20.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.754+0000 7f6dcd7fa640 1 -- 172.21.15.31:0/1654164086 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6dc00776d0 con 0x7f6dd0100740 2024-09-16T08:50:20.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.928+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) v1 -- 0x7f6dd0108860 con 0x7f6dd0100740 2024-09-16T08:50:20.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.932+0000 7f6dcd7fa640 1 -- 172.21.15.31:0/1654164086 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v23)=0 v23) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6dc00ab8b0 con 0x7f6dd0100740 2024-09-16T08:50:20.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6db806cbc0 msgr2=0x7f6db806f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:20.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6db806cbc0 0x7f6db806f080 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0196b10 tx=0x7f6dc400b390 comp rx=0 tx=0).stop 2024-09-16T08:50:20.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 msgr2=0x7f6dd01955f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:20.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd01955f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00099a0 tx=0x7f6dc0004da0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 shutdown_connections 2024-09-16T08:50:20.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6db806cbc0 0x7f6db806f080 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dd0103070 0x7f6dd0195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 --2- 172.21.15.31:0/1654164086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6dd0100740 0x7f6dd01955f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:20.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 >> 172.21.15.31:0/1654164086 conn(0x7f6dd00fa280 msgr2=0x7f6dd00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:20.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 shutdown_connections 2024-09-16T08:50:20.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:20.934+0000 7f6dd6357640 1 -- 172.21.15.31:0/1654164086 wait complete. 2024-09-16T08:50:21.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:21 smithi031 ceph-mon[27844]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:21.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:21 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1654164086' entity='client.admin' 2024-09-16T08:50:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 -- 172.21.15.31:0/4040069970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 msgr2=0x7f512c103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 --2- 172.21.15.31:0/4040069970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c103410 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f511c009880 tx=0x7f511c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 -- 172.21.15.31:0/4040069970 shutdown_connections 2024-09-16T08:50:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 --2- 172.21.15.31:0/4040069970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c103410 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 --2- 172.21.15.31:0/4040069970 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f512c101d90 0x7f512c102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 -- 172.21.15.31:0/4040069970 >> 172.21.15.31:0/4040069970 conn(0x7f512c0fd540 msgr2=0x7f512c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:21.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 -- 172.21.15.31:0/4040069970 shutdown_connections 2024-09-16T08:50:21.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.351+0000 7f5134098640 1 -- 172.21.15.31:0/4040069970 wait complete. 2024-09-16T08:50:21.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5134098640 1 Processor -- start 2024-09-16T08:50:21.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5134098640 1 -- start start 2024-09-16T08:50:21.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5134098640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f512c101d90 0x7f512c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:21.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5134098640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5134098640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c198200 con 0x7f512c102f90 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5134098640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c198370 con 0x7f512c101d90 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131633640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131633640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c197c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44692/0 (socket says 172.21.15.31:44692) 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131633640 1 -- 172.21.15.31:0/2229621453 learned_addr learned my addr 172.21.15.31:0/2229621453 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131e34640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f512c101d90 0x7f512c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131633640 1 -- 172.21.15.31:0/2229621453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f512c101d90 msgr2=0x7f512c1976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131633640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f512c101d90 0x7f512c1976f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131633640 1 -- 172.21.15.31:0/2229621453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c009560 con 0x7f512c102f90 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.352+0000 7f5131e34640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f512c101d90 0x7f512c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.353+0000 7f5131633640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c197c30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f511c0099a0 tx=0x7f511c004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.353+0000 7f5122ffd640 1 -- 172.21.15.31:0/2229621453 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511c005460 con 0x7f512c102f90 2024-09-16T08:50:21.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.353+0000 7f5122ffd640 1 -- 172.21.15.31:0/2229621453 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f511c0055c0 con 0x7f512c102f90 2024-09-16T08:50:21.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.353+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c19cdb0 con 0x7f512c102f90 2024-09-16T08:50:21.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.353+0000 7f5122ffd640 1 -- 172.21.15.31:0/2229621453 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f511c031a70 con 0x7f512c102f90 2024-09-16T08:50:21.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.353+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c19d2a0 con 0x7f512c102f90 2024-09-16T08:50:21.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.354+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c1917d0 con 0x7f512c102f90 2024-09-16T08:50:21.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.355+0000 7f5122ffd640 1 -- 172.21.15.31:0/2229621453 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f511c031c30 con 0x7f512c102f90 2024-09-16T08:50:21.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.355+0000 7f5122ffd640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f511806cbd0 0x7f511806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:21.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.355+0000 7f5122ffd640 1 -- 172.21.15.31:0/2229621453 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f511c0ac2b0 con 0x7f512c102f90 2024-09-16T08:50:21.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.355+0000 7f5131e34640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f511806cbd0 0x7f511806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:21.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.356+0000 7f5131e34640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f511806cbd0 0x7f511806f090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f512c102df0 tx=0x7f5128005e70 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:21.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.359+0000 7f5122ffd640 1 -- 172.21.15.31:0/2229621453 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f511c0777e0 con 0x7f512c102f90 2024-09-16T08:50:21.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:21 smithi097 ceph-mon[35478]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:21.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:21 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1654164086' entity='client.admin' 2024-09-16T08:50:21.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.538+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) v1 -- 0x7f512c10a840 con 0x7f512c102f90 2024-09-16T08:50:21.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.542+0000 7f5122ffd640 1 -- 172.21.15.31:0/2229621453 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v24)=0 v24) v1 ==== 125+0+0 (secure 0 0 0) 0x7f511c04a0c0 con 0x7f512c102f90 2024-09-16T08:50:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f511806cbd0 msgr2=0x7f511806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f511806cbd0 0x7f511806f090 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f512c102df0 tx=0x7f5128005e70 comp rx=0 tx=0).stop 2024-09-16T08:50:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 msgr2=0x7f512c197c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c197c30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f511c0099a0 tx=0x7f511c004da0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 shutdown_connections 2024-09-16T08:50:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f511806cbd0 0x7f511806f090 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f512c102f90 0x7f512c197c30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 --2- 172.21.15.31:0/2229621453 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f512c101d90 0x7f512c1976f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 >> 172.21.15.31:0/2229621453 conn(0x7f512c0fd540 msgr2=0x7f512c1066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.544+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 shutdown_connections 2024-09-16T08:50:21.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:21.545+0000 7f5134098640 1 -- 172.21.15.31:0/2229621453 wait complete. 2024-09-16T08:50:22.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 -- 172.21.15.31:0/4144135988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d40ffb90 msgr2=0x7f37d40fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:22.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/4144135988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d40ffb90 0x7f37d40fff90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009880 tx=0x7f37bc02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 -- 172.21.15.31:0/4144135988 shutdown_connections 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/4144135988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37d4100d90 0x7f37d4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/4144135988 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d40ffb90 0x7f37d40fff90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 -- 172.21.15.31:0/4144135988 >> 172.21.15.31:0/4144135988 conn(0x7f37d40fb320 msgr2=0x7f37d40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 -- 172.21.15.31:0/4144135988 shutdown_connections 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.603+0000 7f37d9bbc640 1 -- 172.21.15.31:0/4144135988 wait complete. 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d9bbc640 1 Processor -- start 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d9bbc640 1 -- start start 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d9bbc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37d40ffb90 0x7f37d41977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d9bbc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d4100d90 0x7f37d4197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d9bbc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d41982f0 con 0x7f37d4100d90 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d9bbc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4198460 con 0x7f37d40ffb90 2024-09-16T08:50:22.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d4100d90 0x7f37d4197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d4100d90 0x7f37d4197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58300/0 (socket says 172.21.15.31:58300) 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d2ffd640 1 -- 172.21.15.31:0/2980556501 learned_addr learned my addr 172.21.15.31:0/2980556501 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.604+0000 7f37d37fe640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37d40ffb90 0x7f37d41977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d2ffd640 1 -- 172.21.15.31:0/2980556501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37d40ffb90 msgr2=0x7f37d41977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d2ffd640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37d40ffb90 0x7f37d41977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d2ffd640 1 -- 172.21.15.31:0/2980556501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc009560 con 0x7f37d4100d90 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d2ffd640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d4100d90 0x7f37d4197d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37c400b450 tx=0x7f37c400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d0ff9640 1 -- 172.21.15.31:0/2980556501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37c4011860 con 0x7f37d4100d90 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d419cf00 con 0x7f37d4100d90 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d0ff9640 1 -- 172.21.15.31:0/2980556501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f37c4011e60 con 0x7f37d4100d90 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d0ff9640 1 -- 172.21.15.31:0/2980556501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37c400f340 con 0x7f37d4100d90 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.605+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d419d4a0 con 0x7f37d4100d90 2024-09-16T08:50:22.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.606+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d41052a0 con 0x7f37d4100d90 2024-09-16T08:50:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.607+0000 7f37d0ff9640 1 -- 172.21.15.31:0/2980556501 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f37c40103e0 con 0x7f37d4100d90 2024-09-16T08:50:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.607+0000 7f37d0ff9640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f37c006cbd0 0x7f37c006f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.607+0000 7f37d0ff9640 1 -- 172.21.15.31:0/2980556501 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f37c408b010 con 0x7f37d4100d90 2024-09-16T08:50:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.607+0000 7f37d37fe640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f37c006cbd0 0x7f37c006f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.608+0000 7f37d37fe640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f37c006cbd0 0x7f37c006f090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000c00 tx=0x7f37bc031400 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:22.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.612+0000 7f37d0ff9640 1 -- 172.21.15.31:0/2980556501 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f37c40564c0 con 0x7f37d4100d90 2024-09-16T08:50:22.720 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:22 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2229621453' entity='client.admin' 2024-09-16T08:50:22.720 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:22 smithi031 ceph-mon[27844]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:22.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.786+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) v1 -- 0x7f37d402c4b0 con 0x7f37d4100d90 2024-09-16T08:50:22.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.790+0000 7f37d0ff9640 1 -- 172.21.15.31:0/2980556501 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v25)=0 v25) v1 ==== 121+0+0 (secure 0 0 0) 0x7f37c408a7d0 con 0x7f37d4100d90 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f37c006cbd0 msgr2=0x7f37c006f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f37c006cbd0 0x7f37c006f090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000c00 tx=0x7f37bc031400 comp rx=0 tx=0).stop 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d4100d90 msgr2=0x7f37d4197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d4100d90 0x7f37d4197d20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f37c400b450 tx=0x7f37c400b910 comp rx=0 tx=0).stop 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 shutdown_connections 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f37c006cbd0 0x7f37c006f090 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37d4100d90 0x7f37d4197d20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 --2- 172.21.15.31:0/2980556501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37d40ffb90 0x7f37d41977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:22.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 >> 172.21.15.31:0/2980556501 conn(0x7f37d40fb320 msgr2=0x7f37d4102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:22.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 shutdown_connections 2024-09-16T08:50:22.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:22.792+0000 7f37d9bbc640 1 -- 172.21.15.31:0/2980556501 wait complete. 2024-09-16T08:50:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:22 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2229621453' entity='client.admin' 2024-09-16T08:50:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:22 smithi097 ceph-mon[35478]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:24.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:23 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2980556501' entity='client.admin' 2024-09-16T08:50:24.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:24.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:23 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2980556501' entity='client.admin' 2024-09-16T08:50:24.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:23 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:25.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:24 smithi031 ceph-mon[27844]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:25.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 -- 172.21.15.31:0/2820239921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fd910 msgr2=0x7f59440fdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:25.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 --2- 172.21.15.31:0/2820239921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fd910 0x7f59440fdd10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5934009880 tx=0x7f593402a4d0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 -- 172.21.15.31:0/2820239921 shutdown_connections 2024-09-16T08:50:25.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 --2- 172.21.15.31:0/2820239921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440fe250 0x7f5944105280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 --2- 172.21.15.31:0/2820239921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fd910 0x7f59440fdd10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 -- 172.21.15.31:0/2820239921 >> 172.21.15.31:0/2820239921 conn(0x7f59440f9540 msgr2=0x7f59440fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 -- 172.21.15.31:0/2820239921 shutdown_connections 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.084+0000 7f5949f6e640 1 -- 172.21.15.31:0/2820239921 wait complete. 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.085+0000 7f5949f6e640 1 Processor -- start 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.085+0000 7f5949f6e640 1 -- start start 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.085+0000 7f5949f6e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440fd910 0x7f5944100bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.085+0000 7f5949f6e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fe250 0x7f59440ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.085+0000 7f5949f6e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440ff750 con 0x7f59440fe250 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.085+0000 7f5949f6e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440ff8c0 con 0x7f59440fd910 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5942ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fe250 0x7f59440ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5942ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fe250 0x7f59440ff210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58322/0 (socket says 172.21.15.31:58322) 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5942ffd640 1 -- 172.21.15.31:0/3911241402 learned_addr learned my addr 172.21.15.31:0/3911241402 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f59437fe640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440fd910 0x7f5944100bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:25.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5942ffd640 1 -- 172.21.15.31:0/3911241402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440fd910 msgr2=0x7f5944100bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5942ffd640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440fd910 0x7f5944100bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5942ffd640 1 -- 172.21.15.31:0/3911241402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934009560 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f59437fe640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440fd910 0x7f5944100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5942ffd640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fe250 0x7f59440ff210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5938006fd0 tx=0x7f593800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5938011800 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(26 keys) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f5938011e00 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59440ffba0 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f593800f2e0 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.086+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594419d5b0 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.087+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59440fdd10 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.088+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f5938010380 con 0x7f59440fe250 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.088+0000 7f5940ff9640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f592c06cbf0 0x7f592c06f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:25.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.089+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f593808ae20 con 0x7f59440fe250 2024-09-16T08:50:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.089+0000 7f59437fe640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f592c06cbf0 0x7f592c06f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:25.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.089+0000 7f59437fe640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f592c06cbf0 0x7f592c06f0b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5934006010 tx=0x7f59340043c0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:25.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.093+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f59380562d0 con 0x7f59440fe250 2024-09-16T08:50:25.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:24 smithi097 ceph-mon[35478]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:25.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.266+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) v1 -- 0x7f59441086d0 con 0x7f59440fe250 2024-09-16T08:50:25.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.270+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 7 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59380562d0 con 0x7f59440fe250 2024-09-16T08:50:25.272 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.270+0000 7f5940ff9640 1 -- 172.21.15.31:0/3911241402 <== mon.0 v2:172.21.15.31:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v26)=0 v26) v1 ==== 123+0+0 (secure 0 0 0) 0x7f593808a5e0 con 0x7f59440fe250 2024-09-16T08:50:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f592c06cbf0 msgr2=0x7f592c06f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f592c06cbf0 0x7f592c06f0b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5934006010 tx=0x7f59340043c0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fe250 msgr2=0x7f59440ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:25.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fe250 0x7f59440ff210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5938006fd0 tx=0x7f593800b860 comp rx=0 tx=0).stop 2024-09-16T08:50:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 shutdown_connections 2024-09-16T08:50:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f592c06cbf0 0x7f592c06f0b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59440fe250 0x7f59440ff210 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 --2- 172.21.15.31:0/3911241402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59440fd910 0x7f5944100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.272+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 >> 172.21.15.31:0/3911241402 conn(0x7f59440f9540 msgr2=0x7f59441035d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.273+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 shutdown_connections 2024-09-16T08:50:25.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:25.273+0000 7f5949f6e640 1 -- 172.21.15.31:0/3911241402 wait complete. 2024-09-16T08:50:26.349 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi031') 2024-09-16T08:50:26.349 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi031') 2024-09-16T08:50:26.350 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi031') 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.524+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/319733677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c28103010 msgr2=0x7f0c28105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.524+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/319733677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c28103010 0x7f0c28105400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009880 tx=0x7f0c1c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.524+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/319733677 shutdown_connections 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.524+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/319733677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c28103010 0x7f0c28105400 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.524+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/319733677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c281006e0 0x7f0c28102ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.524+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/319733677 >> 172.21.15.31:0/319733677 conn(0x7f0c280fa280 msgr2=0x7f0c280fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.524+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/319733677 shutdown_connections 2024-09-16T08:50:26.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.525+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/319733677 wait complete. 2024-09-16T08:50:26.528 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:26 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3911241402' entity='client.admin' 2024-09-16T08:50:26.528 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:26 smithi031 ceph-mon[27844]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:26.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2cfd5640 1 Processor -- start 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2cfd5640 1 -- start start 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2cfd5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c281006e0 0x7f0c281956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2cfd5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28103010 0x7f0c28195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2cfd5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c28196130 con 0x7f0c281006e0 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2cfd5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c281962a0 con 0x7f0c28103010 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c281006e0 0x7f0c281956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c281006e0 0x7f0c281956b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58340/0 (socket says 172.21.15.31:58340) 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2659c640 1 -- 172.21.15.31:0/1105904776 learned_addr learned my addr 172.21.15.31:0/1105904776 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c25d9b640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28103010 0x7f0c28195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2659c640 1 -- 172.21.15.31:0/1105904776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28103010 msgr2=0x7f0c28195bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:26.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.526+0000 7f0c2659c640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28103010 0x7f0c28195bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.527+0000 7f0c2659c640 1 -- 172.21.15.31:0/1105904776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c009560 con 0x7f0c281006e0 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.527+0000 7f0c2659c640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c281006e0 0x7f0c281956b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800c8a0 tx=0x7f0c1800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.527+0000 7f0c177fe640 1 -- 172.21.15.31:0/1105904776 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c18007b80 con 0x7f0c281006e0 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.527+0000 7f0c177fe640 1 -- 172.21.15.31:0/1105904776 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c18007ce0 con 0x7f0c281006e0 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.527+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2819ad30 con 0x7f0c281006e0 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.527+0000 7f0c177fe640 1 -- 172.21.15.31:0/1105904776 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c18018810 con 0x7f0c281006e0 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.527+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2819b1b0 con 0x7f0c281006e0 2024-09-16T08:50:26.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.528+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c28108770 con 0x7f0c281006e0 2024-09-16T08:50:26.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.529+0000 7f0c177fe640 1 -- 172.21.15.31:0/1105904776 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f0c18018970 con 0x7f0c281006e0 2024-09-16T08:50:26.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.529+0000 7f0c177fe640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0c1006cbc0 0x7f0c1006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:26.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.529+0000 7f0c177fe640 1 -- 172.21.15.31:0/1105904776 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f0c1808b1d0 con 0x7f0c281006e0 2024-09-16T08:50:26.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.529+0000 7f0c25d9b640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0c1006cbc0 0x7f0c1006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:26.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.530+0000 7f0c25d9b640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0c1006cbc0 0x7f0c1006f080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c0314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:26.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.533+0000 7f0c177fe640 1 -- 172.21.15.31:0/1105904776 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0c18056700 con 0x7f0c281006e0 2024-09-16T08:50:26.710 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.709+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) v1 -- 0x7f0c2819b450 con 0x7f0c281006e0 2024-09-16T08:50:26.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.713+0000 7f0c177fe640 1 -- 172.21.15.31:0/1105904776 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v27)=0 v27) v1 ==== 106+0+0 (secure 0 0 0) 0x7f0c1808a990 con 0x7f0c281006e0 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0c1006cbc0 msgr2=0x7f0c1006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0c1006cbc0 0x7f0c1006f080 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c006010 tx=0x7f0c1c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c281006e0 msgr2=0x7f0c281956b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c281006e0 0x7f0c281956b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800c8a0 tx=0x7f0c1800cd60 comp rx=0 tx=0).stop 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 shutdown_connections 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0c1006cbc0 0x7f0c1006f080 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c28103010 0x7f0c28195bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 --2- 172.21.15.31:0/1105904776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c281006e0 0x7f0c281956b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 >> 172.21.15.31:0/1105904776 conn(0x7f0c280fa280 msgr2=0x7f0c280fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:26.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 shutdown_connections 2024-09-16T08:50:26.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:26.715+0000 7f0c2cfd5640 1 -- 172.21.15.31:0/1105904776 wait complete. 2024-09-16T08:50:26.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:26 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3911241402' entity='client.admin' 2024-09-16T08:50:26.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:26 smithi097 ceph-mon[35478]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:27.785 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:27 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1105904776' entity='client.admin' 2024-09-16T08:50:28.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:27 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1105904776' entity='client.admin' 2024-09-16T08:50:28.061 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi031') 2024-09-16T08:50:28.061 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi031') 2024-09-16T08:50:28.062 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-16T08:50:28.062 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi031') 2024-09-16T08:50:28.062 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-16T08:50:28.797 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:28 smithi031 ceph-mon[27844]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:28.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 -- 172.21.15.31:0/4129735099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 msgr2=0x7f9bcc0fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 --2- 172.21.15.31:0/4129735099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc0fff80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009880 tx=0x7f9bc802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 -- 172.21.15.31:0/4129735099 shutdown_connections 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 --2- 172.21.15.31:0/4129735099 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bcc100d80 0x7f9bcc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 --2- 172.21.15.31:0/4129735099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc0fff80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 -- 172.21.15.31:0/4129735099 >> 172.21.15.31:0/4129735099 conn(0x7f9bcc0fb2f0 msgr2=0x7f9bcc0fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 -- 172.21.15.31:0/4129735099 shutdown_connections 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.796+0000 7f9bd375d640 1 -- 172.21.15.31:0/4129735099 wait complete. 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd375d640 1 Processor -- start 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd375d640 1 -- start start 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd375d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:28.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd375d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bcc100d80 0x7f9bcc195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd375d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc196250 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd375d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1963c0 con 0x7f9bcc100d80 2024-09-16T08:50:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd14f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd14f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:58362/0 (socket says 172.21.15.31:58362) 2024-09-16T08:50:28.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd14f9640 1 -- 172.21.15.31:0/3850903067 learned_addr learned my addr 172.21.15.31:0/3850903067 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd0cf8640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bcc100d80 0x7f9bcc195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd14f9640 1 -- 172.21.15.31:0/3850903067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bcc100d80 msgr2=0x7f9bcc195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd14f9640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bcc100d80 0x7f9bcc195c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd14f9640 1 -- 172.21.15.31:0/3850903067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8009560 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.797+0000 7f9bd14f9640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc195740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005610 tx=0x7f9bc8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.798+0000 7f9bc27fc640 1 -- 172.21.15.31:0/3850903067 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc8004ba0 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.828 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.798+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc19ae00 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.798+0000 7f9bc27fc640 1 -- 172.21.15.31:0/3850903067 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bc8004d00 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.798+0000 7f9bc27fc640 1 -- 172.21.15.31:0/3850903067 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc8003bd0 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.798+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc19b2f0 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.799+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc100000 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.799+0000 7f9bc27fc640 1 -- 172.21.15.31:0/3850903067 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f9bc8031640 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.800+0000 7f9bc27fc640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9bb806cbd0 0x7f9bb806f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.800+0000 7f9bc27fc640 1 -- 172.21.15.31:0/3850903067 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f9bc80ac490 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.800+0000 7f9bd0cf8640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9bb806cbd0 0x7f9bb806f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.800+0000 7f9bd0cf8640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9bb806cbd0 0x7f9bb806f090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc196c60 tx=0x7f9bbc008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:28.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.804+0000 7f9bc27fc640 1 -- 172.21.15.31:0/3850903067 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9bc8077940 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.989+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) v1 -- 0x7f9bcc19b510 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.993+0000 7f9bc27fc640 1 -- 172.21.15.31:0/3850903067 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) v1 ==== 121+0+0 (secure 0 0 0) 0x7f9bc80abc50 con 0x7f9bcc0ffb80 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9bb806cbd0 msgr2=0x7f9bb806f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9bb806cbd0 0x7f9bb806f090 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc196c60 tx=0x7f9bbc008040 comp rx=0 tx=0).stop 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 msgr2=0x7f9bcc195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc195740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8005610 tx=0x7f9bc8005640 comp rx=0 tx=0).stop 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 shutdown_connections 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9bb806cbd0 0x7f9bb806f090 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9bcc100d80 0x7f9bcc195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 --2- 172.21.15.31:0/3850903067 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9bcc0ffb80 0x7f9bcc195740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 >> 172.21.15.31:0/3850903067 conn(0x7f9bcc0fb2f0 msgr2=0x7f9bcc103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 shutdown_connections 2024-09-16T08:50:28.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:28.995+0000 7f9bd375d640 1 -- 172.21.15.31:0/3850903067 wait complete. 2024-09-16T08:50:29.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:28 smithi097 ceph-mon[35478]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:29.684 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-16T08:50:29.684 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi031') 2024-09-16T08:50:29.684 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi031') 2024-09-16T08:50:29.685 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi031') 2024-09-16T08:50:29.685 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay.ceph.io/ceph-ci/ceph:reef', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1631ad7a-7408-11ef-bceb-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi031') 2024-09-16T08:50:29.685 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T08:50:29.685 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:50:29.685 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T08:50:29.704 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T08:50:29.704 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-16T08:50:29.760 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 08:49:01.395274265 +0000 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 08:43:29.695223828 +0000 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 08:43:29.695223828 +0000 2024-09-16T08:50:29.761 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 08:43:29.695223828 +0000 2024-09-16T08:50:29.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T08:50:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T08:50:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T08:50:29.828 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000372081 s, 1.4 MB/s 2024-09-16T08:50:29.830 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T08:50:29.887 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-16T08:50:29.944 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T08:50:29.944 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:29.944 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-16T08:50:29.944 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:29.945 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:29.945 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 08:49:01.396274248 +0000 2024-09-16T08:50:29.945 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 08:43:30.025218362 +0000 2024-09-16T08:50:29.945 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 08:43:30.025218362 +0000 2024-09-16T08:50:29.945 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 08:43:30.025218362 +0000 2024-09-16T08:50:29.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T08:50:30.010 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T08:50:30.011 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T08:50:30.011 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000196998 s, 2.6 MB/s 2024-09-16T08:50:30.012 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T08:50:30.070 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 08:49:01.396274248 +0000 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 08:43:30.356212881 +0000 2024-09-16T08:50:30.127 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 08:43:30.356212881 +0000 2024-09-16T08:50:30.128 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 08:43:30.356212881 +0000 2024-09-16T08:50:30.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T08:50:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T08:50:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T08:50:30.199 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000198081 s, 2.6 MB/s 2024-09-16T08:50:30.201 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T08:50:30.258 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-16T08:50:30.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:29 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3850903067' entity='client.admin' 2024-09-16T08:50:30.314 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T08:50:30.314 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:30.315 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-16T08:50:30.315 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:30.315 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:30.315 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-16 08:49:01.396274248 +0000 2024-09-16T08:50:30.315 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 08:43:30.691207332 +0000 2024-09-16T08:50:30.315 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 08:43:30.691207332 +0000 2024-09-16T08:50:30.315 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 08:43:30.691207332 +0000 2024-09-16T08:50:30.315 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T08:50:30.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:29 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3850903067' entity='client.admin' 2024-09-16T08:50:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-16T08:50:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-16T08:50:30.378 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000185563 s, 2.8 MB/s 2024-09-16T08:50:30.379 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T08:50:30.436 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:50:30.436 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T08:50:30.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T08:50:30.457 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-09-16T08:50:30.516 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T08:50:30.516 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:30.516 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T08:50:30.517 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:30.517 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:30.517 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-16 08:49:33.393369725 +0000 2024-09-16T08:50:30.517 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-16 08:43:29.595650498 +0000 2024-09-16T08:50:30.517 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-16 08:43:29.595650498 +0000 2024-09-16T08:50:30.517 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-16 08:43:29.595650498 +0000 2024-09-16T08:50:30.517 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T08:50:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-16T08:50:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-16T08:50:30.586 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000212164 s, 2.4 MB/s 2024-09-16T08:50:30.588 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T08:50:30.648 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-09-16T08:50:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T08:50:30.704 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:30.705 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T08:50:30.705 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:30.705 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:30.705 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-16 08:49:33.394369709 +0000 2024-09-16T08:50:30.705 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-16 08:43:29.940645245 +0000 2024-09-16T08:50:30.705 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-16 08:43:29.940645245 +0000 2024-09-16T08:50:30.705 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-16 08:43:29.940645245 +0000 2024-09-16T08:50:30.705 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T08:50:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-16T08:50:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-16T08:50:30.771 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000207959 s, 2.5 MB/s 2024-09-16T08:50:30.772 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T08:50:30.830 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-09-16T08:50:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T08:50:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:30.889 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T08:50:30.889 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:30.890 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:30.890 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-16 08:49:33.394369709 +0000 2024-09-16T08:50:30.890 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-16 08:43:30.292639886 +0000 2024-09-16T08:50:30.890 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-16 08:43:30.292639886 +0000 2024-09-16T08:50:30.890 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-16 08:43:30.292639886 +0000 2024-09-16T08:50:30.890 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T08:50:30.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-16T08:50:30.954 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-16T08:50:30.954 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000182787 s, 2.8 MB/s 2024-09-16T08:50:30.955 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T08:50:31.013 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-09-16T08:50:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T08:50:31.069 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T08:50:31.069 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T08:50:31.069 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:50:31.069 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T08:50:31.069 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-09-16 08:49:33.394369709 +0000 2024-09-16T08:50:31.069 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-16 08:43:30.618634923 +0000 2024-09-16T08:50:31.070 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-16 08:43:30.618634923 +0000 2024-09-16T08:50:31.070 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-16 08:43:30.618634923 +0000 2024-09-16T08:50:31.070 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T08:50:31.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:30 smithi031 ceph-mon[27844]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:31.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-09-16T08:50:31.137 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-09-16T08:50:31.137 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000197369 s, 2.6 MB/s 2024-09-16T08:50:31.138 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T08:50:31.195 INFO:tasks.cephadm:Deploying osd.0 on smithi031 with /dev/vg_nvme/lv_4... 2024-09-16T08:50:31.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T08:50:31.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:30 smithi097 ceph-mon[35478]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:31.662 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:50:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:32 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:32 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:32 smithi031 ceph-mon[27844]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-09-16T08:50:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:32 smithi031 ceph-mon[27844]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-09-16T08:50:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:32 smithi031 ceph-mon[27844]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:33.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:32 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:33.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:32 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:33.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:32 smithi097 ceph-mon[35478]: Reconfiguring grafana.smithi031 (dependencies changed)... 2024-09-16T08:50:33.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:32 smithi097 ceph-mon[35478]: Reconfiguring daemon grafana.smithi031 on smithi031 2024-09-16T08:50:33.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:32 smithi097 ceph-mon[35478]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:35.241 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:35 smithi031 ceph-mon[27844]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:35.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:35 smithi097 ceph-mon[35478]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:37.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:37 smithi031 ceph-mon[27844]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:37.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:37.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:37.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:37 smithi097 ceph-mon[35478]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:38.162 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:50:38.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_4 2024-09-16T08:50:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:38 smithi031 ceph-mon[27844]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-09-16T08:50:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:38 smithi031 ceph-mon[27844]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-09-16T08:50:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:38 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:38 smithi097 ceph-mon[35478]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-09-16T08:50:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:38 smithi097 ceph-mon[35478]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-09-16T08:50:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:38 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:39.015 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:50:39.366 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:39 smithi097 ceph-mon[35478]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:39.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:39 smithi031 ceph-mon[27844]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:41.368 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:41 smithi031 ceph-mon[27844]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:41.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:41 smithi097 ceph-mon[35478]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:42.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.410+0000 7f45be991640 1 -- 172.21.15.31:0/2994397369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 msgr2=0x7f45b80fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.410+0000 7f45be991640 1 --2- 172.21.15.31:0/2994397369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b80fff90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009880 tx=0x7f45a802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.410+0000 7f45be991640 1 -- 172.21.15.31:0/2994397369 shutdown_connections 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.410+0000 7f45be991640 1 --2- 172.21.15.31:0/2994397369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8100d90 0x7f45b8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.410+0000 7f45be991640 1 --2- 172.21.15.31:0/2994397369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b80fff90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.410+0000 7f45be991640 1 -- 172.21.15.31:0/2994397369 >> 172.21.15.31:0/2994397369 conn(0x7f45b80fb360 msgr2=0x7f45b80fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.411+0000 7f45be991640 1 -- 172.21.15.31:0/2994397369 shutdown_connections 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.411+0000 7f45be991640 1 -- 172.21.15.31:0/2994397369 wait complete. 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.411+0000 7f45be991640 1 Processor -- start 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.411+0000 7f45be991640 1 -- start start 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45be991640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b8197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45be991640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8100d90 0x7f45b8197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:42.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45be991640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8198390 con 0x7f45b80ffb90 2024-09-16T08:50:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45be991640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8198500 con 0x7f45b8100d90 2024-09-16T08:50:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b8197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b8197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39466/0 (socket says 172.21.15.31:39466) 2024-09-16T08:50:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b7fff640 1 -- 172.21.15.31:0/2587138598 learned_addr learned my addr 172.21.15.31:0/2587138598 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:50:42.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b77fe640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8100d90 0x7f45b8197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b7fff640 1 -- 172.21.15.31:0/2587138598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8100d90 msgr2=0x7f45b8197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:50:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b7fff640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8100d90 0x7f45b8197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:50:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b7fff640 1 -- 172.21.15.31:0/2587138598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8009560 con 0x7f45b80ffb90 2024-09-16T08:50:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b77fe640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8100d90 0x7f45b8197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:50:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.412+0000 7f45b7fff640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b8197880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005610 tx=0x7f45a8005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:42.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.413+0000 7f45b57fa640 1 -- 172.21.15.31:0/2587138598 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a8004ba0 con 0x7f45b80ffb90 2024-09-16T08:50:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.413+0000 7f45b57fa640 1 -- 172.21.15.31:0/2587138598 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45a8004d00 con 0x7f45b80ffb90 2024-09-16T08:50:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.413+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b819cef0 con 0x7f45b80ffb90 2024-09-16T08:50:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.413+0000 7f45b57fa640 1 -- 172.21.15.31:0/2587138598 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45a8003bf0 con 0x7f45b80ffb90 2024-09-16T08:50:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.413+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b819d3e0 con 0x7f45b80ffb90 2024-09-16T08:50:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.414+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b81052f0 con 0x7f45b80ffb90 2024-09-16T08:50:42.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.414+0000 7f45b57fa640 1 -- 172.21.15.31:0/2587138598 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f45a8031640 con 0x7f45b80ffb90 2024-09-16T08:50:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.415+0000 7f45b57fa640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f45a006cbc0 0x7f45a006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:50:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.415+0000 7f45b57fa640 1 -- 172.21.15.31:0/2587138598 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f45a80ac310 con 0x7f45b80ffb90 2024-09-16T08:50:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.415+0000 7f45b77fe640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f45a006cbc0 0x7f45a006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:50:42.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.415+0000 7f45b77fe640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f45a006cbc0 0x7f45a006f080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45b8198d50 tx=0x7f45ac00a250 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:50:42.420 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.419+0000 7f45b57fa640 1 -- 172.21.15.31:0/2587138598 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f45a80778f0 con 0x7f45b80ffb90 2024-09-16T08:50:42.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:50:42.620+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f45b802c4f0 con 0x7f45a006cbc0 2024-09-16T08:50:43.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:43 smithi031 ceph-mon[27844]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:43.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:50:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:50:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:43.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:43 smithi097 ceph-mon[35478]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:43.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:50:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:50:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:44 smithi031 ceph-mon[27844]: from='client.14318 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:50:44.376 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:44 smithi097 ceph-mon[35478]: from='client.14318 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:50:45.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:45 smithi031 ceph-mon[27844]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:50:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:45.423 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:45 smithi097 ceph-mon[35478]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:45.423 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:45.423 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:45.423 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:50:45.423 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:46.359 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:46 smithi031 ceph-mon[27844]: Reconfiguring ceph-exporter.smithi097 (monmap changed)... 2024-09-16T08:50:46.359 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:46 smithi031 ceph-mon[27844]: Reconfiguring daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:50:46.431 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:46 smithi097 ceph-mon[35478]: Reconfiguring ceph-exporter.smithi097 (monmap changed)... 2024-09-16T08:50:46.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:46 smithi097 ceph-mon[35478]: Reconfiguring daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/199551212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65c1cf82-b6da-4c51-b5b5-6fa8d6662a75"}]: dispatch 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/199551212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65c1cf82-b6da-4c51-b5b5-6fa8d6662a75"}]': finished 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:50:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:47.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:47 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/168801518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:50:47.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:47.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/199551212' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "65c1cf82-b6da-4c51-b5b5-6fa8d6662a75"}]: dispatch 2024-09-16T08:50:47.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/199551212' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "65c1cf82-b6da-4c51-b5b5-6fa8d6662a75"}]': finished 2024-09-16T08:50:47.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T08:50:47.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:50:47.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:47.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:47.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-16T08:50:47.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:47.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:47 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/168801518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:50:48.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:48 smithi097 ceph-mon[35478]: Reconfiguring crash.smithi097 (monmap changed)... 2024-09-16T08:50:48.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:48 smithi097 ceph-mon[35478]: Reconfiguring daemon crash.smithi097 on smithi097 2024-09-16T08:50:48.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:48 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:48.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:48 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:48.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:48 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:50:48.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:48 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:50:48.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:48 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:48 smithi031 ceph-mon[27844]: Reconfiguring crash.smithi097 (monmap changed)... 2024-09-16T08:50:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:48 smithi031 ceph-mon[27844]: Reconfiguring daemon crash.smithi097 on smithi097 2024-09-16T08:50:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:48 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:48 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:48 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:50:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:48 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:50:48.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:48 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: Reconfiguring mgr.smithi097.mpashf (monmap changed)... 2024-09-16T08:50:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: Reconfiguring daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:50:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:50:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:50:49.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: Reconfiguring mgr.smithi097.mpashf (monmap changed)... 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: Reconfiguring daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:50:49.387 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:50 smithi031 ceph-mon[27844]: Reconfiguring mon.smithi097 (monmap changed)... 2024-09-16T08:50:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:50 smithi031 ceph-mon[27844]: Reconfiguring daemon mon.smithi097 on smithi097 2024-09-16T08:50:50.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:50 smithi097 ceph-mon[35478]: Reconfiguring mon.smithi097 (monmap changed)... 2024-09-16T08:50:50.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:50 smithi097 ceph-mon[35478]: Reconfiguring daemon mon.smithi097 on smithi097 2024-09-16T08:50:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:50:51.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:50:51.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:50:51.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:50:51.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:50:51.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:50:51.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:50:51.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:50:51.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.376 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:50:51.376 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:50:51.376 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:50:51.376 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:50:51.376 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.377 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:50:51.473 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:51.473 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.473 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi031.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi031.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:50:51.474 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:50:51.475 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:50:51.475 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi031.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T08:50:51.475 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:51.475 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:50:53.058 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:53.059 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:53.059 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:52 smithi031 ceph-mon[27844]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:53.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:53.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:53.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:52 smithi097 ceph-mon[35478]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:54.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:54.342 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:50:55.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:54 smithi097 ceph-mon[35478]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:55.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:54 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:50:55.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:54 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:55.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:54 smithi031 ceph-mon[27844]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:55.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:54 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T08:50:55.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:54 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:56.259 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:55 smithi031 ceph-mon[27844]: Deploying daemon osd.0 on smithi031 2024-09-16T08:50:56.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:55 smithi097 ceph-mon[35478]: Deploying daemon osd.0 on smithi031 2024-09-16T08:50:57.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:56 smithi097 ceph-mon[35478]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:57.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:56 smithi031 ceph-mon[27844]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:58.117 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:57 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:58.118 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:57 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:50:58.118 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:57 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:58.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:57 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:50:58.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:57 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:50:58.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:57 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:50:59.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:50:58 smithi031 ceph-mon[27844]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:50:59.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:50:58 smithi097 ceph-mon[35478]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:01.244 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:01 smithi031 ceph-mon[27844]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:01.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:01 smithi097 ceph-mon[35478]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:02.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:02.359 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:02.359 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:02.359 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:03.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:03 smithi097 ceph-mon[35478]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:03.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:03 smithi031 ceph-mon[27844]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:04.506 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:04 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:04.507 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:04 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:04.507 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:04 smithi031 ceph-mon[27844]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:04.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:04 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:04.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:04 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:04.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:04 smithi097 ceph-mon[35478]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:05.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:05 smithi031 ceph-mon[27844]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:51:05.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.657+0000 7f45b57fa640 1 -- 172.21.15.31:0/2587138598 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f45b802c4f0 con 0x7f45a006cbc0 2024-09-16T08:51:05.658 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 0 on host 'smithi031' 2024-09-16T08:51:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.659+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f45a006cbc0 msgr2=0x7f45a006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.659+0000 7f45be991640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f45a006cbc0 0x7f45a006f080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f45b8198d50 tx=0x7f45ac00a250 comp rx=0 tx=0).stop 2024-09-16T08:51:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.659+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 msgr2=0x7f45b8197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:05.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.659+0000 7f45be991640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b8197880 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f45a8005610 tx=0x7f45a8005640 comp rx=0 tx=0).stop 2024-09-16T08:51:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.660+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 shutdown_connections 2024-09-16T08:51:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.660+0000 7f45be991640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f45a006cbc0 0x7f45a006f080 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.660+0000 7f45be991640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45b8100d90 0x7f45b8197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.660+0000 7f45be991640 1 --2- 172.21.15.31:0/2587138598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f45b80ffb90 0x7f45b8197880 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.660+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 >> 172.21.15.31:0/2587138598 conn(0x7f45b80fb360 msgr2=0x7f45b81024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:51:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.660+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 shutdown_connections 2024-09-16T08:51:05.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:05.660+0000 7f45be991640 1 -- 172.21.15.31:0/2587138598 wait complete. 2024-09-16T08:51:06.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:05 smithi097 ceph-mon[35478]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T08:51:06.288 DEBUG:teuthology.orchestra.run.smithi031:osd.0> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T08:51:06.291 INFO:tasks.cephadm:Deploying osd.1 on smithi031 with /dev/vg_nvme/lv_3... 2024-09-16T08:51:06.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T08:51:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-mon[27844]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:51:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-mon[27844]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:51:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-mon[27844]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T08:51:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-mon[27844]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:06.873 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[49733]: 2024-09-16T08:51:06.597+0000 7f2e611f1640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T08:51:06.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:51:06 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[49733]: 2024-09-16T08:51:06.600+0000 7f2e5d82c640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:51:06.999 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:51:07.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:06 smithi097 ceph-mon[35478]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T08:51:07.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:06 smithi097 ceph-mon[35478]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T08:51:07.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:06 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:06 smithi097 ceph-mon[35478]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T08:51:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:06 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:06 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:06 smithi097 ceph-mon[35478]: pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:07.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:07 smithi031 ceph-mon[27844]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T08:51:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:07 smithi031 ceph-mon[27844]: osdmap e8: 1 total, 0 up, 1 in 2024-09-16T08:51:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:08.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:07 smithi097 ceph-mon[35478]: from='osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T08:51:08.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:07 smithi097 ceph-mon[35478]: osdmap e8: 1 total, 0 up, 1 in 2024-09-16T08:51:08.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:08.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: purged_snaps scrub starts 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: purged_snaps scrub ok 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] boot 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:09.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: purged_snaps scrub starts 2024-09-16T08:51:09.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: purged_snaps scrub ok 2024-09-16T08:51:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: osd.0 [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] boot 2024-09-16T08:51:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T08:51:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:51:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T08:51:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:09.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:09 smithi097 ceph-mon[35478]: osdmap e10: 1 total, 1 up, 1 in 2024-09-16T08:51:09.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:09 smithi031 ceph-mon[27844]: osdmap e10: 1 total, 1 up, 1 in 2024-09-16T08:51:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:10 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:10 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:10 smithi031 ceph-mon[27844]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:11.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:10 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:11.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:10 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:11.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:10 smithi097 ceph-mon[35478]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:11.277 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:51:11.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_3 2024-09-16T08:51:11.986 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:51:13.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:13 smithi031 ceph-mon[27844]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:13.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:13 smithi097 ceph-mon[35478]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:14.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 -- 172.21.15.31:0/2896098237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 msgr2=0x7f7ea8102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 --2- 172.21.15.31:0/2896098237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea8102ac0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e98009880 tx=0x7f7e9802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 -- 172.21.15.31:0/2896098237 shutdown_connections 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 --2- 172.21.15.31:0/2896098237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ea8103000 0x7f7ea81053f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 --2- 172.21.15.31:0/2896098237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea8102ac0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 -- 172.21.15.31:0/2896098237 >> 172.21.15.31:0/2896098237 conn(0x7f7ea80fa230 msgr2=0x7f7ea80fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 -- 172.21.15.31:0/2896098237 shutdown_connections 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.581+0000 7f7eaea48640 1 -- 172.21.15.31:0/2896098237 wait complete. 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.582+0000 7f7eaea48640 1 Processor -- start 2024-09-16T08:51:14.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.582+0000 7f7eaea48640 1 -- start start 2024-09-16T08:51:14.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.582+0000 7f7eaea48640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.582+0000 7f7eaea48640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ea8103000 0x7f7ea8197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.582+0000 7f7eaea48640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea81982f0 con 0x7f7ea81006d0 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.582+0000 7f7eaea48640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8198460 con 0x7f7ea8103000 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.582+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea81977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54604/0 (socket says 172.21.15.31:54604) 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea7fff640 1 -- 172.21.15.31:0/831435984 learned_addr learned my addr 172.21.15.31:0/831435984 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea77fe640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ea8103000 0x7f7ea8197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea7fff640 1 -- 172.21.15.31:0/831435984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ea8103000 msgr2=0x7f7ea8197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea7fff640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ea8103000 0x7f7ea8197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea7fff640 1 -- 172.21.15.31:0/831435984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e98009560 con 0x7f7ea81006d0 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea7fff640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea81977e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e980099a0 tx=0x7f7e98032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea57fa640 1 -- 172.21.15.31:0/831435984 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e98005220 con 0x7f7ea81006d0 2024-09-16T08:51:14.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea57fa640 1 -- 172.21.15.31:0/831435984 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e98005380 con 0x7f7ea81006d0 2024-09-16T08:51:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea819cea0 con 0x7f7ea81006d0 2024-09-16T08:51:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea819d390 con 0x7f7ea81006d0 2024-09-16T08:51:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.583+0000 7f7ea57fa640 1 -- 172.21.15.31:0/831435984 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e98003b10 con 0x7f7ea81006d0 2024-09-16T08:51:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.585+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e94005280 con 0x7f7ea81006d0 2024-09-16T08:51:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.585+0000 7f7ea57fa640 1 -- 172.21.15.31:0/831435984 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f7e980046c0 con 0x7f7ea81006d0 2024-09-16T08:51:14.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.585+0000 7f7ea57fa640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7e9006cbc0 0x7f7e9006f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:51:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.586+0000 7f7ea57fa640 1 -- 172.21.15.31:0/831435984 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f7e980ac110 con 0x7f7ea81006d0 2024-09-16T08:51:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.586+0000 7f7ea77fe640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7e9006cbc0 0x7f7e9006f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:51:14.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.586+0000 7f7ea77fe640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7e9006cbc0 0x7f7e9006f080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005fd0 tx=0x7f7e9c005850 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:51:14.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.590+0000 7f7ea57fa640 1 -- 172.21.15.31:0/831435984 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7e98077360 con 0x7f7ea81006d0 2024-09-16T08:51:14.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:14.780+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e94000bf0 con 0x7f7e9006cbc0 2024-09-16T08:51:14.781 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: Detected new or changed devices on smithi031 2024-09-16T08:51:14.781 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.781 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.781 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:51:14.781 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi031 to 11844M 2024-09-16T08:51:14.781 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:14.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:14.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:14.782 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:14 smithi031 ceph-mon[27844]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:14.883 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: Detected new or changed devices on smithi031 2024-09-16T08:51:14.883 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.883 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.883 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:51:14.883 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi031 to 11844M 2024-09-16T08:51:14.884 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.884 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:14.884 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:14.884 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:14.884 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:14.884 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:14 smithi097 ceph-mon[35478]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:15.778 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:15 smithi031 ceph-mon[27844]: from='client.14336 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:51:15.778 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:51:15.778 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:51:15.778 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:15.778 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:15.778 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:16.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:15 smithi097 ceph-mon[35478]: from='client.14336 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:51:16.692 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:51:16.692 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:51:16.692 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:16.692 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:16.692 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:16.693 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:16 smithi031 ceph-mon[27844]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:17.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:16 smithi097 ceph-mon[35478]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:17.848 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:17 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1042551845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5026ce7-8f65-462a-8e4a-68886fa6a266"}]: dispatch 2024-09-16T08:51:17.848 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:17 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1042551845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5026ce7-8f65-462a-8e4a-68886fa6a266"}]': finished 2024-09-16T08:51:17.848 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:17 smithi031 ceph-mon[27844]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T08:51:17.848 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:17 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:17 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1042551845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5026ce7-8f65-462a-8e4a-68886fa6a266"}]: dispatch 2024-09-16T08:51:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:17 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1042551845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5026ce7-8f65-462a-8e4a-68886fa6a266"}]': finished 2024-09-16T08:51:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:17 smithi097 ceph-mon[35478]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T08:51:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:17 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:18.829 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:18 smithi031 ceph-mon[27844]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:18.829 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:18 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/4103383932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:51:19.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:18 smithi097 ceph-mon[35478]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:19.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:18 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/4103383932' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:51:21.332 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:21 smithi031 ceph-mon[27844]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:21.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:21.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:21.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:21.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:21.333 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:21.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:21 smithi097 ceph-mon[35478]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:21.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:21.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:21.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:21.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:21.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:23.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:23 smithi031 ceph-mon[27844]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:23.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:23.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:23 smithi097 ceph-mon[35478]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:23.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:23 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:25.317 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:25 smithi031 ceph-mon[27844]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:25.317 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:25 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T08:51:25.317 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:25 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:25.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:25 smithi097 ceph-mon[35478]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:25.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:25 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T08:51:25.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:25 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:26.344 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:26 smithi031 ceph-mon[27844]: Deploying daemon osd.1 on smithi031 2024-09-16T08:51:26.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:26 smithi097 ceph-mon[35478]: Deploying daemon osd.1 on smithi031 2024-09-16T08:51:27.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:27 smithi031 ceph-mon[27844]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:27.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:27 smithi097 ceph-mon[35478]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:29.238 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:29 smithi031 ceph-mon[27844]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:29.466 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:29 smithi097 ceph-mon[35478]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:30.345 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:30 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:30.345 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:30 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:30.345 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:30 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:30.377 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:30 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:30.377 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:30 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:30.377 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:30 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:31.241 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:31 smithi031 ceph-mon[27844]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:31.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:31 smithi097 ceph-mon[35478]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:32.817 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:32 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:32.817 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:32 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:32.817 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:32 smithi031 ceph-mon[27844]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:33.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:32 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:33.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:32 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:33.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:32 smithi097 ceph-mon[35478]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:33.406 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 1 on host 'smithi031' 2024-09-16T08:51:33.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.405+0000 7f7ea57fa640 1 -- 172.21.15.31:0/831435984 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7e94000bf0 con 0x7f7e9006cbc0 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7e9006cbc0 msgr2=0x7f7e9006f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7e9006cbc0 0x7f7e9006f080 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c005fd0 tx=0x7f7e9c005850 comp rx=0 tx=0).stop 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 msgr2=0x7f7ea81977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea81977e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7e980099a0 tx=0x7f7e98032bb0 comp rx=0 tx=0).stop 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 shutdown_connections 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7e9006cbc0 0x7f7e9006f080 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ea8103000 0x7f7ea8197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 --2- 172.21.15.31:0/831435984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ea81006d0 0x7f7ea81977e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 >> 172.21.15.31:0/831435984 conn(0x7f7ea80fa230 msgr2=0x7f7ea80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 shutdown_connections 2024-09-16T08:51:33.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:33.408+0000 7f7eaea48640 1 -- 172.21.15.31:0/831435984 wait complete. 2024-09-16T08:51:34.049 DEBUG:teuthology.orchestra.run.smithi031:osd.1> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T08:51:34.052 INFO:tasks.cephadm:Deploying osd.2 on smithi031 with /dev/vg_nvme/lv_2... 2024-09-16T08:51:34.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T08:51:34.381 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:51:34.756 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:34 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:34.757 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:34 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:34.757 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:34 smithi031 ceph-mon[27844]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:51:34.757 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:34 smithi031 ceph-mon[27844]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:34.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:34.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:34 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:34.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:34 smithi097 ceph-mon[35478]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T08:51:34.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:34 smithi097 ceph-mon[35478]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:35.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:35 smithi097 ceph-mon[35478]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:51:35.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:35 smithi097 ceph-mon[35478]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T08:51:35.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:35 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:35.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:35 smithi097 ceph-mon[35478]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T08:51:35.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:35 smithi031 ceph-mon[27844]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T08:51:35.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:35 smithi031 ceph-mon[27844]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T08:51:35.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:35 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:35.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:35 smithi031 ceph-mon[27844]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T08:51:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-mon[27844]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T08:51:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-mon[27844]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T08:51:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-mon[27844]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-mon[27844]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' 2024-09-16T08:51:36.623 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[56981]: 2024-09-16T08:51:36.289+0000 7f8fb1e28640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T08:51:36.624 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:51:36 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[56981]: 2024-09-16T08:51:36.298+0000 7f8faa45b640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:51:36.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:36 smithi097 ceph-mon[35478]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T08:51:36.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:36 smithi097 ceph-mon[35478]: osdmap e13: 2 total, 1 up, 2 in 2024-09-16T08:51:36.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:36 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:36.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:36 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:36.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:36 smithi097 ceph-mon[35478]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T08:51:36.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:36 smithi097 ceph-mon[35478]: from='osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079]' entity='osd.1' 2024-09-16T08:51:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:37 smithi031 ceph-mon[27844]: purged_snaps scrub starts 2024-09-16T08:51:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:37 smithi031 ceph-mon[27844]: purged_snaps scrub ok 2024-09-16T08:51:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:37 smithi031 ceph-mon[27844]: osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] boot 2024-09-16T08:51:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:37 smithi031 ceph-mon[27844]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T08:51:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:37.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:37 smithi097 ceph-mon[35478]: purged_snaps scrub starts 2024-09-16T08:51:37.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:37 smithi097 ceph-mon[35478]: purged_snaps scrub ok 2024-09-16T08:51:37.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:37.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:37 smithi097 ceph-mon[35478]: osd.1 [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] boot 2024-09-16T08:51:37.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:37 smithi097 ceph-mon[35478]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T08:51:37.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:51:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:51:38.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi031:vg_nvme/lv_2 2024-09-16T08:51:38.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:38 smithi097 ceph-mon[35478]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:38.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:38 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:38.862 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:51:38.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:38 smithi031 ceph-mon[27844]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:38.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:38 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:39.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:39 smithi031 ceph-mon[27844]: osdmap e15: 2 total, 2 up, 2 in 2024-09-16T08:51:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:39.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:39 smithi097 ceph-mon[35478]: osdmap e15: 2 total, 2 up, 2 in 2024-09-16T08:51:39.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:39.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:40.574 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:40 smithi031 ceph-mon[27844]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:40.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:40 smithi097 ceph-mon[35478]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:41.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 -- 172.21.15.31:0/1949543390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c40ffb90 msgr2=0x7f41c40fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/1949543390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c40ffb90 0x7f41c40fff90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41c0009880 tx=0x7f41c002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 -- 172.21.15.31:0/1949543390 shutdown_connections 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/1949543390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c4100d90 0x7f41c4101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/1949543390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c40ffb90 0x7f41c40fff90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 -- 172.21.15.31:0/1949543390 >> 172.21.15.31:0/1949543390 conn(0x7f41c40fb320 msgr2=0x7f41c40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 -- 172.21.15.31:0/1949543390 shutdown_connections 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 -- 172.21.15.31:0/1949543390 wait complete. 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 Processor -- start 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.022+0000 7f41cc6e6640 1 -- start start 2024-09-16T08:51:41.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41cc6e6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ffb90 0x7f41c4197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41cc6e6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c4100d90 0x7f41c4197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41cc6e6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c41983a0 con 0x7f41c4100d90 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41cc6e6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c4198510 con 0x7f41c40ffb90 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41ca482640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ffb90 0x7f41c4197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41c9c81640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c4100d90 0x7f41c4197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41ca482640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ffb90 0x7f41c4197890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:53576/0 (socket says 172.21.15.31:53576) 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41ca482640 1 -- 172.21.15.31:0/96443637 learned_addr learned my addr 172.21.15.31:0/96443637 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41c9c81640 1 -- 172.21.15.31:0/96443637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ffb90 msgr2=0x7f41c4197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41c9c81640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ffb90 0x7f41c4197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:51:41.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41c9c81640 1 -- 172.21.15.31:0/96443637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c0009560 con 0x7f41c4100d90 2024-09-16T08:51:41.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41c9c81640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c4100d90 0x7f41c4197dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f41b400d6d0 tx=0x7f41b400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:51:41.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.023+0000 7f41bb7fe640 1 -- 172.21.15.31:0/96443637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b40051f0 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.024+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c419cf40 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.024+0000 7f41bb7fe640 1 -- 172.21.15.31:0/96443637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41b4009c80 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.024+0000 7f41bb7fe640 1 -- 172.21.15.31:0/96443637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b4004e50 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.024+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c419d4e0 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.025+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c40fff90 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.025+0000 7f41bb7fe640 1 -- 172.21.15.31:0/96443637 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f41b4005350 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.026+0000 7f41bb7fe640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41b006cb70 0x7f41b006f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.026+0000 7f41bb7fe640 1 -- 172.21.15.31:0/96443637 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f41b408af20 con 0x7f41c4100d90 2024-09-16T08:51:41.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.026+0000 7f41ca482640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41b006cb70 0x7f41b006f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:51:41.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.026+0000 7f41ca482640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41b006cb70 0x7f41b006f030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c00058d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:51:41.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.030+0000 7f41bb7fe640 1 -- 172.21.15.31:0/96443637 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f41b4056070 con 0x7f41c4100d90 2024-09-16T08:51:41.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:51:41.217+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f41c402c450 con 0x7f41b006cb70 2024-09-16T08:51:41.746 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:51:41.746 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:51:41.746 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:41.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:51:41.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:51:41.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:42.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:42 smithi031 ceph-mon[27844]: from='client.14354 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:51:42.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:42 smithi031 ceph-mon[27844]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:42.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:42 smithi097 ceph-mon[35478]: from='client.14354 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi031:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:51:42.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:42 smithi097 ceph-mon[35478]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: Detected new or changed devices on smithi031 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi031 to 5922M 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:45.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:45.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:45.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:44 smithi031 ceph-mon[27844]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:45.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: Detected new or changed devices on smithi031 2024-09-16T08:51:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:51:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:51:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi031 to 5922M 2024-09-16T08:51:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:45.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:45.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:45.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:45.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:44 smithi097 ceph-mon[35478]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:45 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3828089789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13430a00-6986-4da5-86fe-aa675b63f155"}]: dispatch 2024-09-16T08:51:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:45 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3828089789' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13430a00-6986-4da5-86fe-aa675b63f155"}]': finished 2024-09-16T08:51:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:45 smithi097 ceph-mon[35478]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T08:51:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:51:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:45 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/26121581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:51:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:45 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3828089789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "13430a00-6986-4da5-86fe-aa675b63f155"}]: dispatch 2024-09-16T08:51:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:45 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3828089789' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "13430a00-6986-4da5-86fe-aa675b63f155"}]': finished 2024-09-16T08:51:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:45 smithi031 ceph-mon[27844]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T08:51:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:51:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:45 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/26121581' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:51:47.240 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:46 smithi031 ceph-mon[27844]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:47.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:46 smithi097 ceph-mon[35478]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:49.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:49 smithi031 ceph-mon[27844]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:49.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:49.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:49.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:49.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:49.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:49.507 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:49 smithi097 ceph-mon[35478]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:49.507 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:49.507 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:49.507 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:49.507 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:51:49.507 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:51.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:51 smithi097 ceph-mon[35478]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:51 smithi031 ceph-mon[27844]: pgmap v85: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:53 smithi031 ceph-mon[27844]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T08:51:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:53.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:53 smithi097 ceph-mon[35478]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:53.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T08:51:53.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:51:53.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:51:54.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:54 smithi031 ceph-mon[27844]: Deploying daemon osd.2 on smithi031 2024-09-16T08:51:54.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:54 smithi097 ceph-mon[35478]: Deploying daemon osd.2 on smithi031 2024-09-16T08:51:55.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:55 smithi031 ceph-mon[27844]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:55.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:55 smithi097 ceph-mon[35478]: pgmap v87: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:57.349 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:57 smithi031 ceph-mon[27844]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:57.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:57 smithi097 ceph-mon[35478]: pgmap v88: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:58 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:58 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:58 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:58.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:58 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:51:58.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:58 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:58.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:58 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:51:59.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:51:59 smithi031 ceph-mon[27844]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:51:59.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:51:59 smithi097 ceph-mon[35478]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:52:01.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:01 smithi031 ceph-mon[27844]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:52:01.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.154+0000 7f41bb7fe640 1 -- 172.21.15.31:0/96443637 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f41c402c450 con 0x7f41b006cb70 2024-09-16T08:52:01.156 INFO:teuthology.orchestra.run.smithi031.stdout:Created osd(s) 2 on host 'smithi031' 2024-09-16T08:52:01.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41b006cb70 msgr2=0x7f41b006f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:01.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41b006cb70 0x7f41b006f030 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f41c0000c00 tx=0x7f41c00058d0 comp rx=0 tx=0).stop 2024-09-16T08:52:01.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c4100d90 msgr2=0x7f41c4197dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c4100d90 0x7f41c4197dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f41b400d6d0 tx=0x7f41b400db90 comp rx=0 tx=0).stop 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 shutdown_connections 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41b006cb70 0x7f41b006f030 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41c4100d90 0x7f41c4197dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 --2- 172.21.15.31:0/96443637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41c40ffb90 0x7f41c4197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.156+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 >> 172.21.15.31:0/96443637 conn(0x7f41c40fb320 msgr2=0x7f41c4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.157+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 shutdown_connections 2024-09-16T08:52:01.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:52:01.157+0000 7f41cc6e6640 1 -- 172.21.15.31:0/96443637 wait complete. 2024-09-16T08:52:01.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:01 smithi097 ceph-mon[35478]: pgmap v90: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:52:01.765 DEBUG:teuthology.orchestra.run.smithi031:osd.2> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T08:52:01.768 INFO:tasks.cephadm:Deploying osd.3 on smithi097 with /dev/vg_nvme/lv_4... 2024-09-16T08:52:01.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T08:52:01.975 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:52:02.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:02.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:02.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:02 smithi097 ceph-mon[35478]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T08:52:02.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:02 smithi097 ceph-mon[35478]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:52:02.525 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:02.525 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:02.525 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:02 smithi031 ceph-mon[27844]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T08:52:02.525 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:02 smithi031 ceph-mon[27844]: pgmap v91: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:52:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:03 smithi031 ceph-mon[27844]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:52:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:03 smithi031 ceph-mon[27844]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T08:52:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:03 smithi031 ceph-mon[27844]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T08:52:03.373 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:03 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[64315]: 2024-09-16T08:52:03.162+0000 7f27727e8640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T08:52:03.373 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:03 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[64315]: 2024-09-16T08:52:03.169+0000 7f276ee23640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:52:03.395 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:03 smithi097 ceph-mon[35478]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T08:52:03.395 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:03 smithi097 ceph-mon[35478]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T08:52:03.396 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:03.396 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:03 smithi097 ceph-mon[35478]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]: dispatch 2024-09-16T08:52:04.485 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:04 smithi097 ceph-mon[35478]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T08:52:04.485 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:04 smithi097 ceph-mon[35478]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T08:52:04.485 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:04 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:04.485 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:04 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:04.485 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:04 smithi097 ceph-mon[35478]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:52:04.488 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:52:04.507 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-09-16T08:52:04.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:04 smithi031 ceph-mon[27844]: from='osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]}]': finished 2024-09-16T08:52:04.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:04 smithi031 ceph-mon[27844]: osdmap e18: 3 total, 2 up, 3 in 2024-09-16T08:52:04.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:04 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:04.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:04 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:04.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:04 smithi031 ceph-mon[27844]: pgmap v94: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T08:52:04.709 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:52:05.490 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: purged_snaps scrub starts 2024-09-16T08:52:05.490 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: purged_snaps scrub ok 2024-09-16T08:52:05.490 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:05.490 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] boot 2024-09-16T08:52:05.491 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T08:52:05.491 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:05.491 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:05.491 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:05 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:05.508 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: purged_snaps scrub starts 2024-09-16T08:52:05.509 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: purged_snaps scrub ok 2024-09-16T08:52:05.509 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:05.509 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: osd.2 [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] boot 2024-09-16T08:52:05.509 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T08:52:05.509 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:52:05.509 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:05.509 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:05 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 -- 172.21.15.97:0/293908230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084100680 msgr2=0x7f2084102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 --2- 172.21.15.97:0/293908230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084100680 0x7f2084102a70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2074009880 tx=0x7f207402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 -- 172.21.15.97:0/293908230 shutdown_connections 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 --2- 172.21.15.97:0/293908230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2084102fb0 0x7f20841053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 --2- 172.21.15.97:0/293908230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084100680 0x7f2084102a70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 -- 172.21.15.97:0/293908230 >> 172.21.15.97:0/293908230 conn(0x7f20840fa280 msgr2=0x7f20840fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 -- 172.21.15.97:0/293908230 shutdown_connections 2024-09-16T08:52:06.210 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 -- 172.21.15.97:0/293908230 wait complete. 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 Processor -- start 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.206+0000 7f208a855640 1 -- start start 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f208a855640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2084100680 0x7f20841956b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f208a855640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084102fb0 0x7f2084195bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f208a855640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20841961c0 con 0x7f2084100680 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f208a855640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2084196330 con 0x7f2084102fb0 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20837fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084102fb0 0x7f2084195bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20837fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084102fb0 0x7f2084195bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38726/0 (socket says 172.21.15.97:38726) 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20837fe640 1 -- 172.21.15.97:0/100706937 learned_addr learned my addr 172.21.15.97:0/100706937 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f2083fff640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2084100680 0x7f20841956b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:06.211 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20837fe640 1 -- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2084100680 msgr2=0x7f20841956b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20837fe640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2084100680 0x7f20841956b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20837fe640 1 -- 172.21.15.97:0/100706937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2074009560 con 0x7f2084102fb0 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20837fe640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084102fb0 0x7f2084195bf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f207800d6c0 tx=0x7f207800db80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20780051f0 con 0x7f2084102fb0 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2078009ce0 con 0x7f2084102fb0 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2078010460 con 0x7f2084102fb0 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.207+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208419add0 con 0x7f2084102fb0 2024-09-16T08:52:06.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.208+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208419b2f0 con 0x7f2084102fb0 2024-09-16T08:52:06.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.208+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2084100a60 con 0x7f2084102fb0 2024-09-16T08:52:06.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.209+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 18) v1 ==== 90794+0+0 (secure 0 0 0) 0x7f2078005350 con 0x7f2084102fb0 2024-09-16T08:52:06.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.209+0000 7f20817fa640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f206c06cbc0 0x7f206c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:06.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.209+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2846+0+0 (secure 0 0 0) 0x7f207808b090 con 0x7f2084102fb0 2024-09-16T08:52:06.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.210+0000 7f2083fff640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f206c06cbc0 0x7f206c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:06.213 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.210+0000 7f2083fff640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f206c06cbc0 0x7f206c06f080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2074005750 tx=0x7f20740314a0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:52:06.215 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.215+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2078056030 con 0x7f2084102fb0 2024-09-16T08:52:06.386 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:06.384+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2084103970 con 0x7f206c06cbc0 2024-09-16T08:52:06.722 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:06 smithi031 ceph-mon[27844]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T08:52:06.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:06 smithi097 ceph-mon[35478]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T08:52:07.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: pgmap v97: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:07.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:52:07.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:52:07.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:52:07.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:52:07.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:07.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:07.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:52:07.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:52:07.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: pgmap v97: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:07.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:52:07.782 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: from='client.24181 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:52:07.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:52:07.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:52:07.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:07.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:07.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T08:52:07.783 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: pgmap v100: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: Detected new or changed devices on smithi031 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi031 to 3948M 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:08.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.873 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:08 smithi031 sudo[68595]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:52:08.873 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:08 smithi031 sudo[68595]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:08.873 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:08 smithi031 sudo[68595]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:08.873 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:08 smithi031 sudo[68595]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T08:52:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: osdmap e22: 3 total, 3 up, 3 in 2024-09-16T08:52:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: pgmap v100: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: Detected new or changed devices on smithi031 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi031 to 3948M 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:08.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:08.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:08.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:08.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:09.016 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:52:09.017 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:09.017 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68598]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:09.372 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68598]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: osdmap e23: 3 total, 3 up, 3 in 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/3061829032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cad641c2-0145-4535-bf84-9ea6ce8a7730"}]: dispatch 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cad641c2-0145-4535-bf84-9ea6ce8a7730"}]: dispatch 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cad641c2-0145-4535-bf84-9ea6ce8a7730"}]': finished 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:09.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:09 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/1499344442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:52:09.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68601]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:52:09.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68601]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:09.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68601]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:09.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68601]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:10.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: osdmap e23: 3 total, 3 up, 3 in 2024-09-16T08:52:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/3061829032' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cad641c2-0145-4535-bf84-9ea6ce8a7730"}]: dispatch 2024-09-16T08:52:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cad641c2-0145-4535-bf84-9ea6ce8a7730"}]: dispatch 2024-09-16T08:52:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cad641c2-0145-4535-bf84-9ea6ce8a7730"}]': finished 2024-09-16T08:52:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T08:52:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:09 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/1499344442' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:52:10.372 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68604]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:52:10.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68604]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:10.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68604]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:10.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 08:52:09 smithi031 sudo[68604]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:10.604 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:10.603+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2078056400 con 0x7f2084102fb0 2024-09-16T08:52:10.835 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:10 smithi031 ceph-mon[27844]: pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:10.835 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68607]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:52:10.835 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68607]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:10.835 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68607]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:10.835 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68607]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:11.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:10 smithi097 ceph-mon[35478]: pgmap v103: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:11.123 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68610]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:52:11.124 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68610]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:11.124 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68610]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:11.124 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 08:52:10 smithi031 sudo[68610]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:11.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68613]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:52:11.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68613]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:11.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68613]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:11.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68613]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:11.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:11 smithi097 ceph-mon[35478]: mgrmap e19: smithi031.dcmqpv(active, since 3m), standbys: smithi097.mpashf 2024-09-16T08:52:11.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:11 smithi097 ceph-mon[35478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:52:11.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:11 smithi097 sudo[40683]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T08:52:12.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68616]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:52:12.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:12.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68616]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:12.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 sudo[68616]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:12.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 ceph-mon[27844]: mgrmap e19: smithi031.dcmqpv(active, since 3m), standbys: smithi097.mpashf 2024-09-16T08:52:12.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:11 smithi031 ceph-mon[27844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:52:12.246 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:11 smithi097 sudo[40683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:12.246 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:11 smithi097 sudo[40683]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:12.247 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 sudo[40683]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:12.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 sudo[40687]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T08:52:12.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 sudo[40687]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T08:52:12.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 sudo[40687]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T08:52:12.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 sudo[40687]: pam_unix(sudo:session): session closed for user root 2024-09-16T08:52:12.826 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:52:12.826 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:52:12.826 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:52:12.826 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:52:12.826 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:52:12.826 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:52:12.826 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: pgmap v104: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:12.827 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:52:12.827 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:12.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:52:12.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:52:12.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:52:12.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T08:52:12.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:52:12.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:52:12.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: pgmap v104: 1 pgs: 1 creating+peering; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:12.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T08:52:12.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:15 smithi031 ceph-mon[27844]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:15.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:15 smithi097 ceph-mon[35478]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:16.241 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:16 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T08:52:16.241 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:16 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:16.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:16 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T08:52:16.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:16 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:17.355 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:17 smithi097 ceph-mon[35478]: Deploying daemon osd.3 on smithi097 2024-09-16T08:52:17.355 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:17 smithi097 ceph-mon[35478]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:17.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:17 smithi031 ceph-mon[27844]: Deploying daemon osd.3 on smithi097 2024-09-16T08:52:17.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:17 smithi031 ceph-mon[27844]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:19.213 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:19 smithi097 ceph-mon[35478]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:19 smithi031 ceph-mon[27844]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:21.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:21 smithi097 ceph-mon[35478]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:21.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:21.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:21.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:21 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:21.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:21 smithi031 ceph-mon[27844]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:21.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:21.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:21.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:21 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:23 smithi031 ceph-mon[27844]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:23.384 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:23 smithi097 ceph-mon[35478]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:23.384 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:23 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:24.244 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.243+0000 7f20817fa640 1 -- 172.21.15.97:0/100706937 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2084103970 con 0x7f206c06cbc0 2024-09-16T08:52:24.244 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 3 on host 'smithi097' 2024-09-16T08:52:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f206c06cbc0 msgr2=0x7f206c06f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:24.246 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f206c06cbc0 0x7f206c06f080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2074005750 tx=0x7f20740314a0 comp rx=0 tx=0).stop 2024-09-16T08:52:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084102fb0 msgr2=0x7f2084195bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084102fb0 0x7f2084195bf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f207800d6c0 tx=0x7f207800db80 comp rx=0 tx=0).stop 2024-09-16T08:52:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 shutdown_connections 2024-09-16T08:52:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f206c06cbc0 0x7f206c06f080 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:24.247 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2084102fb0 0x7f2084195bf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 --2- 172.21.15.97:0/100706937 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2084100680 0x7f20841956b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 >> 172.21.15.97:0/100706937 conn(0x7f20840fa280 msgr2=0x7f20840fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:52:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 shutdown_connections 2024-09-16T08:52:24.248 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:24.245+0000 7f208a855640 1 -- 172.21.15.97:0/100706937 wait complete. 2024-09-16T08:52:24.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:24.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:24.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:24.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='osd.3 [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:52:24.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:52:24.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:24.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:24 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:24.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:24.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:24.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='osd.3 [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:52:24.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T08:52:24.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:24.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:24 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:24.877 DEBUG:teuthology.orchestra.run.smithi097:osd.3> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T08:52:24.879 INFO:tasks.cephadm:Deploying osd.4 on smithi097 with /dev/vg_nvme/lv_3... 2024-09-16T08:52:24.880 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T08:52:25.634 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:52:25.755 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:25 smithi097 ceph-mon[35478]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:52:25.755 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:25 smithi097 ceph-mon[35478]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:52:25.755 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:25 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:25.756 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:25 smithi097 ceph-mon[35478]: from='osd.3 [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:25.756 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:25 smithi097 ceph-mon[35478]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:25.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:25 smithi031 ceph-mon[27844]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T08:52:25.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:25 smithi031 ceph-mon[27844]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T08:52:25.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:25 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:25.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:25 smithi031 ceph-mon[27844]: from='osd.3 [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:25.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:25 smithi031 ceph-mon[27844]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:26.308 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[41513]: 2024-09-16T08:52:26.277+0000 7f6e90063640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T08:52:26.308 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[41513]: 2024-09-16T08:52:26.283+0000 7f6e88696640 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:52:26.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-mon[35478]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-16T08:52:26.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-mon[35478]: osdmap e26: 4 total, 3 up, 4 in 2024-09-16T08:52:26.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:26.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:26.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-mon[35478]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:26.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:26 smithi097 ceph-mon[35478]: from='osd.3 ' entity='osd.3' 2024-09-16T08:52:26.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:26 smithi031 ceph-mon[27844]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-16T08:52:26.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:26 smithi031 ceph-mon[27844]: osdmap e26: 4 total, 3 up, 4 in 2024-09-16T08:52:26.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:26 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:26.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:26 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:26.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:26 smithi031 ceph-mon[27844]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T08:52:26.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:26 smithi031 ceph-mon[27844]: from='osd.3 ' entity='osd.3' 2024-09-16T08:52:27.699 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:27 smithi031 ceph-mon[27844]: purged_snaps scrub starts 2024-09-16T08:52:27.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:27 smithi031 ceph-mon[27844]: purged_snaps scrub ok 2024-09-16T08:52:27.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:27 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:27.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:27 smithi031 ceph-mon[27844]: osd.3 [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] boot 2024-09-16T08:52:27.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:27 smithi031 ceph-mon[27844]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T08:52:27.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:27 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:27.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:27 smithi097 ceph-mon[35478]: purged_snaps scrub starts 2024-09-16T08:52:27.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:27 smithi097 ceph-mon[35478]: purged_snaps scrub ok 2024-09-16T08:52:27.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:27 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:27.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:27 smithi097 ceph-mon[35478]: osd.3 [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] boot 2024-09-16T08:52:27.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:27 smithi097 ceph-mon[35478]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T08:52:27.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:27 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:52:28.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:28 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T08:52:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:28 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:29.537 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:52:29.557 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-09-16T08:52:29.760 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:52:29.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:29 smithi097 ceph-mon[35478]: Detected new or changed devices on smithi097 2024-09-16T08:52:29.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:29 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi097 to 14912M 2024-09-16T08:52:29.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:29 smithi097 ceph-mon[35478]: osdmap e29: 4 total, 4 up, 4 in 2024-09-16T08:52:29.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:29 smithi031 ceph-mon[27844]: Detected new or changed devices on smithi097 2024-09-16T08:52:29.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:29 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi097 to 14912M 2024-09-16T08:52:29.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:29 smithi031 ceph-mon[27844]: osdmap e29: 4 total, 4 up, 4 in 2024-09-16T08:52:30.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:30 smithi097 ceph-mon[35478]: osdmap e30: 4 total, 4 up, 4 in 2024-09-16T08:52:30.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:30 smithi097 ceph-mon[35478]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:30.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:30 smithi031 ceph-mon[27844]: osdmap e30: 4 total, 4 up, 4 in 2024-09-16T08:52:30.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:30 smithi031 ceph-mon[27844]: pgmap v119: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:31.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 -- 172.21.15.97:0/3759065726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84441000b0 msgr2=0x7f84441004b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:31.409 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 --2- 172.21.15.97:0/3759065726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84441000b0 0x7f84441004b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8434009880 tx=0x7f843402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 -- 172.21.15.97:0/3759065726 shutdown_connections 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 --2- 172.21.15.97:0/3759065726 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8444100a80 0x7f84440fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 --2- 172.21.15.97:0/3759065726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84441000b0 0x7f84441004b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 -- 172.21.15.97:0/3759065726 >> 172.21.15.97:0/3759065726 conn(0x7f84440f99b0 msgr2=0x7f84440fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 -- 172.21.15.97:0/3759065726 shutdown_connections 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.406+0000 7f8449a14640 1 -- 172.21.15.97:0/3759065726 wait complete. 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.407+0000 7f8449a14640 1 Processor -- start 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.407+0000 7f8449a14640 1 -- start start 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.407+0000 7f8449a14640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84441000b0 0x7f8444197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.407+0000 7f8449a14640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8444100a80 0x7f8444197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.407+0000 7f8449a14640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444198460 con 0x7f84441000b0 2024-09-16T08:52:31.410 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.407+0000 7f8449a14640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84441985d0 con 0x7f8444100a80 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f84427fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8444100a80 0x7f8444197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f84427fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8444100a80 0x7f8444197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:38750/0 (socket says 172.21.15.97:38750) 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f84427fc640 1 -- 172.21.15.97:0/3248937814 learned_addr learned my addr 172.21.15.97:0/3248937814 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f8442ffd640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84441000b0 0x7f8444197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f84427fc640 1 -- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84441000b0 msgr2=0x7f8444197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f84427fc640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84441000b0 0x7f8444197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f84427fc640 1 -- 172.21.15.97:0/3248937814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434009560 con 0x7f8444100a80 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f84427fc640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8444100a80 0x7f8444197e90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b3d0 tx=0x7f842c00b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f8448a12640 1 -- 172.21.15.97:0/3248937814 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842c011800 con 0x7f8444100a80 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f8448a12640 1 -- 172.21.15.97:0/3248937814 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f842c011e20 con 0x7f8444100a80 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844419d070 con 0x7f8444100a80 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f8448a12640 1 -- 172.21.15.97:0/3248937814 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842c00f2e0 con 0x7f8444100a80 2024-09-16T08:52:31.411 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.408+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844419d640 con 0x7f8444100a80 2024-09-16T08:52:31.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.409+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844410aa40 con 0x7f8444100a80 2024-09-16T08:52:31.412 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.410+0000 7f8448a12640 1 -- 172.21.15.97:0/3248937814 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f842c010380 con 0x7f8444100a80 2024-09-16T08:52:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.410+0000 7f8448a12640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f843006cc60 0x7f843006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.410+0000 7f8448a12640 1 -- 172.21.15.97:0/3248937814 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3774+0+0 (secure 0 0 0) 0x7f842c08b8e0 con 0x7f8444100a80 2024-09-16T08:52:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.410+0000 7f8442ffd640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f843006cc60 0x7f843006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:31.413 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.411+0000 7f8442ffd640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f843006cc60 0x7f843006f120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f843402a9c0 tx=0x7f8434000bc0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:52:31.415 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.415+0000 7f8448a12640 1 -- 172.21.15.97:0/3248937814 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f842c056350 con 0x7f8444100a80 2024-09-16T08:52:31.588 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:31.588+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f844419d8c0 con 0x7f843006cc60 2024-09-16T08:52:31.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:31 smithi097 ceph-mon[35478]: osdmap e31: 4 total, 4 up, 4 in 2024-09-16T08:52:31.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:31 smithi031 ceph-mon[27844]: osdmap e31: 4 total, 4 up, 4 in 2024-09-16T08:52:32.715 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:32 smithi097 ceph-mon[35478]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:52:32.715 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:32 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:52:32.715 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:32 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:52:32.715 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:32 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:32.716 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:32 smithi097 ceph-mon[35478]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:32 smithi031 ceph-mon[27844]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:52:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:32 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:52:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:32 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:52:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:32 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:32 smithi031 ceph-mon[27844]: pgmap v121: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:34.247 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:33 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/1484070033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51"}]: dispatch 2024-09-16T08:52:34.247 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:33 smithi097 ceph-mon[35478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51"}]: dispatch 2024-09-16T08:52:34.247 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:33 smithi097 ceph-mon[35478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51"}]': finished 2024-09-16T08:52:34.247 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:33 smithi097 ceph-mon[35478]: osdmap e32: 5 total, 4 up, 5 in 2024-09-16T08:52:34.247 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:33 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:33 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/1484070033' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51"}]: dispatch 2024-09-16T08:52:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:33 smithi031 ceph-mon[27844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51"}]: dispatch 2024-09-16T08:52:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:33 smithi031 ceph-mon[27844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51"}]': finished 2024-09-16T08:52:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:33 smithi031 ceph-mon[27844]: osdmap e32: 5 total, 4 up, 5 in 2024-09-16T08:52:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:33 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:35.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:34 smithi097 ceph-mon[35478]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-16T08:52:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/3632115412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:52:35.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:34 smithi031 ceph-mon[27844]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-16T08:52:35.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:34 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/3632115412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:52:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:37 smithi031 ceph-mon[27844]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-16T08:52:37.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:37 smithi097 ceph-mon[35478]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-09-16T08:52:38.281 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:38 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:38.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:38 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:39.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:39 smithi097 ceph-mon[35478]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:52:39.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:39 smithi031 ceph-mon[27844]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T08:52:41.372 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:41 smithi097 ceph-mon[35478]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-16T08:52:41.372 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T08:52:41.372 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:41 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:41 smithi031 ceph-mon[27844]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-16T08:52:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T08:52:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:41 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:42.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:42 smithi097 ceph-mon[35478]: Deploying daemon osd.4 on smithi097 2024-09-16T08:52:42.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:42 smithi031 ceph-mon[27844]: Deploying daemon osd.4 on smithi097 2024-09-16T08:52:43.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:43 smithi031 ceph-mon[27844]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:52:43.431 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:43 smithi097 ceph-mon[35478]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T08:52:45.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:45 smithi031 ceph-mon[27844]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:45.374 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:45 smithi097 ceph-mon[35478]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:46.162 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:46.162 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:46.162 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:46.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:47 smithi097 ceph-mon[35478]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:47 smithi031 ceph-mon[27844]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:49.163 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.161+0000 7f8448a12640 1 -- 172.21.15.97:0/3248937814 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f844419d8c0 con 0x7f843006cc60 2024-09-16T08:52:49.163 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 4 on host 'smithi097' 2024-09-16T08:52:49.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:49 smithi097 ceph-mon[35478]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f843006cc60 msgr2=0x7f843006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f843006cc60 0x7f843006f120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f843402a9c0 tx=0x7f8434000bc0 comp rx=0 tx=0).stop 2024-09-16T08:52:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8444100a80 msgr2=0x7f8444197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8444100a80 0x7f8444197e90 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b3d0 tx=0x7f842c00b890 comp rx=0 tx=0).stop 2024-09-16T08:52:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 shutdown_connections 2024-09-16T08:52:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f843006cc60 0x7f843006f120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:49.165 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8444100a80 0x7f8444197e90 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 --2- 172.21.15.97:0/3248937814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84441000b0 0x7f8444197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.163+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 >> 172.21.15.97:0/3248937814 conn(0x7f84440f99b0 msgr2=0x7f8444107340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:52:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.164+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 shutdown_connections 2024-09-16T08:52:49.166 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:49.164+0000 7f8449a14640 1 -- 172.21.15.97:0/3248937814 wait complete. 2024-09-16T08:52:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:49 smithi031 ceph-mon[27844]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:49.737 DEBUG:teuthology.orchestra.run.smithi097:osd.4> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T08:52:49.740 INFO:tasks.cephadm:Deploying osd.5 on smithi097 with /dev/vg_nvme/lv_2... 2024-09-16T08:52:49.740 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T08:52:49.982 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:52:50.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:50 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:50 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:50 smithi097 ceph-mon[35478]: from='osd.4 [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:52:50.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:50 smithi097 ceph-mon[35478]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:52:50.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:50 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:50 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:50 smithi097 ceph-mon[35478]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:50 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:50 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:50 smithi031 ceph-mon[27844]: from='osd.4 [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:52:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:50 smithi031 ceph-mon[27844]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T08:52:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:50 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:50 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:50 smithi031 ceph-mon[27844]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:51.409 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:51 smithi097 ceph-mon[35478]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:52:51.409 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:51 smithi097 ceph-mon[35478]: osdmap e33: 5 total, 4 up, 5 in 2024-09-16T08:52:51.409 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:51 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:51.409 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:51 smithi097 ceph-mon[35478]: from='osd.4 [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:51.409 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:51 smithi097 ceph-mon[35478]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:51.410 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 08:52:51 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[48369]: 2024-09-16T08:52:51.170+0000 7f6a2acc9640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T08:52:51.410 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 08:52:51 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[48369]: 2024-09-16T08:52:51.177+0000 7f6a27304640 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:52:51.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:51 smithi031 ceph-mon[27844]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T08:52:51.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:51 smithi031 ceph-mon[27844]: osdmap e33: 5 total, 4 up, 5 in 2024-09-16T08:52:51.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:51 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:51.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:51 smithi031 ceph-mon[27844]: from='osd.4 [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:51.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:51 smithi031 ceph-mon[27844]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:52:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:52 smithi097 ceph-mon[35478]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-16T08:52:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:52 smithi097 ceph-mon[35478]: osdmap e34: 5 total, 4 up, 5 in 2024-09-16T08:52:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:52 smithi097 ceph-mon[35478]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:52 smithi031 ceph-mon[27844]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-16T08:52:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:52 smithi031 ceph-mon[27844]: osdmap e34: 5 total, 4 up, 5 in 2024-09-16T08:52:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:52 smithi031 ceph-mon[27844]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T08:52:53.087 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-09-16T08:52:53.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-09-16T08:52:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:53 smithi031 ceph-mon[27844]: purged_snaps scrub starts 2024-09-16T08:52:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:53 smithi031 ceph-mon[27844]: purged_snaps scrub ok 2024-09-16T08:52:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:53 smithi031 ceph-mon[27844]: osd.4 [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] boot 2024-09-16T08:52:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:53 smithi031 ceph-mon[27844]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T08:52:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:53.456 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:53 smithi097 ceph-mon[35478]: purged_snaps scrub starts 2024-09-16T08:52:53.456 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:53 smithi097 ceph-mon[35478]: purged_snaps scrub ok 2024-09-16T08:52:53.456 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:53.456 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:53 smithi097 ceph-mon[35478]: osd.4 [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] boot 2024-09-16T08:52:53.456 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:53 smithi097 ceph-mon[35478]: osdmap e35: 5 total, 5 up, 5 in 2024-09-16T08:52:53.456 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:52:53.456 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:52:53.710 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:52:54.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:54 smithi097 ceph-mon[35478]: osdmap e36: 5 total, 5 up, 5 in 2024-09-16T08:52:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:54 smithi097 ceph-mon[35478]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:52:54.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:54 smithi031 ceph-mon[27844]: osdmap e36: 5 total, 5 up, 5 in 2024-09-16T08:52:54.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:54 smithi031 ceph-mon[27844]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.354+0000 7fab04210640 1 -- 172.21.15.97:0/4292969086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc102f40 msgr2=0x7faafc105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.354+0000 7fab04210640 1 --2- 172.21.15.97:0/4292969086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc102f40 0x7faafc105330 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009880 tx=0x7faaf802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.354+0000 7fab04210640 1 -- 172.21.15.97:0/4292969086 shutdown_connections 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.354+0000 7fab04210640 1 --2- 172.21.15.97:0/4292969086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc102f40 0x7faafc105330 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.354+0000 7fab04210640 1 --2- 172.21.15.97:0/4292969086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faafc100610 0x7faafc102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.354+0000 7fab04210640 1 -- 172.21.15.97:0/4292969086 >> 172.21.15.97:0/4292969086 conn(0x7faafc0fa230 msgr2=0x7faafc0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.355+0000 7fab04210640 1 -- 172.21.15.97:0/4292969086 shutdown_connections 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.355+0000 7fab04210640 1 -- 172.21.15.97:0/4292969086 wait complete. 2024-09-16T08:52:55.356 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.355+0000 7fab04210640 1 Processor -- start 2024-09-16T08:52:55.357 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.355+0000 7fab04210640 1 -- start start 2024-09-16T08:52:55.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.355+0000 7fab04210640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc100610 0x7faafc195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:55.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.355+0000 7fab04210640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faafc102f40 0x7faafc195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:55.358 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab04210640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc196220 con 0x7faafc102f40 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab04210640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc196390 con 0x7faafc100610 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab01fac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc100610 0x7faafc195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab01fac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc100610 0x7faafc195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36810/0 (socket says 172.21.15.97:36810) 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab01fac640 1 -- 172.21.15.97:0/763440604 learned_addr learned my addr 172.21.15.97:0/763440604 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab017ab640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faafc102f40 0x7faafc195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab01fac640 1 -- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faafc102f40 msgr2=0x7faafc195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab01fac640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faafc102f40 0x7faafc195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab01fac640 1 -- 172.21.15.97:0/763440604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8009560 con 0x7faafc100610 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab01fac640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc100610 0x7faafc195710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faaec00b420 tx=0x7faaec00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7faaf2ffd640 1 -- 172.21.15.97:0/763440604 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaec011800 con 0x7faafc100610 2024-09-16T08:52:55.359 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7faaf2ffd640 1 -- 172.21.15.97:0/763440604 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faaec011e20 con 0x7faafc100610 2024-09-16T08:52:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc19ae30 con 0x7faafc100610 2024-09-16T08:52:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7faaf2ffd640 1 -- 172.21.15.97:0/763440604 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaec00f2e0 con 0x7faafc100610 2024-09-16T08:52:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.356+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc19b3a0 con 0x7faafc100610 2024-09-16T08:52:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.357+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc1087f0 con 0x7faafc100610 2024-09-16T08:52:55.360 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.358+0000 7faaf2ffd640 1 -- 172.21.15.97:0/763440604 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faaec010380 con 0x7faafc100610 2024-09-16T08:52:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.358+0000 7faaf2ffd640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faae806cc50 0x7faae806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:52:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.358+0000 7faaf2ffd640 1 -- 172.21.15.97:0/763440604 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4206+0+0 (secure 0 0 0) 0x7faaec08b6f0 con 0x7faafc100610 2024-09-16T08:52:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.359+0000 7fab017ab640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faae806cc50 0x7faae806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:52:55.361 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.359+0000 7fab017ab640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faae806cc50 0x7faae806f110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faaf8031500 tx=0x7faaf8000bc0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:52:55.363 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.362+0000 7faaf2ffd640 1 -- 172.21.15.97:0/763440604 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faaec055fb0 con 0x7faafc100610 2024-09-16T08:52:55.543 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:52:55.542+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7faafc19b620 con 0x7faae806cc50 2024-09-16T08:52:55.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:55 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:52:55.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:55 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:52:55.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:55 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:55.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:55 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T08:52:55.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:55 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T08:52:55.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:55 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:56.981 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:52:56.982 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: Detected new or changed devices on smithi097 2024-09-16T08:52:56.982 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:56.982 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:56.982 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:56.982 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:56.983 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi097 to 7456M 2024-09-16T08:52:56.983 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:56.983 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:56.983 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:56.983 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:56.984 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:56.984 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:56.984 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:56.984 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:56.984 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:56 smithi097 ceph-mon[35478]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:52:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:52:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: Detected new or changed devices on smithi097 2024-09-16T08:52:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi097 to 7456M 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:52:57.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:56 smithi031 ceph-mon[27844]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:52:58.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:57 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/1258568855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7061f423-5ae3-4a14-b7f9-2713ca675eec"}]: dispatch 2024-09-16T08:52:58.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:57 smithi097 ceph-mon[35478]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7061f423-5ae3-4a14-b7f9-2713ca675eec"}]: dispatch 2024-09-16T08:52:58.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:57 smithi097 ceph-mon[35478]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7061f423-5ae3-4a14-b7f9-2713ca675eec"}]': finished 2024-09-16T08:52:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:57 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/1258568855' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7061f423-5ae3-4a14-b7f9-2713ca675eec"}]: dispatch 2024-09-16T08:52:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:57 smithi031 ceph-mon[27844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7061f423-5ae3-4a14-b7f9-2713ca675eec"}]: dispatch 2024-09-16T08:52:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:57 smithi031 ceph-mon[27844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7061f423-5ae3-4a14-b7f9-2713ca675eec"}]': finished 2024-09-16T08:52:59.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:58 smithi097 ceph-mon[35478]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T08:52:59.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:58 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:52:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:58 smithi097 ceph-mon[35478]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:52:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:52:58 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/4266436147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:52:59.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:58 smithi031 ceph-mon[27844]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T08:52:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:58 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:52:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:58 smithi031 ceph-mon[27844]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:52:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:52:58 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/4266436147' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T08:53:01.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:01 smithi031 ceph-mon[27844]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:01.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:01 smithi097 ceph-mon[35478]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:03.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:03 smithi031 ceph-mon[27844]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:03.494 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:03 smithi097 ceph-mon[35478]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:05.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:05 smithi097 ceph-mon[35478]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:05 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T08:53:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:05 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:53:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:05 smithi031 ceph-mon[27844]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:05 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T08:53:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:05 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:53:06.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:06 smithi031 ceph-mon[27844]: Deploying daemon osd.5 on smithi097 2024-09-16T08:53:06.398 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:06 smithi097 ceph-mon[35478]: Deploying daemon osd.5 on smithi097 2024-09-16T08:53:07.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:07 smithi097 ceph-mon[35478]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:07.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:07 smithi031 ceph-mon[27844]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:08.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:08.433 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:09.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:09 smithi031 ceph-mon[27844]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:09.458 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:09 smithi097 ceph-mon[35478]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:10.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:10 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:53:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:10 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:10 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:10.421 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:10 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:53:10.421 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:10 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:10.421 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:10 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:11 smithi031 ceph-mon[27844]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:11.451 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:11 smithi097 ceph-mon[35478]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:12.792 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.791+0000 7faaf2ffd640 1 -- 172.21.15.97:0/763440604 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faafc19b620 con 0x7faae806cc50 2024-09-16T08:53:12.792 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 5 on host 'smithi097' 2024-09-16T08:53:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faae806cc50 msgr2=0x7faae806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faae806cc50 0x7faae806f110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faaf8031500 tx=0x7faaf8000bc0 comp rx=0 tx=0).stop 2024-09-16T08:53:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc100610 msgr2=0x7faafc195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc100610 0x7faafc195710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faaec00b420 tx=0x7faaec00b8e0 comp rx=0 tx=0).stop 2024-09-16T08:53:12.794 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 shutdown_connections 2024-09-16T08:53:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faae806cc50 0x7faae806f110 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faafc102f40 0x7faafc195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 --2- 172.21.15.97:0/763440604 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faafc100610 0x7faafc195710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 >> 172.21.15.97:0/763440604 conn(0x7faafc0fa230 msgr2=0x7faafc0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 shutdown_connections 2024-09-16T08:53:12.795 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:12.793+0000 7fab04210640 1 -- 172.21.15.97:0/763440604 wait complete. 2024-09-16T08:53:13.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:13 smithi031 ceph-mon[27844]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:13.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:13 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:13.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:13 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:13.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:13 smithi097 ceph-mon[35478]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:13.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:13 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:13.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:13 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:13.944 DEBUG:teuthology.orchestra.run.smithi097:osd.5> sudo journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T08:53:13.946 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-09-16T08:53:13.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T08:53:14.162 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:14.227 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:14 smithi097 ceph-mon[35478]: from='osd.5 [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:53:14.227 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:14 smithi097 ceph-mon[35478]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:53:14.280 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:14 smithi031 ceph-mon[27844]: from='osd.5 [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:53:14.281 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:14 smithi031 ceph-mon[27844]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: from='osd.5 [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:15.263 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[54831]: 2024-09-16T08:53:15.141+0000 7f02e484e640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T08:53:15.263 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 08:53:15 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[54831]: 2024-09-16T08:53:15.148+0000 7f02e0e89640 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: from='osd.5 [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:15.447 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:15 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 -- 172.21.15.31:0/3003264510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 msgr2=0x7f51a4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/3003264510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4101210 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5194009880 tx=0x7f519402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 -- 172.21.15.31:0/3003264510 shutdown_connections 2024-09-16T08:53:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/3003264510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4101210 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/3003264510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a40ffb90 0x7f51a40fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 -- 172.21.15.31:0/3003264510 >> 172.21.15.31:0/3003264510 conn(0x7f51a40fb320 msgr2=0x7f51a40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:15.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 -- 172.21.15.31:0/3003264510 shutdown_connections 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.784+0000 7f51ab7c8640 1 -- 172.21.15.31:0/3003264510 wait complete. 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51ab7c8640 1 Processor -- start 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51ab7c8640 1 -- start start 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51ab7c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a40ffb90 0x7f51a4195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51ab7c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51ab7c8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4196240 con 0x7f51a4100d90 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51ab7c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41963b0 con 0x7f51a40ffb90 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a8d63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a8d63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4195c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36002/0 (socket says 172.21.15.31:36002) 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a8d63640 1 -- 172.21.15.31:0/963452709 learned_addr learned my addr 172.21.15.31:0/963452709 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a9564640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a40ffb90 0x7f51a4195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:15.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a8d63640 1 -- 172.21.15.31:0/963452709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a40ffb90 msgr2=0x7f51a4195730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a8d63640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a40ffb90 0x7f51a4195730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a8d63640 1 -- 172.21.15.31:0/963452709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009560 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.785+0000 7f51a9564640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a40ffb90 0x7f51a4195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.786+0000 7f51a8d63640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4195c70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5194005640 tx=0x7f5194005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.786+0000 7f519a7fc640 1 -- 172.21.15.31:0/963452709 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5194004ba0 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.786+0000 7f519a7fc640 1 -- 172.21.15.31:0/963452709 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5194004d00 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.786+0000 7f519a7fc640 1 -- 172.21.15.31:0/963452709 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5194003c30 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.786+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a419adf0 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.786+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a419b330 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.787+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4100010 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.787+0000 7f519a7fc640 1 -- 172.21.15.31:0/963452709 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5194031640 con 0x7f51a4100d90 2024-09-16T08:53:15.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.788+0000 7f519a7fc640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f519006cc50 0x7f519006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:15.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.788+0000 7f51a9564640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f519006cc50 0x7f519006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:15.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.788+0000 7f519a7fc640 1 -- 172.21.15.31:0/963452709 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4358+0+0 (secure 0 0 0) 0x7f51940acfd0 con 0x7f51a4100d90 2024-09-16T08:53:15.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.788+0000 7f51a9564640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f519006cc50 0x7f519006f110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f51a0006fd0 tx=0x7f51a0008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:15.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.793+0000 7f519a7fc640 1 -- 172.21.15.31:0/963452709 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5194077870 con 0x7f51a4100d90 2024-09-16T08:53:15.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.965+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f51a4108820 con 0x7f51a4100d90 2024-09-16T08:53:15.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.966+0000 7f519a7fc640 1 -- 172.21.15.31:0/963452709 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f519404a0a0 con 0x7f51a4100d90 2024-09-16T08:53:15.968 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f519006cc50 msgr2=0x7f519006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f519006cc50 0x7f519006f110 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f51a0006fd0 tx=0x7f51a0008040 comp rx=0 tx=0).stop 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 msgr2=0x7f51a4195c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4195c70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5194005640 tx=0x7f5194005670 comp rx=0 tx=0).stop 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 shutdown_connections 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f519006cc50 0x7f519006f110 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51a4100d90 0x7f51a4195c70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:15.970 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 --2- 172.21.15.31:0/963452709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a40ffb90 0x7f51a4195730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:15.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 >> 172.21.15.31:0/963452709 conn(0x7f51a40fb320 msgr2=0x7f51a4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:15.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 shutdown_connections 2024-09-16T08:53:15.971 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:15.968+0000 7f51ab7c8640 1 -- 172.21.15.31:0/963452709 wait complete. 2024-09-16T08:53:16.283 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:16 smithi031 ceph-mon[27844]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-16T08:53:16.283 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:16 smithi031 ceph-mon[27844]: osdmap e39: 6 total, 5 up, 6 in 2024-09-16T08:53:16.283 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:16 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:16.283 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:16 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:16.283 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:16 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/963452709' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T08:53:16.284 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:16 smithi031 ceph-mon[27844]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:16.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:16 smithi097 ceph-mon[35478]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-09-16T08:53:16.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:16 smithi097 ceph-mon[35478]: osdmap e39: 6 total, 5 up, 6 in 2024-09-16T08:53:16.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:16 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:16.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:16 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:16.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:16 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/963452709' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T08:53:16.426 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:16 smithi097 ceph-mon[35478]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-16T08:53:16.620 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":39,"num_osds":6,"num_up_osds":5,"osd_up_since":1726476772,"num_in_osds":6,"osd_in_since":1726476777,"num_remapped_pgs":0} 2024-09-16T08:53:17.442 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:17 smithi097 ceph-mon[35478]: purged_snaps scrub starts 2024-09-16T08:53:17.442 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:17 smithi097 ceph-mon[35478]: purged_snaps scrub ok 2024-09-16T08:53:17.442 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:17 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:17.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:17 smithi097 ceph-mon[35478]: osd.5 [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] boot 2024-09-16T08:53:17.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:17 smithi097 ceph-mon[35478]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T08:53:17.443 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:17 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:17.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T08:53:17.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:17 smithi031 ceph-mon[27844]: purged_snaps scrub starts 2024-09-16T08:53:17.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:17 smithi031 ceph-mon[27844]: purged_snaps scrub ok 2024-09-16T08:53:17.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:17 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:17.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:17 smithi031 ceph-mon[27844]: osd.5 [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] boot 2024-09-16T08:53:17.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:17 smithi031 ceph-mon[27844]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T08:53:17.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:17 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:53:17.831 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:18.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: Detected new or changed devices on smithi097 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:53:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:18 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:18.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T08:53:18.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:18.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: Detected new or changed devices on smithi097 2024-09-16T08:53:18.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:18.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:18.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:53:18.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:53:18.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:53:18.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:18.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:53:18.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:53:18.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:18 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:19.279 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:19 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi097 to 4970M 2024-09-16T08:53:19.279 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:53:19.279 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:53:19.279 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:53:19.279 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:19.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:19 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi097 to 4970M 2024-09-16T08:53:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:53:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:53:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:53:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:53:19.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 -- 172.21.15.31:0/1199902931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc840ffbb0 msgr2=0x7efc840fffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 --2- 172.21.15.31:0/1199902931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc840ffbb0 0x7efc840fffb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efc74009880 tx=0x7efc7402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 -- 172.21.15.31:0/1199902931 shutdown_connections 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 --2- 172.21.15.31:0/1199902931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc84100db0 0x7efc84101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 --2- 172.21.15.31:0/1199902931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc840ffbb0 0x7efc840fffb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 -- 172.21.15.31:0/1199902931 >> 172.21.15.31:0/1199902931 conn(0x7efc840fb360 msgr2=0x7efc840fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 -- 172.21.15.31:0/1199902931 shutdown_connections 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.564+0000 7efc8958d640 1 -- 172.21.15.31:0/1199902931 wait complete. 2024-09-16T08:53:19.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.565+0000 7efc8958d640 1 Processor -- start 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.565+0000 7efc8958d640 1 -- start start 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc8958d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc840ffbb0 0x7efc84195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc8958d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84100db0 0x7efc84195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc8958d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc84196280 con 0x7efc84100db0 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc8958d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc841963f0 con 0x7efc840ffbb0 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84100db0 0x7efc84195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc827fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84100db0 0x7efc84195cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36012/0 (socket says 172.21.15.31:36012) 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc827fc640 1 -- 172.21.15.31:0/434962533 learned_addr learned my addr 172.21.15.31:0/434962533 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:19.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc82ffd640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc840ffbb0 0x7efc84195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc827fc640 1 -- 172.21.15.31:0/434962533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc840ffbb0 msgr2=0x7efc84195770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc827fc640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc840ffbb0 0x7efc84195770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc827fc640 1 -- 172.21.15.31:0/434962533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc74009560 con 0x7efc84100db0 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc827fc640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84100db0 0x7efc84195cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00b450 tx=0x7efc6c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc7bfff640 1 -- 172.21.15.31:0/434962533 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc6c011860 con 0x7efc84100db0 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc7bfff640 1 -- 172.21.15.31:0/434962533 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc6c011e80 con 0x7efc84100db0 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.566+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc8419ae90 con 0x7efc84100db0 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.567+0000 7efc7bfff640 1 -- 172.21.15.31:0/434962533 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc6c00f570 con 0x7efc84100db0 2024-09-16T08:53:19.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.567+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc8419b430 con 0x7efc84100db0 2024-09-16T08:53:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.567+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc84100030 con 0x7efc84100db0 2024-09-16T08:53:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.568+0000 7efc7bfff640 1 -- 172.21.15.31:0/434962533 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7efc6c0103e0 con 0x7efc84100db0 2024-09-16T08:53:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.568+0000 7efc7bfff640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7efc7006cc50 0x7efc7006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.569+0000 7efc7bfff640 1 -- 172.21.15.31:0/434962533 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7efc6c08bd80 con 0x7efc84100db0 2024-09-16T08:53:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.569+0000 7efc82ffd640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7efc7006cc50 0x7efc7006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:19.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.569+0000 7efc82ffd640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7efc7006cc50 0x7efc7006f110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc74031540 tx=0x7efc74031400 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.573+0000 7efc7bfff640 1 -- 172.21.15.31:0/434962533 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7efc6c056490 con 0x7efc84100db0 2024-09-16T08:53:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.744+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7efc84108820 con 0x7efc84100db0 2024-09-16T08:53:19.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.745+0000 7efc7bfff640 1 -- 172.21.15.31:0/434962533 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v41) v1 ==== 74+0+130 (secure 0 0 0) 0x7efc6c059ab0 con 0x7efc84100db0 2024-09-16T08:53:19.747 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:53:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.746+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7efc7006cc50 msgr2=0x7efc7006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:19.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.746+0000 7efc8958d640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7efc7006cc50 0x7efc7006f110 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efc74031540 tx=0x7efc74031400 comp rx=0 tx=0).stop 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.746+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84100db0 msgr2=0x7efc84195cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.746+0000 7efc8958d640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84100db0 0x7efc84195cb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00b450 tx=0x7efc6c00b910 comp rx=0 tx=0).stop 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.747+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 shutdown_connections 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.747+0000 7efc8958d640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7efc7006cc50 0x7efc7006f110 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.747+0000 7efc8958d640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc84100db0 0x7efc84195cb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.747+0000 7efc8958d640 1 --2- 172.21.15.31:0/434962533 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efc840ffbb0 0x7efc84195770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.747+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 >> 172.21.15.31:0/434962533 conn(0x7efc840fb360 msgr2=0x7efc84103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.747+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 shutdown_connections 2024-09-16T08:53:19.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:19.747+0000 7efc8958d640 1 -- 172.21.15.31:0/434962533 wait complete. 2024-09-16T08:53:20.457 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:20 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/434962533' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T08:53:20.458 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:20 smithi031 ceph-mon[27844]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:20.483 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":41,"num_osds":6,"num_up_osds":6,"osd_up_since":1726476796,"num_in_osds":6,"osd_in_since":1726476777,"num_remapped_pgs":0} 2024-09-16T08:53:20.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:53:20.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:20 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/434962533' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T08:53:20.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:20 smithi097 ceph-mon[35478]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:20.697 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 -- 172.21.15.31:0/1891076140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac1049f0 msgr2=0x7fdaac104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 --2- 172.21.15.31:0/1891076140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac1049f0 0x7fdaac104dd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009c20 tx=0x7fda9c02a970 comp rx=0 tx=0).stop 2024-09-16T08:53:22.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 -- 172.21.15.31:0/1891076140 shutdown_connections 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 --2- 172.21.15.31:0/1891076140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac0fe9f0 0x7fdaac0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 --2- 172.21.15.31:0/1891076140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac1049f0 0x7fdaac104dd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 -- 172.21.15.31:0/1891076140 >> 172.21.15.31:0/1891076140 conn(0x7fdaac0758a0 msgr2=0x7fdaac0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 -- 172.21.15.31:0/1891076140 shutdown_connections 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.291+0000 7fdab11af640 1 -- 172.21.15.31:0/1891076140 wait complete. 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdab11af640 1 Processor -- start 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdab11af640 1 -- start start 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdab11af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac0fe9f0 0x7fdaac105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdab11af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac1049f0 0x7fdaac105e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdab11af640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac1063a0 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdab11af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac1064e0 con 0x7fdaac1049f0 2024-09-16T08:53:22.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdaaad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac0fe9f0 0x7fdaac105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdaaad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac0fe9f0 0x7fdaac105920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34822/0 (socket says 172.21.15.31:34822) 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdaaad9d640 1 -- 172.21.15.31:0/1703435140 learned_addr learned my addr 172.21.15.31:0/1703435140 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.292+0000 7fdaaa59c640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac1049f0 0x7fdaac105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fdaaad9d640 1 -- 172.21.15.31:0/1703435140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac1049f0 msgr2=0x7fdaac105e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fdaaad9d640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac1049f0 0x7fdaac105e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fdaaad9d640 1 -- 172.21.15.31:0/1703435140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c009900 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fdaaad9d640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac0fe9f0 0x7fdaac105920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda9c02ae90 tx=0x7fda9c0308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:22.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fda93fff640 1 -- 172.21.15.31:0/1703435140 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c02f590 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac106760 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fda93fff640 1 -- 172.21.15.31:0/1703435140 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda9c030ca0 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fda93fff640 1 -- 172.21.15.31:0/1703435140 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9c039450 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.293+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac1a1ec0 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.294+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaac100130 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.295+0000 7fda93fff640 1 -- 172.21.15.31:0/1703435140 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fda9c00a040 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.295+0000 7fda93fff640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fda9406cc50 0x7fda9406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.295+0000 7fda93fff640 1 -- 172.21.15.31:0/1703435140 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fda9c02c040 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.295+0000 7fdaaa59c640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fda9406cc50 0x7fda9406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:22.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.296+0000 7fdaaa59c640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fda9406cc50 0x7fda9406f110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdaac107420 tx=0x7fdaa000a2d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:22.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.300+0000 7fda93fff640 1 -- 172.21.15.31:0/1703435140 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fda9c077950 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.474+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdaac10bc60 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.475+0000 7fda93fff640 1 -- 172.21.15.31:0/1703435140 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11298 (secure 0 0 0) 0x7fda9c07af70 con 0x7fdaac0fe9f0 2024-09-16T08:53:22.477 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:53:22.480 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":41,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","created":"2024-09-16T08:47:02.944131+0000","modified":"2024-09-16T08:53:17.144689+0000","last_up_change":"2024-09-16T08:53:16.136832+0000","last_in_change":"2024-09-16T08:52:57.972093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:52:06.518678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"65c1cf82-b6da-4c51-b5b5-6fa8d6662a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6803","nonce":2831525169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6805","nonce":2831525169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6809","nonce":2831525169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6807","nonce":2831525169}]},"public_addr":"172.21.15.31:6803/2831525169","cluster_addr":"172.21.15.31:6805/2831525169","heartbeat_back_addr":"172.21.15.31:6809/2831525169","heartbeat_front_addr":"172.21.15.31:6807/2831525169","state":["exists","up"]},{"osd":1,"uuid":"f5026ce7-8f65-462a-8e4a-68886fa6a266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6811","nonce":3918099079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6813","nonce":3918099079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6817","nonce":3918099079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6815","nonce":3918099079}]},"public_addr":"172.21.15.31:6811/3918099079","cluster_addr":"172.21.15.31:6813/3918099079","heartbeat_back_addr":"172.21.15.31:6817/3918099079","heartbeat_front_addr":"172.21.15.31:6815/3918099079","state":["exists","up"]},{"osd":2,"uuid":"13430a00-6986-4da5-86fe-aa675b63f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6819","nonce":1861226285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6821","nonce":1861226285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6825","nonce":1861226285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6823","nonce":1861226285}]},"public_addr":"172.21.15.31:6819/1861226285","cluster_addr":"172.21.15.31:6821/1861226285","heartbeat_back_addr":"172.21.15.31:6825/1861226285","heartbeat_front_addr":"172.21.15.31:6823/1861226285","state":["exists","up"]},{"osd":3,"uuid":"cad641c2-0145-4535-bf84-9ea6ce8a7730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6801","nonce":797496523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6803","nonce":797496523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6807","nonce":797496523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6805","nonce":797496523}]},"public_addr":"172.21.15.97:6801/797496523","cluster_addr":"172.21.15.97:6803/797496523","heartbeat_back_addr":"172.21.15.97:6807/797496523","heartbeat_front_addr":"172.21.15.97:6805/797496523","state":["exists","up"]},{"osd":4,"uuid":"fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6809","nonce":2124153532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6811","nonce":2124153532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6815","nonce":2124153532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6813","nonce":2124153532}]},"public_addr":"172.21.15.97:6809/2124153532","cluster_addr":"172.21.15.97:6811/2124153532","heartbeat_back_addr":"172.21.15.97:6815/2124153532","heartbeat_front_addr":"172.21.15.97:6813/2124153532","state":["exists","up"]},{"osd":5,"uuid":"7061f423-5ae3-4a14-b7f9-2713ca675eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6817","nonce":2373483469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6819","nonce":2373483469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6823","nonce":2373483469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6821","nonce":2373483469}]},"public_addr":"172.21.15.97:6817/2373483469","cluster_addr":"172.21.15.97:6819/2373483469","heartbeat_back_addr":"172.21.15.97:6823/2373483469","heartbeat_front_addr":"172.21.15.97:6821/2373483469","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:51:06.500188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:51:34.676646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:02.261985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:25.239670+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:50.384789+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:53:14.865496+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/838457214":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/488511267":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6800/2103860772":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6801/2103860772":"2024-09-17T08:47:27.231716+0000","172.21.15.31:0/3789167564":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/2552418024":"2024-09-17T08:49:08.003661+0000","172.21.15.31:6800/3080093455":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/2621367533":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6801/494755020":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/596031472":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6800/494755020":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/2518533508":"2024-09-17T08:48:00.118833+0000","172.21.15.31:6801/3080093455":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/4102636839":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/1714541219":"2024-09-17T08:48:00.118833+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:53:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fda9406cc50 msgr2=0x7fda9406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fda9406cc50 0x7fda9406f110 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdaac107420 tx=0x7fdaa000a2d0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac0fe9f0 msgr2=0x7fdaac105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac0fe9f0 0x7fdaac105920 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fda9c02ae90 tx=0x7fda9c0308a0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 shutdown_connections 2024-09-16T08:53:22.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fda9406cc50 0x7fda9406f110 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaac1049f0 0x7fdaac105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 --2- 172.21.15.31:0/1703435140 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdaac0fe9f0 0x7fdaac105920 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:22.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 >> 172.21.15.31:0/1703435140 conn(0x7fdaac0758a0 msgr2=0x7fdaac067f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:22.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 shutdown_connections 2024-09-16T08:53:22.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:22.477+0000 7fdab11af640 1 -- 172.21.15.31:0/1703435140 wait complete. 2024-09-16T08:53:23.179 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T08:52:06.518678+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T08:53:23.179 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T08:53:23.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:23 smithi031 ceph-mon[27844]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:23.365 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:23 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1703435140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:53:23.365 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:23.388 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:23.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:23 smithi097 ceph-mon[35478]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:23.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:23 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1703435140' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:53:23.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:23 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:25.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.041+0000 7fd640c93640 1 -- 172.21.15.31:0/2032119291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c100750 msgr2=0x7fd63c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:25.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.041+0000 7fd640c93640 1 --2- 172.21.15.31:0/2032119291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c100750 0x7fd63c1047a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd630009880 tx=0x7fd63002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 -- 172.21.15.31:0/2032119291 shutdown_connections 2024-09-16T08:53:25.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 --2- 172.21.15.31:0/2032119291 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c100750 0x7fd63c1047a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 --2- 172.21.15.31:0/2032119291 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd63c0ffda0 0x7fd63c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 -- 172.21.15.31:0/2032119291 >> 172.21.15.31:0/2032119291 conn(0x7fd63c0fb840 msgr2=0x7fd63c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:25.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 -- 172.21.15.31:0/2032119291 shutdown_connections 2024-09-16T08:53:25.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 -- 172.21.15.31:0/2032119291 wait complete. 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 Processor -- start 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.042+0000 7fd640c93640 1 -- start start 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd640c93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c0ffda0 0x7fd63c0727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd640c93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd63c100750 0x7fd63c072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd640c93640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c073230 con 0x7fd63c0ffda0 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd640c93640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0733a0 con 0x7fd63c100750 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c0ffda0 0x7fd63c0727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:25.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd63a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c0ffda0 0x7fd63c0727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34842/0 (socket says 172.21.15.31:34842) 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd63a59c640 1 -- 172.21.15.31:0/2321785357 learned_addr learned my addr 172.21.15.31:0/2321785357 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd639d9b640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd63c100750 0x7fd63c072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd63a59c640 1 -- 172.21.15.31:0/2321785357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd63c100750 msgr2=0x7fd63c072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd63a59c640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd63c100750 0x7fd63c072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd63a59c640 1 -- 172.21.15.31:0/2321785357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009560 con 0x7fd63c0ffda0 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd639d9b640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd63c100750 0x7fd63c072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd63a59c640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c0ffda0 0x7fd63c0727b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b420 tx=0x7fd62c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd62b7fe640 1 -- 172.21.15.31:0/2321785357 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62c011800 con 0x7fd63c0ffda0 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.043+0000 7fd62b7fe640 1 -- 172.21.15.31:0/2321785357 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd62c011e20 con 0x7fd63c0ffda0 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.044+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c1a1d90 con 0x7fd63c0ffda0 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.044+0000 7fd62b7fe640 1 -- 172.21.15.31:0/2321785357 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd62c00f2e0 con 0x7fd63c0ffda0 2024-09-16T08:53:25.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.044+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1a20f0 con 0x7fd63c0ffda0 2024-09-16T08:53:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.045+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c101280 con 0x7fd63c0ffda0 2024-09-16T08:53:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.045+0000 7fd62b7fe640 1 -- 172.21.15.31:0/2321785357 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd62c00f5c0 con 0x7fd63c0ffda0 2024-09-16T08:53:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.045+0000 7fd62b7fe640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd62406cc50 0x7fd62406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.046+0000 7fd62b7fe640 1 -- 172.21.15.31:0/2321785357 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd62c08b8e0 con 0x7fd63c0ffda0 2024-09-16T08:53:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.046+0000 7fd639d9b640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd62406cc50 0x7fd62406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:25.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.046+0000 7fd639d9b640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd62406cc50 0x7fd62406f110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd630006010 tx=0x7fd630005e60 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:25.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.051+0000 7fd62b7fe640 1 -- 172.21.15.31:0/2321785357 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd62c056060 con 0x7fd63c0ffda0 2024-09-16T08:53:25.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.220+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd63c100200 con 0x7fd63c0ffda0 2024-09-16T08:53:25.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.220+0000 7fd62b7fe640 1 -- 172.21.15.31:0/2321785357 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd63c100200 con 0x7fd63c0ffda0 2024-09-16T08:53:25.232 INFO:teuthology.orchestra.run.smithi031.stdout:pg_num: 1 2024-09-16T08:53:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd62406cc50 msgr2=0x7fd62406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd62406cc50 0x7fd62406f110 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd630006010 tx=0x7fd630005e60 comp rx=0 tx=0).stop 2024-09-16T08:53:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c0ffda0 msgr2=0x7fd63c0727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c0ffda0 0x7fd63c0727b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd62c00b420 tx=0x7fd62c00b8e0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 shutdown_connections 2024-09-16T08:53:25.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd62406cc50 0x7fd62406f110 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd63c100750 0x7fd63c072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 --2- 172.21.15.31:0/2321785357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd63c0ffda0 0x7fd63c0727b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 >> 172.21.15.31:0/2321785357 conn(0x7fd63c0fb840 msgr2=0x7fd63c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 shutdown_connections 2024-09-16T08:53:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:25.222+0000 7fd640c93640 1 -- 172.21.15.31:0/2321785357 wait complete. 2024-09-16T08:53:25.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:25 smithi031 ceph-mon[27844]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:25.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:25 smithi097 ceph-mon[35478]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:25.926 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T08:53:25.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T08:53:26.133 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:26.246 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:26 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2321785357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T08:53:26.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:26 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2321785357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T08:53:27.269 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:27 smithi031 ceph-mon[27844]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:27.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:27 smithi097 ceph-mon[35478]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 -- 172.21.15.31:0/2121135803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4340ffb80 msgr2=0x7fe4340fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 --2- 172.21.15.31:0/2121135803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4340ffb80 0x7fe4340fff80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe430009880 tx=0x7fe43002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 -- 172.21.15.31:0/2121135803 shutdown_connections 2024-09-16T08:53:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 --2- 172.21.15.31:0/2121135803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe434100d80 0x7fe434101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 --2- 172.21.15.31:0/2121135803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4340ffb80 0x7fe4340fff80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 -- 172.21.15.31:0/2121135803 >> 172.21.15.31:0/2121135803 conn(0x7fe4340fb2f0 msgr2=0x7fe4340fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:27.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 -- 172.21.15.31:0/2121135803 shutdown_connections 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 -- 172.21.15.31:0/2121135803 wait complete. 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 Processor -- start 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.779+0000 7fe43b44b640 1 -- start start 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe43b44b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4340ffb80 0x7fe434195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe43b44b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe434100d80 0x7fe434195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe43b44b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434196210 con 0x7fe434100d80 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe43b44b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434196380 con 0x7fe4340ffb80 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4389e6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe434100d80 0x7fe434195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4389e6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe434100d80 0x7fe434195c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34850/0 (socket says 172.21.15.31:34850) 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4389e6640 1 -- 172.21.15.31:0/1145847850 learned_addr learned my addr 172.21.15.31:0/1145847850 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4391e7640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4340ffb80 0x7fe434195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4389e6640 1 -- 172.21.15.31:0/1145847850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4340ffb80 msgr2=0x7fe434195700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4389e6640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4340ffb80 0x7fe434195700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:27.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4389e6640 1 -- 172.21.15.31:0/1145847850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe430009560 con 0x7fe434100d80 2024-09-16T08:53:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe4389e6640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe434100d80 0x7fe434195c40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe42400b450 tx=0x7fe42400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe42a7fc640 1 -- 172.21.15.31:0/1145847850 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe424011860 con 0x7fe434100d80 2024-09-16T08:53:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe42a7fc640 1 -- 172.21.15.31:0/1145847850 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe424011e80 con 0x7fe434100d80 2024-09-16T08:53:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43419ae20 con 0x7fe434100d80 2024-09-16T08:53:27.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.780+0000 7fe42a7fc640 1 -- 172.21.15.31:0/1145847850 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe42400f570 con 0x7fe434100d80 2024-09-16T08:53:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.781+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43419b3c0 con 0x7fe434100d80 2024-09-16T08:53:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.781+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe434100000 con 0x7fe434100d80 2024-09-16T08:53:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.782+0000 7fe42a7fc640 1 -- 172.21.15.31:0/1145847850 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe4240103e0 con 0x7fe434100d80 2024-09-16T08:53:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.782+0000 7fe42a7fc640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe42006cc50 0x7fe42006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.783+0000 7fe4391e7640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe42006cc50 0x7fe42006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.783+0000 7fe42a7fc640 1 -- 172.21.15.31:0/1145847850 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe42408bd60 con 0x7fe434100d80 2024-09-16T08:53:27.784 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.783+0000 7fe4391e7640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe42006cc50 0x7fe42006f110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe430009230 tx=0x7fe430005b50 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:27.787 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:27.787+0000 7fe42a7fc640 1 -- 172.21.15.31:0/1145847850 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe4240564f0 con 0x7fe434100d80 2024-09-16T08:53:28.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.021+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe4341087f0 con 0x7fe434100d80 2024-09-16T08:53:28.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.025+0000 7fe42a7fc640 1 -- 172.21.15.31:0/1145847850 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe424015050 con 0x7fe434100d80 2024-09-16T08:53:28.025 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-16T08:53:28.025 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAI8udmn0FYARAAK8OlNl81gW2pjyj+SLL/cw== 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe42006cc50 msgr2=0x7fe42006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe42006cc50 0x7fe42006f110 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe430009230 tx=0x7fe430005b50 comp rx=0 tx=0).stop 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe434100d80 msgr2=0x7fe434195c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe434100d80 0x7fe434195c40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe42400b450 tx=0x7fe42400b910 comp rx=0 tx=0).stop 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 shutdown_connections 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe42006cc50 0x7fe42006f110 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe434100d80 0x7fe434195c40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 --2- 172.21.15.31:0/1145847850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4340ffb80 0x7fe434195700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 >> 172.21.15.31:0/1145847850 conn(0x7fe4340fb2f0 msgr2=0x7fe434103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 shutdown_connections 2024-09-16T08:53:28.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:28.027+0000 7fe43b44b640 1 -- 172.21.15.31:0/1145847850 wait complete. 2024-09-16T08:53:28.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:28 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1145847850' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:53:28.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:28 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1145847850' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:53:28.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:28 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1145847850' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:53:28.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:28 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1145847850' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:53:28.615 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:53:28.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T08:53:28.615 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T08:53:28.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T08:53:28.867 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi097/config 2024-09-16T08:53:29.252 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:29 smithi097 ceph-mon[35478]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:29.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:29 smithi031 ceph-mon[27844]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 -- 172.21.15.97:0/1980798725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 msgr2=0x7ff354100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 --2- 172.21.15.97:0/1980798725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354100100 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff344009880 tx=0x7ff34402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 -- 172.21.15.97:0/1980798725 shutdown_connections 2024-09-16T08:53:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 --2- 172.21.15.97:0/1980798725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354100100 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.400 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 --2- 172.21.15.97:0/1980798725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3540fea80 0x7ff3540fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 -- 172.21.15.97:0/1980798725 >> 172.21.15.97:0/1980798725 conn(0x7ff3540fa230 msgr2=0x7ff3540fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 -- 172.21.15.97:0/1980798725 shutdown_connections 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 -- 172.21.15.97:0/1980798725 wait complete. 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.386+0000 7ff35baa7640 1 Processor -- start 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff35baa7640 1 -- start start 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff35baa7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3540fea80 0x7ff354195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff35baa7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:30.401 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff35baa7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354196220 con 0x7ff3540fea80 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff35baa7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354196390 con 0x7ff3540ffc80 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359042640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359042640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:49826/0 (socket says 172.21.15.97:49826) 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359042640 1 -- 172.21.15.97:0/3259662772 learned_addr learned my addr 172.21.15.97:0/3259662772 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359843640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3540fea80 0x7ff354195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359042640 1 -- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3540fea80 msgr2=0x7ff354195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359042640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3540fea80 0x7ff354195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359042640 1 -- 172.21.15.97:0/3259662772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff344009560 con 0x7ff3540ffc80 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff359042640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354195c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff344006010 tx=0x7ff3440055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:30.402 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.387+0000 7ff34affd640 1 -- 172.21.15.97:0/3259662772 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff344004b20 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.388+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35419add0 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.388+0000 7ff34affd640 1 -- 172.21.15.97:0/3259662772 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff344004c80 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.388+0000 7ff34affd640 1 -- 172.21.15.97:0/3259662772 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff344003b80 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.388+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35419b340 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.388+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff354107710 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.389+0000 7ff34affd640 1 -- 172.21.15.97:0/3259662772 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff344031640 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.389+0000 7ff34affd640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff34006cc60 0x7ff34006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.389+0000 7ff34affd640 1 -- 172.21.15.97:0/3259662772 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff3440ad220 con 0x7ff3540ffc80 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.390+0000 7ff359843640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff34006cc60 0x7ff34006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.390+0000 7ff359843640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff34006cc60 0x7ff34006f120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3540ffae0 tx=0x7ff350008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:30.403 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.394+0000 7ff34affd640 1 -- 172.21.15.97:0/3259662772 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ff344077930 con 0x7ff3540ffc80 2024-09-16T08:53:30.616 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.616+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff35419b530 con 0x7ff3540ffc80 2024-09-16T08:53:30.622 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.621+0000 7ff34affd640 1 -- 172.21.15.97:0/3259662772 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff34407af50 con 0x7ff3540ffc80 2024-09-16T08:53:30.622 INFO:teuthology.orchestra.run.smithi097.stdout:[client.1] 2024-09-16T08:53:30.622 INFO:teuthology.orchestra.run.smithi097.stdout: key = AQAK8udmZv7SJBAAJ0ocG27J9GC//z/Ro0c5IQ== 2024-09-16T08:53:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff34006cc60 msgr2=0x7ff34006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff34006cc60 0x7ff34006f120 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff3540ffae0 tx=0x7ff350008040 comp rx=0 tx=0).stop 2024-09-16T08:53:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 msgr2=0x7ff354195c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354195c50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff344006010 tx=0x7ff3440055f0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 shutdown_connections 2024-09-16T08:53:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff34006cc60 0x7ff34006f120 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.624 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3540ffc80 0x7ff354195c50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 --2- 172.21.15.97:0/3259662772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3540fea80 0x7ff354195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 >> 172.21.15.97:0/3259662772 conn(0x7ff3540fa230 msgr2=0x7ff3540fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 shutdown_connections 2024-09-16T08:53:30.625 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:53:30.623+0000 7ff35baa7640 1 -- 172.21.15.97:0/3259662772 wait complete. 2024-09-16T08:53:31.193 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:53:31.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T08:53:31.193 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T08:53:31.220 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:31 smithi097 ceph-mon[35478]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:31.220 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:31 smithi097 ceph-mon[35478]: from='client.? 172.21.15.97:0/3259662772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:53:31.220 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:31 smithi097 ceph-mon[35478]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:53:31.220 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:31 smithi097 ceph-mon[35478]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:53:31.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph config log 1 --format=json 2024-09-16T08:53:31.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:31 smithi031 ceph-mon[27844]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:31 smithi031 ceph-mon[27844]: from='client.? 172.21.15.97:0/3259662772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:53:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:31 smithi031 ceph-mon[27844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T08:53:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:31 smithi031 ceph-mon[27844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T08:53:31.449 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:33.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 -- 172.21.15.31:0/1641474900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb68105bf0 msgr2=0x7feb68105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 --2- 172.21.15.31:0/1641474900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb68105bf0 0x7feb68105fd0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7feb58009880 tx=0x7feb5802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 -- 172.21.15.31:0/1641474900 shutdown_connections 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 --2- 172.21.15.31:0/1641474900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb680ffb40 0x7feb680fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 --2- 172.21.15.31:0/1641474900 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb68105bf0 0x7feb68105fd0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 -- 172.21.15.31:0/1641474900 >> 172.21.15.31:0/1641474900 conn(0x7feb680fb7e0 msgr2=0x7feb680fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 -- 172.21.15.31:0/1641474900 shutdown_connections 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.137+0000 7feb6e381640 1 -- 172.21.15.31:0/1641474900 wait complete. 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb6e381640 1 Processor -- start 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb6e381640 1 -- start start 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb6e381640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb680ffb40 0x7feb6819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb6e381640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb68105bf0 0x7feb6819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:33.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb6e381640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6819c2b0 con 0x7feb680ffb40 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb6e381640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb681958d0 con 0x7feb68105bf0 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb67fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb680ffb40 0x7feb6819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb67fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb680ffb40 0x7feb6819b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41008/0 (socket says 172.21.15.31:41008) 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.138+0000 7feb67fff640 1 -- 172.21.15.31:0/3007170084 learned_addr learned my addr 172.21.15.31:0/3007170084 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb677fe640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb68105bf0 0x7feb6819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb67fff640 1 -- 172.21.15.31:0/3007170084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb68105bf0 msgr2=0x7feb6819bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb67fff640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb68105bf0 0x7feb6819bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb67fff640 1 -- 172.21.15.31:0/3007170084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb58009560 con 0x7feb680ffb40 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb67fff640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb680ffb40 0x7feb6819b7e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb5802a9c0 tx=0x7feb580055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb657fa640 1 -- 172.21.15.31:0/3007170084 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb58004aa0 con 0x7feb680ffb40 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb657fa640 1 -- 172.21.15.31:0/3007170084 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb58004c00 con 0x7feb680ffb40 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb68195b50 con 0x7feb680ffb40 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb657fa640 1 -- 172.21.15.31:0/3007170084 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb58003ab0 con 0x7feb680ffb40 2024-09-16T08:53:33.163 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.139+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb681960c0 con 0x7feb680ffb40 2024-09-16T08:53:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.140+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb681011f0 con 0x7feb680ffb40 2024-09-16T08:53:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.141+0000 7feb657fa640 1 -- 172.21.15.31:0/3007170084 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7feb58003ce0 con 0x7feb680ffb40 2024-09-16T08:53:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.141+0000 7feb657fa640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7feb5006cc50 0x7feb5006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.141+0000 7feb657fa640 1 -- 172.21.15.31:0/3007170084 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7feb580ad050 con 0x7feb680ffb40 2024-09-16T08:53:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.141+0000 7feb677fe640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7feb5006cc50 0x7feb5006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.142+0000 7feb677fe640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7feb5006cc50 0x7feb5006f110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005e10 tx=0x7feb5c005da0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:33.164 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.145+0000 7feb657fa640 1 -- 172.21.15.31:0/3007170084 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7feb58077760 con 0x7feb680ffb40 2024-09-16T08:53:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.316+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) v1 -- 0x7feb68196e40 con 0x7feb680ffb40 2024-09-16T08:53:33.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.317+0000 7feb657fa640 1 -- 172.21.15.31:0/3007170084 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v39) v1 ==== 86+0+187 (secure 0 0 0) 0x7feb58081090 con 0x7feb680ffb40 2024-09-16T08:53:33.317 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7feb5006cc50 msgr2=0x7feb5006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7feb5006cc50 0x7feb5006f110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb5c005e10 tx=0x7feb5c005da0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb680ffb40 msgr2=0x7feb6819b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb680ffb40 0x7feb6819b7e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7feb5802a9c0 tx=0x7feb580055a0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 shutdown_connections 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7feb5006cc50 0x7feb5006f110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb68105bf0 0x7feb6819bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 --2- 172.21.15.31:0/3007170084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb680ffb40 0x7feb6819b7e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 >> 172.21.15.31:0/3007170084 conn(0x7feb680fb7e0 msgr2=0x7feb680fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 shutdown_connections 2024-09-16T08:53:33.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:33.319+0000 7feb6e381640 1 -- 172.21.15.31:0/3007170084 wait complete. 2024-09-16T08:53:33.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:33 smithi031 ceph-mon[27844]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:33.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:33 smithi097 ceph-mon[35478]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:33.987 INFO:teuthology.orchestra.run.smithi031.stdout:[{"version":39,"timestamp":"2024-09-16T08:53:18.142690+0000","name":"","changes":[{"name":"osd/host:smithi097/osd_memory_target","previous_value":"7818237337","new_value":"5212158225"}]}] 2024-09-16T08:53:33.988 INFO:tasks.ceph_manager:config epoch is 39 2024-09-16T08:53:33.988 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T08:53:33.988 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T08:53:33.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T08:53:34.197 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:34.318 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:34 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3007170084' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-16T08:53:34.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:34 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3007170084' entity='client.admin' cmd=[{"prefix": "config log", "num": 1, "format": "json"}]: dispatch 2024-09-16T08:53:35.422 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:35 smithi031 ceph-mon[27844]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:35.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:35 smithi097 ceph-mon[35478]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:35.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 -- 172.21.15.31:0/1842132746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c0ffda0 msgr2=0x7f1a3c100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:35.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1842132746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c0ffda0 0x7f1a3c100180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009880 tx=0x7f1a2c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:35.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 -- 172.21.15.31:0/1842132746 shutdown_connections 2024-09-16T08:53:35.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1842132746 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a3c100750 0x7f1a3c1047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:35.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1842132746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c0ffda0 0x7f1a3c100180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:35.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 -- 172.21.15.31:0/1842132746 >> 172.21.15.31:0/1842132746 conn(0x7f1a3c0fb840 msgr2=0x7f1a3c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:35.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 -- 172.21.15.31:0/1842132746 shutdown_connections 2024-09-16T08:53:35.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.918+0000 7f1a40de2640 1 -- 172.21.15.31:0/1842132746 wait complete. 2024-09-16T08:53:35.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a40de2640 1 Processor -- start 2024-09-16T08:53:35.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a40de2640 1 -- start start 2024-09-16T08:53:35.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a40de2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a3c0ffda0 0x7f1a3c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:35.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a40de2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c100750 0x7f1a3c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a40de2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c19c360 con 0x7f1a3c100750 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a40de2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c1958f0 con 0x7f1a3c0ffda0 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a39d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c100750 0x7f1a3c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a39d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c100750 0x7f1a3c19bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41030/0 (socket says 172.21.15.31:41030) 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a39d9b640 1 -- 172.21.15.31:0/1670987207 learned_addr learned my addr 172.21.15.31:0/1670987207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.919+0000 7f1a3a59c640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a3c0ffda0 0x7f1a3c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a39d9b640 1 -- 172.21.15.31:0/1670987207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a3c0ffda0 msgr2=0x7f1a3c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a39d9b640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a3c0ffda0 0x7f1a3c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a39d9b640 1 -- 172.21.15.31:0/1670987207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c009560 con 0x7f1a3c100750 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a39d9b640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c100750 0x7f1a3c19bd40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000d8d0 tx=0x7f1a3000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a2b7fe640 1 -- 172.21.15.31:0/1670987207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3000bea0 con 0x7f1a3c100750 2024-09-16T08:53:35.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c195bd0 con 0x7f1a3c100750 2024-09-16T08:53:35.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a2b7fe640 1 -- 172.21.15.31:0/1670987207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a30009c00 con 0x7f1a3c100750 2024-09-16T08:53:35.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a2b7fe640 1 -- 172.21.15.31:0/1670987207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3000f710 con 0x7f1a3c100750 2024-09-16T08:53:35.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.920+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c196120 con 0x7f1a3c100750 2024-09-16T08:53:35.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.921+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c101280 con 0x7f1a3c100750 2024-09-16T08:53:35.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.921+0000 7f1a2b7fe640 1 -- 172.21.15.31:0/1670987207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1a3000f990 con 0x7f1a3c100750 2024-09-16T08:53:35.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.922+0000 7f1a2b7fe640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1a2406cc50 0x7f1a2406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:35.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.922+0000 7f1a2b7fe640 1 -- 172.21.15.31:0/1670987207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1a3008bbe0 con 0x7f1a3c100750 2024-09-16T08:53:35.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.922+0000 7f1a3a59c640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1a2406cc50 0x7f1a2406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:35.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.923+0000 7f1a3a59c640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1a2406cc50 0x7f1a2406f110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009230 tx=0x7f1a2c0058d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:35.927 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:35.926+0000 7f1a2b7fe640 1 -- 172.21.15.31:0/1670987207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1a300563a0 con 0x7f1a3c100750 2024-09-16T08:53:36.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.143+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1a3c100180 con 0x7f1a3c100750 2024-09-16T08:53:36.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.146+0000 7f1a2b7fe640 1 -- 172.21.15.31:0/1670987207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+174010 (secure 0 0 0) 0x7f1a300599c0 con 0x7f1a3c100750 2024-09-16T08:53:36.150 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:53:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1a2406cc50 msgr2=0x7f1a2406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1a2406cc50 0x7f1a2406f110 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c009230 tx=0x7f1a2c0058d0 comp rx=0 tx=0).stop 2024-09-16T08:53:36.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c100750 msgr2=0x7f1a3c19bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c100750 0x7f1a3c19bd40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000d8d0 tx=0x7f1a3000dd90 comp rx=0 tx=0).stop 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 shutdown_connections 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1a2406cc50 0x7f1a2406f110 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a3c100750 0x7f1a3c19bd40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 --2- 172.21.15.31:0/1670987207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a3c0ffda0 0x7f1a3c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 >> 172.21.15.31:0/1670987207 conn(0x7f1a3c0fb840 msgr2=0x7f1a3c0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 shutdown_connections 2024-09-16T08:53:36.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:36.149+0000 7f1a40de2640 1 -- 172.21.15.31:0/1670987207 wait complete. 2024-09-16T08:53:36.787 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"flags":0,"active_gid":14211,"active_name":"smithi031.dcmqpv","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":2575391456},{"type":"v1","addr":"172.21.15.31:6801","nonce":2575391456}]},"active_addr":"172.21.15.31:6801/2575391456","active_change":"2024-09-16T08:49:08.003771+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14234,"name":"smithi097.mpashf","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.31:8443/","prometheus":"http://172.21.15.31:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":489385232}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":320751586}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":202749585}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2012899681}]}]} 2024-09-16T08:53:36.792 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T08:53:36.792 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T08:53:36.792 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:53:37.009 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:37.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:37 smithi031 ceph-mon[27844]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:37 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1670987207' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T08:53:37.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:37 smithi097 ceph-mon[35478]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:37.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:37 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1670987207' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T08:53:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:38 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:38 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 -- 172.21.15.31:0/1818865058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a80ffda0 msgr2=0x7fd5a8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/1818865058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a80ffda0 0x7fd5a8100180 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009880 tx=0x7fd5a402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 -- 172.21.15.31:0/1818865058 shutdown_connections 2024-09-16T08:53:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/1818865058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a8100750 0x7fd5a81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/1818865058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a80ffda0 0x7fd5a8100180 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 -- 172.21.15.31:0/1818865058 >> 172.21.15.31:0/1818865058 conn(0x7fd5a80fb840 msgr2=0x7fd5a80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 -- 172.21.15.31:0/1818865058 shutdown_connections 2024-09-16T08:53:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.582+0000 7fd5af2b1640 1 -- 172.21.15.31:0/1818865058 wait complete. 2024-09-16T08:53:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5af2b1640 1 Processor -- start 2024-09-16T08:53:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5af2b1640 1 -- start start 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5af2b1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a80ffda0 0x7fd5a819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5af2b1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a8100750 0x7fd5a819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5af2b1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a819c360 con 0x7fd5a8100750 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5af2b1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81958f0 con 0x7fd5a80ffda0 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5ac84c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a8100750 0x7fd5a819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5ac84c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a8100750 0x7fd5a819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41050/0 (socket says 172.21.15.31:41050) 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5ac84c640 1 -- 172.21.15.31:0/3437913637 learned_addr learned my addr 172.21.15.31:0/3437913637 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.583+0000 7fd5ad04d640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a80ffda0 0x7fd5a819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd5ac84c640 1 -- 172.21.15.31:0/3437913637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a80ffda0 msgr2=0x7fd5a819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd5ac84c640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a80ffda0 0x7fd5a819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd5ac84c640 1 -- 172.21.15.31:0/3437913637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4009560 con 0x7fd5a8100750 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd5ac84c640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a8100750 0x7fd5a819bd40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd59800d8d0 tx=0x7fd59800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:38.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd59e7fc640 1 -- 172.21.15.31:0/3437913637 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd59800bea0 con 0x7fd5a8100750 2024-09-16T08:53:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd59e7fc640 1 -- 172.21.15.31:0/3437913637 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd598009c00 con 0x7fd5a8100750 2024-09-16T08:53:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a8195bd0 con 0x7fd5a8100750 2024-09-16T08:53:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a8196120 con 0x7fd5a8100750 2024-09-16T08:53:38.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.584+0000 7fd59e7fc640 1 -- 172.21.15.31:0/3437913637 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd59800f710 con 0x7fd5a8100750 2024-09-16T08:53:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.586+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c005250 con 0x7fd5a8100750 2024-09-16T08:53:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.586+0000 7fd59e7fc640 1 -- 172.21.15.31:0/3437913637 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd59800f940 con 0x7fd5a8100750 2024-09-16T08:53:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.586+0000 7fd59e7fc640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd59406cc50 0x7fd59406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.586+0000 7fd59e7fc640 1 -- 172.21.15.31:0/3437913637 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fd59808b9b0 con 0x7fd5a8100750 2024-09-16T08:53:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.586+0000 7fd5ad04d640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd59406cc50 0x7fd59406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.587+0000 7fd5ad04d640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd59406cc50 0x7fd59406f110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009230 tx=0x7fd5a40058d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.591+0000 7fd59e7fc640 1 -- 172.21.15.31:0/3437913637 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd5980561f0 con 0x7fd5a8100750 2024-09-16T08:53:38.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.778+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd58c0050c0 con 0x7fd5a8100750 2024-09-16T08:53:38.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.779+0000 7fd59e7fc640 1 -- 172.21.15.31:0/3437913637 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11298 (secure 0 0 0) 0x7fd598020090 con 0x7fd5a8100750 2024-09-16T08:53:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:53:38.780 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":41,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","created":"2024-09-16T08:47:02.944131+0000","modified":"2024-09-16T08:53:17.144689+0000","last_up_change":"2024-09-16T08:53:16.136832+0000","last_in_change":"2024-09-16T08:52:57.972093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:52:06.518678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"65c1cf82-b6da-4c51-b5b5-6fa8d6662a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6803","nonce":2831525169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6805","nonce":2831525169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6809","nonce":2831525169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6807","nonce":2831525169}]},"public_addr":"172.21.15.31:6803/2831525169","cluster_addr":"172.21.15.31:6805/2831525169","heartbeat_back_addr":"172.21.15.31:6809/2831525169","heartbeat_front_addr":"172.21.15.31:6807/2831525169","state":["exists","up"]},{"osd":1,"uuid":"f5026ce7-8f65-462a-8e4a-68886fa6a266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6811","nonce":3918099079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6813","nonce":3918099079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6817","nonce":3918099079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6815","nonce":3918099079}]},"public_addr":"172.21.15.31:6811/3918099079","cluster_addr":"172.21.15.31:6813/3918099079","heartbeat_back_addr":"172.21.15.31:6817/3918099079","heartbeat_front_addr":"172.21.15.31:6815/3918099079","state":["exists","up"]},{"osd":2,"uuid":"13430a00-6986-4da5-86fe-aa675b63f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6819","nonce":1861226285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6821","nonce":1861226285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6825","nonce":1861226285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6823","nonce":1861226285}]},"public_addr":"172.21.15.31:6819/1861226285","cluster_addr":"172.21.15.31:6821/1861226285","heartbeat_back_addr":"172.21.15.31:6825/1861226285","heartbeat_front_addr":"172.21.15.31:6823/1861226285","state":["exists","up"]},{"osd":3,"uuid":"cad641c2-0145-4535-bf84-9ea6ce8a7730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6801","nonce":797496523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6803","nonce":797496523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6807","nonce":797496523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6805","nonce":797496523}]},"public_addr":"172.21.15.97:6801/797496523","cluster_addr":"172.21.15.97:6803/797496523","heartbeat_back_addr":"172.21.15.97:6807/797496523","heartbeat_front_addr":"172.21.15.97:6805/797496523","state":["exists","up"]},{"osd":4,"uuid":"fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6809","nonce":2124153532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6811","nonce":2124153532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6815","nonce":2124153532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6813","nonce":2124153532}]},"public_addr":"172.21.15.97:6809/2124153532","cluster_addr":"172.21.15.97:6811/2124153532","heartbeat_back_addr":"172.21.15.97:6815/2124153532","heartbeat_front_addr":"172.21.15.97:6813/2124153532","state":["exists","up"]},{"osd":5,"uuid":"7061f423-5ae3-4a14-b7f9-2713ca675eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6817","nonce":2373483469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6819","nonce":2373483469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6823","nonce":2373483469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6821","nonce":2373483469}]},"public_addr":"172.21.15.97:6817/2373483469","cluster_addr":"172.21.15.97:6819/2373483469","heartbeat_back_addr":"172.21.15.97:6823/2373483469","heartbeat_front_addr":"172.21.15.97:6821/2373483469","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:51:06.500188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:51:34.676646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:02.261985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:25.239670+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:50.384789+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:53:14.865496+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/838457214":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/488511267":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6800/2103860772":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6801/2103860772":"2024-09-17T08:47:27.231716+0000","172.21.15.31:0/3789167564":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/2552418024":"2024-09-17T08:49:08.003661+0000","172.21.15.31:6800/3080093455":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/2621367533":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6801/494755020":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/596031472":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6800/494755020":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/2518533508":"2024-09-17T08:48:00.118833+0000","172.21.15.31:6801/3080093455":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/4102636839":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/1714541219":"2024-09-17T08:48:00.118833+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:53:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd59406cc50 msgr2=0x7fd59406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd59406cc50 0x7fd59406f110 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009230 tx=0x7fd5a40058d0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a8100750 msgr2=0x7fd5a819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a8100750 0x7fd5a819bd40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd59800d8d0 tx=0x7fd59800dd90 comp rx=0 tx=0).stop 2024-09-16T08:53:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 shutdown_connections 2024-09-16T08:53:38.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd59406cc50 0x7fd59406f110 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5a8100750 0x7fd5a819bd40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 --2- 172.21.15.31:0/3437913637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd5a80ffda0 0x7fd5a819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:38.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 >> 172.21.15.31:0/3437913637 conn(0x7fd5a80fb840 msgr2=0x7fd5a80fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:38.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 shutdown_connections 2024-09-16T08:53:38.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:38.781+0000 7fd5af2b1640 1 -- 172.21.15.31:0/3437913637 wait complete. 2024-09-16T08:53:39.350 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:39 smithi031 ceph-mon[27844]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:39.351 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:39 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3437913637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:53:39.374 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T08:53:39.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T08:53:39.412 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:39 smithi097 ceph-mon[35478]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:39.412 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:39 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3437913637' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:53:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:41.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1266932065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 msgr2=0x7fdef80fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1266932065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef80fee50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009880 tx=0x7fdee802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1266932065 shutdown_connections 2024-09-16T08:53:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1266932065 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef80fee50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1266932065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef81049f0 0x7fdef8104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1266932065 >> 172.21.15.31:0/1266932065 conn(0x7fdef80758a0 msgr2=0x7fdef80fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1266932065 shutdown_connections 2024-09-16T08:53:41.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.168+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1266932065 wait complete. 2024-09-16T08:53:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.169+0000 7fdeff3b7640 1 Processor -- start 2024-09-16T08:53:41.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.169+0000 7fdeff3b7640 1 -- start start 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.169+0000 7fdeff3b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef8197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.169+0000 7fdeff3b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef81049f0 0x7fdef8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.169+0000 7fdeff3b7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81988f0 con 0x7fdef80fe9f0 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.169+0000 7fdeff3b7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef819b560 con 0x7fdef81049f0 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefd153640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef8197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefd153640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef8197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41082/0 (socket says 172.21.15.31:41082) 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefd153640 1 -- 172.21.15.31:0/1422349591 learned_addr learned my addr 172.21.15.31:0/1422349591 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefc952640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef81049f0 0x7fdef8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefd153640 1 -- 172.21.15.31:0/1422349591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef81049f0 msgr2=0x7fdef8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefd153640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef81049f0 0x7fdef8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefd153640 1 -- 172.21.15.31:0/1422349591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee8009560 con 0x7fdef80fe9f0 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdefd153640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef8197cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e970 tx=0x7fdef400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:41.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdeee7fc640 1 -- 172.21.15.31:0/1422349591 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef400cc90 con 0x7fdef80fe9f0 2024-09-16T08:53:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef819b840 con 0x7fdef80fe9f0 2024-09-16T08:53:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdeee7fc640 1 -- 172.21.15.31:0/1422349591 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdef400cdf0 con 0x7fdef80fe9f0 2024-09-16T08:53:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdeee7fc640 1 -- 172.21.15.31:0/1422349591 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef4010430 con 0x7fdef80fe9f0 2024-09-16T08:53:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.170+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef819bd60 con 0x7fdef80fe9f0 2024-09-16T08:53:41.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.172+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef8100130 con 0x7fdef80fe9f0 2024-09-16T08:53:41.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.172+0000 7fdeee7fc640 1 -- 172.21.15.31:0/1422349591 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdef40106f0 con 0x7fdef80fe9f0 2024-09-16T08:53:41.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.172+0000 7fdeee7fc640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdee406cc50 0x7fdee406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:41.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.173+0000 7fdeee7fc640 1 -- 172.21.15.31:0/1422349591 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdef4014070 con 0x7fdef80fe9f0 2024-09-16T08:53:41.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.173+0000 7fdefc952640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdee406cc50 0x7fdee406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:41.174 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.173+0000 7fdefc952640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdee406cc50 0x7fdee406f110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009230 tx=0x7fdee80058d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:41.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.177+0000 7fdeee7fc640 1 -- 172.21.15.31:0/1422349591 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdef4056b80 con 0x7fdef80fe9f0 2024-09-16T08:53:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.348+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdef810bc80 con 0x7fdef80fe9f0 2024-09-16T08:53:41.349 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:41 smithi031 ceph-mon[27844]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:41.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.349+0000 7fdeee7fc640 1 -- 172.21.15.31:0/1422349591 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11298 (secure 0 0 0) 0x7fdef405a1a0 con 0x7fdef80fe9f0 2024-09-16T08:53:41.350 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:53:41.350 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":41,"fsid":"1631ad7a-7408-11ef-bceb-c7b262605968","created":"2024-09-16T08:47:02.944131+0000","modified":"2024-09-16T08:53:17.144689+0000","last_up_change":"2024-09-16T08:53:16.136832+0000","last_in_change":"2024-09-16T08:52:57.972093+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T08:52:06.518678+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"65c1cf82-b6da-4c51-b5b5-6fa8d6662a75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6803","nonce":2831525169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6805","nonce":2831525169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6809","nonce":2831525169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":2831525169},{"type":"v1","addr":"172.21.15.31:6807","nonce":2831525169}]},"public_addr":"172.21.15.31:6803/2831525169","cluster_addr":"172.21.15.31:6805/2831525169","heartbeat_back_addr":"172.21.15.31:6809/2831525169","heartbeat_front_addr":"172.21.15.31:6807/2831525169","state":["exists","up"]},{"osd":1,"uuid":"f5026ce7-8f65-462a-8e4a-68886fa6a266","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6811","nonce":3918099079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6813","nonce":3918099079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6817","nonce":3918099079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":3918099079},{"type":"v1","addr":"172.21.15.31:6815","nonce":3918099079}]},"public_addr":"172.21.15.31:6811/3918099079","cluster_addr":"172.21.15.31:6813/3918099079","heartbeat_back_addr":"172.21.15.31:6817/3918099079","heartbeat_front_addr":"172.21.15.31:6815/3918099079","state":["exists","up"]},{"osd":2,"uuid":"13430a00-6986-4da5-86fe-aa675b63f155","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6819","nonce":1861226285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6821","nonce":1861226285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6825","nonce":1861226285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":1861226285},{"type":"v1","addr":"172.21.15.31:6823","nonce":1861226285}]},"public_addr":"172.21.15.31:6819/1861226285","cluster_addr":"172.21.15.31:6821/1861226285","heartbeat_back_addr":"172.21.15.31:6825/1861226285","heartbeat_front_addr":"172.21.15.31:6823/1861226285","state":["exists","up"]},{"osd":3,"uuid":"cad641c2-0145-4535-bf84-9ea6ce8a7730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6801","nonce":797496523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6803","nonce":797496523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6807","nonce":797496523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":797496523},{"type":"v1","addr":"172.21.15.97:6805","nonce":797496523}]},"public_addr":"172.21.15.97:6801/797496523","cluster_addr":"172.21.15.97:6803/797496523","heartbeat_back_addr":"172.21.15.97:6807/797496523","heartbeat_front_addr":"172.21.15.97:6805/797496523","state":["exists","up"]},{"osd":4,"uuid":"fe11cc1f-6391-4dc0-bc3b-5d7bb88f7d51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6809","nonce":2124153532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6811","nonce":2124153532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6815","nonce":2124153532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":2124153532},{"type":"v1","addr":"172.21.15.97:6813","nonce":2124153532}]},"public_addr":"172.21.15.97:6809/2124153532","cluster_addr":"172.21.15.97:6811/2124153532","heartbeat_back_addr":"172.21.15.97:6815/2124153532","heartbeat_front_addr":"172.21.15.97:6813/2124153532","state":["exists","up"]},{"osd":5,"uuid":"7061f423-5ae3-4a14-b7f9-2713ca675eec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6817","nonce":2373483469}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6819","nonce":2373483469}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6823","nonce":2373483469}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2373483469},{"type":"v1","addr":"172.21.15.97:6821","nonce":2373483469}]},"public_addr":"172.21.15.97:6817/2373483469","cluster_addr":"172.21.15.97:6819/2373483469","heartbeat_back_addr":"172.21.15.97:6823/2373483469","heartbeat_front_addr":"172.21.15.97:6821/2373483469","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:51:06.500188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:51:34.676646+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:02.261985+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:25.239670+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:52:50.384789+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T08:53:14.865496+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:0/838457214":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/488511267":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6800/2103860772":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6801/2103860772":"2024-09-17T08:47:27.231716+0000","172.21.15.31:0/3789167564":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/2552418024":"2024-09-17T08:49:08.003661+0000","172.21.15.31:6800/3080093455":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/2621367533":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6801/494755020":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/596031472":"2024-09-17T08:47:27.231716+0000","172.21.15.31:6800/494755020":"2024-09-17T08:48:00.118833+0000","172.21.15.31:0/2518533508":"2024-09-17T08:48:00.118833+0000","172.21.15.31:6801/3080093455":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/4102636839":"2024-09-17T08:49:08.003661+0000","172.21.15.31:0/1714541219":"2024-09-17T08:48:00.118833+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T08:53:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.351+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdee406cc50 msgr2=0x7fdee406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.351+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdee406cc50 0x7fdee406f110 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009230 tx=0x7fdee80058d0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.351+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 msgr2=0x7fdef8197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef8197cd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdef400e970 tx=0x7fdef400ed70 comp rx=0 tx=0).stop 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 shutdown_connections 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdee406cc50 0x7fdee406f110 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef81049f0 0x7fdef8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 --2- 172.21.15.31:0/1422349591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdef80fe9f0 0x7fdef8197cd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 >> 172.21.15.31:0/1422349591 conn(0x7fdef80758a0 msgr2=0x7fdef8067c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 shutdown_connections 2024-09-16T08:53:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:41.352+0000 7fdeff3b7640 1 -- 172.21.15.31:0/1422349591 wait complete. 2024-09-16T08:53:41.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:41 smithi097 ceph-mon[35478]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:41.962 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T08:53:41.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T08:53:41.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T08:53:41.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T08:53:41.964 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T08:53:41.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T08:53:42.280 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:42.283 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:42.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:42 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1422349591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:53:42.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:42 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1422349591' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T08:53:42.587 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:42.588 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:42.588 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:42.589 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:43.320 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:43 smithi031 ceph-mon[27844]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:43.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:43 smithi097 ceph-mon[35478]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:44.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:44 smithi031 ceph-mon[27844]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:44.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:44 smithi097 ceph-mon[35478]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:47.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:47 smithi031 ceph-mon[27844]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:47.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:47 smithi097 ceph-mon[35478]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:47.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 -- 172.21.15.31:0/3218030132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47440ffda0 msgr2=0x7f4744100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:47.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 --2- 172.21.15.31:0/3218030132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47440ffda0 0x7f4744100180 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4734009880 tx=0x7f473402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 -- 172.21.15.31:0/3218030132 shutdown_connections 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 --2- 172.21.15.31:0/3218030132 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4744100750 0x7f47441047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 --2- 172.21.15.31:0/3218030132 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47440ffda0 0x7f4744100180 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 -- 172.21.15.31:0/3218030132 >> 172.21.15.31:0/3218030132 conn(0x7f47440fb840 msgr2=0x7f47440fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 -- 172.21.15.31:0/3218030132 shutdown_connections 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.865+0000 7f474a74b640 1 -- 172.21.15.31:0/3218030132 wait complete. 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f474a74b640 1 Processor -- start 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f474a74b640 1 -- start start 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f474a74b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47440ffda0 0x7f474419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f474a74b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4744100750 0x7f474419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f474a74b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744197b40 con 0x7f4744100750 2024-09-16T08:53:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f474a74b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4744197cb0 con 0x7f47440ffda0 2024-09-16T08:53:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f47437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4744100750 0x7f474419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f47437fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4744100750 0x7f474419df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57062/0 (socket says 172.21.15.31:57062) 2024-09-16T08:53:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f47437fe640 1 -- 172.21.15.31:0/3652362917 learned_addr learned my addr 172.21.15.31:0/3652362917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.866+0000 7f4743fff640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47440ffda0 0x7f474419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f47437fe640 1 -- 172.21.15.31:0/3652362917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47440ffda0 msgr2=0x7f474419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f47437fe640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47440ffda0 0x7f474419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f47437fe640 1 -- 172.21.15.31:0/3652362917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4734009560 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f47437fe640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4744100750 0x7f474419df90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f473800c8f0 tx=0x7f473800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4738007b80 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47380054e0 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f473800f450 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.867+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4744197f90 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.868+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4744198490 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.868+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4744101280 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.869+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4738004f60 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.869+0000 7f47417fa640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f472c06cc50 0x7f472c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f473808bcc0 con 0x7f4744100750 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f4743fff640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f472c06cc50 0x7f472c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f47417fa640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] conn(0x7f472c0729e0 0x7f472c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 --> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f472c0754b0 con 0x7f472c0729e0 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f473808f020 con 0x7f4744100750 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f4748ce8640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] conn(0x7f472c0729e0 0x7f472c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f4743fff640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f472c06cc50 0x7f472c06f110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4734009230 tx=0x7f47340058d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.870+0000 7f4748ce8640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] conn(0x7f472c0729e0 0x7f472c074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.871+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== osd.0 v2:172.21.15.31:6802/2831525169 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f472c0754b0 con 0x7f472c0729e0 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 --> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47441989a0 con 0x7f472c0729e0 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f47417fa640 1 -- 172.21.15.31:0/3652362917 <== osd.0 v2:172.21.15.31:6802/2831525169 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f47441989a0 con 0x7f472c0729e0 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] conn(0x7f472c0729e0 msgr2=0x7f472c074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f474a74b640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] conn(0x7f472c0729e0 0x7f472c074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f472c06cc50 msgr2=0x7f472c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f474a74b640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f472c06cc50 0x7f472c06f110 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4734009230 tx=0x7f47340058d0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4744100750 msgr2=0x7f474419df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.884+0000 7f474a74b640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4744100750 0x7f474419df90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f473800c8f0 tx=0x7f473800cdb0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 shutdown_connections 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6802/2831525169,v1:172.21.15.31:6803/2831525169] conn(0x7f472c0729e0 0x7f472c074e00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f472c06cc50 0x7f472c06f110 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4744100750 0x7f474419df90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 --2- 172.21.15.31:0/3652362917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47440ffda0 0x7f474419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 >> 172.21.15.31:0/3652362917 conn(0x7f47440fb840 msgr2=0x7f47440fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:47.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 shutdown_connections 2024-09-16T08:53:47.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:47.885+0000 7f474a74b640 1 -- 172.21.15.31:0/3652362917 wait complete. 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 -- 172.21.15.31:0/2742867605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 msgr2=0x7f1618104f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 --2- 172.21.15.31:0/2742867605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 0x7f1618104f40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1608009880 tx=0x7f160802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 -- 172.21.15.31:0/2742867605 shutdown_connections 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 --2- 172.21.15.31:0/2742867605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16180fdf80 0x7f16180fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 --2- 172.21.15.31:0/2742867605 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 0x7f1618104f40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 -- 172.21.15.31:0/2742867605 >> 172.21.15.31:0/2742867605 conn(0x7f16180f9e70 msgr2=0x7f16180fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 -- 172.21.15.31:0/2742867605 shutdown_connections 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.327+0000 7f161e1ae640 1 -- 172.21.15.31:0/2742867605 wait complete. 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.328+0000 7f161e1ae640 1 Processor -- start 2024-09-16T08:53:48.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.328+0000 7f161e1ae640 1 -- start start 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.328+0000 7f161e1ae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16180fdf80 0x7f16180ff160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.328+0000 7f161e1ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 0x7f16180ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.328+0000 7f161e1ae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16180ffbe0 con 0x7f1618104b60 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.328+0000 7f161e1ae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16180ffd50 con 0x7f16180fdf80 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f1616ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 0x7f16180ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f16177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16180fdf80 0x7f16180ff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f16177fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16180fdf80 0x7f16180ff160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:47440/0 (socket says 172.21.15.31:47440) 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f16177fe640 1 -- 172.21.15.31:0/110934435 learned_addr learned my addr 172.21.15.31:0/110934435 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:48.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f1616ffd640 1 -- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16180fdf80 msgr2=0x7f16180ff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f1616ffd640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16180fdf80 0x7f16180ff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f1616ffd640 1 -- 172.21.15.31:0/110934435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608009560 con 0x7f1618104b60 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.329+0000 7f1616ffd640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 0x7f16180ff6a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f160c00d6d0 tx=0x7f160c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.330+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160c0051f0 con 0x7f1618104b60 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.330+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f160c004c60 con 0x7f1618104b60 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.330+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16181a1d80 con 0x7f1618104b60 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.330+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160c009d30 con 0x7f1618104b60 2024-09-16T08:53:48.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.330+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16181a20e0 con 0x7f1618104b60 2024-09-16T08:53:48.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.331+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f161810d0e0 con 0x7f1618104b60 2024-09-16T08:53:48.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.331+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f160c0047a0 con 0x7f1618104b60 2024-09-16T08:53:48.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f1614ff9640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f160006cc50 0x7f160006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:48.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f160c08b9b0 con 0x7f1618104b60 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f16177fe640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f160006cc50 0x7f160006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f1614ff9640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] conn(0x7f16000729e0 0x7f1600074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 --> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16000754b0 con 0x7f16000729e0 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f160c0902a0 con 0x7f1618104b60 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f1617fff640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] conn(0x7f16000729e0 0x7f1600074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.332+0000 7f16177fe640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f160006cc50 0x7f160006f110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1608009230 tx=0x7f1608005a20 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.333+0000 7f1617fff640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] conn(0x7f16000729e0 0x7f1600074e00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:48.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.333+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== osd.5 v2:172.21.15.97:6816/2373483469 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f16000754b0 con 0x7f16000729e0 2024-09-16T08:53:48.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.347+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 --> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f16180fe3e0 con 0x7f16000729e0 2024-09-16T08:53:48.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.347+0000 7f1614ff9640 1 -- 172.21.15.31:0/110934435 <== osd.5 v2:172.21.15.97:6816/2373483469 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f16180fe3e0 con 0x7f16000729e0 2024-09-16T08:53:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.347+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] conn(0x7f16000729e0 msgr2=0x7f1600074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.347+0000 7f161e1ae640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] conn(0x7f16000729e0 0x7f1600074e00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f160006cc50 msgr2=0x7f160006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f160006cc50 0x7f160006f110 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1608009230 tx=0x7f1608005a20 comp rx=0 tx=0).stop 2024-09-16T08:53:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 msgr2=0x7f16180ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 0x7f16180ff6a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f160c00d6d0 tx=0x7f160c00db90 comp rx=0 tx=0).stop 2024-09-16T08:53:48.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 shutdown_connections 2024-09-16T08:53:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:6816/2373483469,v1:172.21.15.97:6817/2373483469] conn(0x7f16000729e0 0x7f1600074e00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f160006cc50 0x7f160006f110 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1618104b60 0x7f16180ff6a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 --2- 172.21.15.31:0/110934435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f16180fdf80 0x7f16180ff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 >> 172.21.15.31:0/110934435 conn(0x7f16180f9e70 msgr2=0x7f16180fab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 shutdown_connections 2024-09-16T08:53:48.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:48.348+0000 7f161e1ae640 1 -- 172.21.15.31:0/110934435 wait complete. 2024-09-16T08:53:49.282 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:49 smithi031 ceph-mon[27844]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:49.303 INFO:teuthology.orchestra.run.smithi031.stdout:171798691849 2024-09-16T08:53:49.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T08:53:49.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:49 smithi097 ceph-mon[35478]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:50.217 INFO:teuthology.orchestra.run.smithi031.stdout:38654705697 2024-09-16T08:53:50.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T08:53:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 -- 172.21.15.31:0/1048025148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b401049f0 msgr2=0x7f4b40104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 --2- 172.21.15.31:0/1048025148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b401049f0 0x7f4b40104dd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4b34009880 tx=0x7f4b3402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 -- 172.21.15.31:0/1048025148 shutdown_connections 2024-09-16T08:53:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 --2- 172.21.15.31:0/1048025148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b400fe9f0 0x7f4b400fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 --2- 172.21.15.31:0/1048025148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b401049f0 0x7f4b40104dd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 -- 172.21.15.31:0/1048025148 >> 172.21.15.31:0/1048025148 conn(0x7f4b400758a0 msgr2=0x7f4b400fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 -- 172.21.15.31:0/1048025148 shutdown_connections 2024-09-16T08:53:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.145+0000 7f4b4840d640 1 -- 172.21.15.31:0/1048025148 wait complete. 2024-09-16T08:53:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b4840d640 1 Processor -- start 2024-09-16T08:53:51.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b4840d640 1 -- start start 2024-09-16T08:53:51.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b4840d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b400fe9f0 0x7f4b40109980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:51.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b4840d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401049f0 0x7f4b40109ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b4840d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4010a400 con 0x7f4b400fe9f0 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b4840d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4010a570 con 0x7f4b401049f0 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b461a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b400fe9f0 0x7f4b40109980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b461a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b400fe9f0 0x7f4b40109980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57090/0 (socket says 172.21.15.31:57090) 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b461a9640 1 -- 172.21.15.31:0/3962130472 learned_addr learned my addr 172.21.15.31:0/3962130472 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b459a8640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401049f0 0x7f4b40109ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b461a9640 1 -- 172.21.15.31:0/3962130472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401049f0 msgr2=0x7f4b40109ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b461a9640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401049f0 0x7f4b40109ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b461a9640 1 -- 172.21.15.31:0/3962130472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b34009560 con 0x7f4b400fe9f0 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.146+0000 7f4b459a8640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401049f0 0x7f4b40109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.147+0000 7f4b461a9640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b400fe9f0 0x7f4b40109980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402a9c0 tx=0x7f4b340055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.147+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b34004aa0 con 0x7f4b400fe9f0 2024-09-16T08:53:51.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.147+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b4010a7f0 con 0x7f4b400fe9f0 2024-09-16T08:53:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.147+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b34004c00 con 0x7f4b400fe9f0 2024-09-16T08:53:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.147+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b34003ab0 con 0x7f4b400fe9f0 2024-09-16T08:53:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.147+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b401a1ff0 con 0x7f4b400fe9f0 2024-09-16T08:53:51.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.148+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b4010d0f0 con 0x7f4b400fe9f0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.149+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4b34031d20 con 0x7f4b400fe9f0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.149+0000 7f4b337fe640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4b2c06cc50 0x7f4b2c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.149+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4b340ace60 con 0x7f4b400fe9f0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.149+0000 7f4b459a8640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4b2c06cc50 0x7f4b2c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.150+0000 7f4b337fe640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] conn(0x7f4b2c0729e0 0x7f4b2c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.150+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 --> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b2c0754b0 con 0x7f4b2c0729e0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.150+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b340b2540 con 0x7f4b400fe9f0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.150+0000 7f4b469aa640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] conn(0x7f4b2c0729e0 0x7f4b2c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.150+0000 7f4b459a8640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4b2c06cc50 0x7f4b2c06f110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c006fd0 tx=0x7f4b3c008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:51.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.150+0000 7f4b469aa640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] conn(0x7f4b2c0729e0 0x7f4b2c074e00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:51.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.151+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== osd.1 v2:172.21.15.31:6810/3918099079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f4b2c0754b0 con 0x7f4b2c0729e0 2024-09-16T08:53:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.166+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 --> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4b4010a7f0 con 0x7f4b2c0729e0 2024-09-16T08:53:51.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.166+0000 7f4b337fe640 1 -- 172.21.15.31:0/3962130472 <== osd.1 v2:172.21.15.31:6810/3918099079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4b4010a7f0 con 0x7f4b2c0729e0 2024-09-16T08:53:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.166+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] conn(0x7f4b2c0729e0 msgr2=0x7f4b2c074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.166+0000 7f4b4840d640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] conn(0x7f4b2c0729e0 0x7f4b2c074e00 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.166+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4b2c06cc50 msgr2=0x7f4b2c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4b2c06cc50 0x7f4b2c06f110 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c006fd0 tx=0x7f4b3c008040 comp rx=0 tx=0).stop 2024-09-16T08:53:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b400fe9f0 msgr2=0x7f4b40109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:51.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b400fe9f0 0x7f4b40109980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4b3402a9c0 tx=0x7f4b340055a0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 shutdown_connections 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6810/3918099079,v1:172.21.15.31:6811/3918099079] conn(0x7f4b2c0729e0 0x7f4b2c074e00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4b2c06cc50 0x7f4b2c06f110 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b401049f0 0x7f4b40109ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 --2- 172.21.15.31:0/3962130472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b400fe9f0 0x7f4b40109980 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 >> 172.21.15.31:0/3962130472 conn(0x7f4b400758a0 msgr2=0x7f4b400fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 shutdown_connections 2024-09-16T08:53:51.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.167+0000 7f4b4840d640 1 -- 172.21.15.31:0/3962130472 wait complete. 2024-09-16T08:53:51.250 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:51 smithi031 ceph-mon[27844]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:51.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:51 smithi097 ceph-mon[35478]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:52.366 INFO:teuthology.orchestra.run.smithi031.stdout:60129542174 2024-09-16T08:53:52.367 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T08:53:52.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 -- 172.21.15.31:0/767083611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 msgr2=0x7ff1c8104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/767083611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8104dd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8009c20 tx=0x7ff1b802a970 comp rx=0 tx=0).stop 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 -- 172.21.15.31:0/767083611 shutdown_connections 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/767083611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c80fe9f0 0x7ff1c80fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/767083611 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8104dd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 -- 172.21.15.31:0/767083611 >> 172.21.15.31:0/767083611 conn(0x7ff1c80758a0 msgr2=0x7ff1c80fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 -- 172.21.15.31:0/767083611 shutdown_connections 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 -- 172.21.15.31:0/767083611 wait complete. 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 Processor -- start 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.577+0000 7ff1d00d2640 1 -- start start 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1d00d2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c80fe9f0 0x7ff1c8196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:52.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1d00d2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1d00d2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c8197450 con 0x7ff1c81049f0 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1d00d2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c8197590 con 0x7ff1c80fe9f0 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cd66d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cd66d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60408/0 (socket says 172.21.15.31:60408) 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cd66d640 1 -- 172.21.15.31:0/1218160965 learned_addr learned my addr 172.21.15.31:0/1218160965 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cde6e640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c80fe9f0 0x7ff1c8196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cd66d640 1 -- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c80fe9f0 msgr2=0x7ff1c8196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cd66d640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c80fe9f0 0x7ff1c8196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cd66d640 1 -- 172.21.15.31:0/1218160965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b8009900 con 0x7ff1c81049f0 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1cd66d640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8196e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00d8d0 tx=0x7ff1bc00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1bc009c00 con 0x7ff1c81049f0 2024-09-16T08:53:52.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1bc009d60 con 0x7ff1c81049f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.578+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c819f9b0 con 0x7ff1c81049f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.579+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1bc0049d0 con 0x7ff1c81049f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.579+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c819fdd0 con 0x7ff1c81049f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.579+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1c810bee0 con 0x7ff1c81049f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.580+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ff1bc004c90 con 0x7ff1c81049f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1c6ffd640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff1b006cc60 0x7ff1b006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff1bc08bc00 con 0x7ff1c81049f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1cde6e640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff1b006cc60 0x7ff1b006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1c6ffd640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] conn(0x7ff1b00729f0 0x7ff1b0074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 --> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1b00754c0 con 0x7ff1b00729f0 2024-09-16T08:53:52.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1bc0912e0 con 0x7ff1c81049f0 2024-09-16T08:53:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1cde6e640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff1b006cc60 0x7ff1b006f120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8000f80 tx=0x7ff1b803d040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.581+0000 7ff1ce66f640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] conn(0x7ff1b00729f0 0x7ff1b0074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.582+0000 7ff1ce66f640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] conn(0x7ff1b00729f0 0x7ff1b0074e10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:52.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.582+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== osd.4 v2:172.21.15.97:6808/2124153532 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff1b00754c0 con 0x7ff1b00729f0 2024-09-16T08:53:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.596+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 --> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff1c8060b30 con 0x7ff1b00729f0 2024-09-16T08:53:52.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.596+0000 7ff1c6ffd640 1 -- 172.21.15.31:0/1218160965 <== osd.4 v2:172.21.15.97:6808/2124153532 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff1c8060b30 con 0x7ff1b00729f0 2024-09-16T08:53:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] conn(0x7ff1b00729f0 msgr2=0x7ff1b0074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] conn(0x7ff1b00729f0 0x7ff1b0074e10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff1b006cc60 msgr2=0x7ff1b006f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff1b006cc60 0x7ff1b006f120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8000f80 tx=0x7ff1b803d040 comp rx=0 tx=0).stop 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 msgr2=0x7ff1c8196e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8196e80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc00d8d0 tx=0x7ff1bc00dd90 comp rx=0 tx=0).stop 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 shutdown_connections 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:6808/2124153532,v1:172.21.15.97:6809/2124153532] conn(0x7ff1b00729f0 0x7ff1b0074e10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ff1b006cc60 0x7ff1b006f120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1c81049f0 0x7ff1c8196e80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 --2- 172.21.15.31:0/1218160965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff1c80fe9f0 0x7ff1c8196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 >> 172.21.15.31:0/1218160965 conn(0x7ff1c80758a0 msgr2=0x7ff1c80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 shutdown_connections 2024-09-16T08:53:52.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:52.597+0000 7ff1d00d2640 1 -- 172.21.15.31:0/1218160965 wait complete. 2024-09-16T08:53:53.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:53 smithi031 ceph-mon[27844]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:53.165 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:53.186 INFO:teuthology.orchestra.run.smithi031.stdout:150323855375 2024-09-16T08:53:53.187 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T08:53:53.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:53 smithi097 ceph-mon[35478]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:53.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.686+0000 7f59d587d640 1 -- 172.21.15.31:0/1546445821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d0100cb0 msgr2=0x7f59d0101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.686+0000 7f59d587d640 1 --2- 172.21.15.31:0/1546445821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d0100cb0 0x7f59d0101090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009c20 tx=0x7f59c002aa00 comp rx=0 tx=0).stop 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.686+0000 7f59d587d640 1 -- 172.21.15.31:0/1546445821 shutdown_connections 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.686+0000 7f59d587d640 1 --2- 172.21.15.31:0/1546445821 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d01015d0 0x7f59d0109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.686+0000 7f59d587d640 1 --2- 172.21.15.31:0/1546445821 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d0100cb0 0x7f59d0101090 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.686+0000 7f59d587d640 1 -- 172.21.15.31:0/1546445821 >> 172.21.15.31:0/1546445821 conn(0x7f59d00758a0 msgr2=0x7f59d00fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 -- 172.21.15.31:0/1546445821 shutdown_connections 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 -- 172.21.15.31:0/1546445821 wait complete. 2024-09-16T08:53:54.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 Processor -- start 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 -- start start 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d0100cb0 0x7f59d019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d01015d0 0x7f59d019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d019c380 con 0x7f59d01015d0 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.687+0000 7f59d587d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d01959a0 con 0x7f59d0100cb0 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d01015d0 0x7f59d019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ce7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d01015d0 0x7f59d019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60398/0 (socket says 172.21.15.31:60398) 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ce7fc640 1 -- 172.21.15.31:0/3317068592 learned_addr learned my addr 172.21.15.31:0/3317068592 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ceffd640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d0100cb0 0x7f59d019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ce7fc640 1 -- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d0100cb0 msgr2=0x7f59d019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ce7fc640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d0100cb0 0x7f59d019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ce7fc640 1 -- 172.21.15.31:0/3317068592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c0009900 con 0x7f59d01015d0 2024-09-16T08:53:54.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ceffd640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d0100cb0 0x7f59d019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:53:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59ce7fc640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d01015d0 0x7f59d019bdf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c8f0 tx=0x7f59c400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59c4007b80 con 0x7f59d01015d0 2024-09-16T08:53:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d0195c80 con 0x7f59d01015d0 2024-09-16T08:53:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59c4007ce0 con 0x7f59d01015d0 2024-09-16T08:53:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59c4018830 con 0x7f59d01015d0 2024-09-16T08:53:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.688+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d01961d0 con 0x7f59d01015d0 2024-09-16T08:53:54.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.689+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59d010d000 con 0x7f59d01015d0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.690+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f59c4018aa0 con 0x7f59d01015d0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.690+0000 7f59d487b640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f59b806cc50 0x7f59b806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.690+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f59c408c010 con 0x7f59d01015d0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.690+0000 7f59ceffd640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f59b806cc50 0x7f59b806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.690+0000 7f59d487b640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] conn(0x7f59b80729e0 0x7f59b8074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.691+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 --> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59b80754b0 con 0x7f59b80729e0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.691+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c408e6e0 con 0x7f59d01015d0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.691+0000 7f59cf7fe640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] conn(0x7f59b80729e0 0x7f59b8074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.691+0000 7f59ceffd640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f59b806cc50 0x7f59b806f110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f59c0000f80 tx=0x7f59c003b040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.691+0000 7f59cf7fe640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] conn(0x7f59b80729e0 0x7f59b8074e00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.692+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== osd.3 v2:172.21.15.97:6800/797496523 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f59b80754b0 con 0x7f59b80729e0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.705+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 --> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f59d0196690 con 0x7f59b80729e0 2024-09-16T08:53:54.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.705+0000 7f59d487b640 1 -- 172.21.15.31:0/3317068592 <== osd.3 v2:172.21.15.97:6800/797496523 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f59d0196690 con 0x7f59b80729e0 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] conn(0x7f59b80729e0 msgr2=0x7f59b8074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] conn(0x7f59b80729e0 0x7f59b8074e00 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f59b806cc50 msgr2=0x7f59b806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f59b806cc50 0x7f59b806f110 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f59c0000f80 tx=0x7f59c003b040 comp rx=0 tx=0).stop 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d01015d0 msgr2=0x7f59d019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d01015d0 0x7f59d019bdf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f59c400c8f0 tx=0x7f59c400cdb0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 shutdown_connections 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:6800/797496523,v1:172.21.15.97:6801/797496523] conn(0x7f59b80729e0 0x7f59b8074e00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f59b806cc50 0x7f59b806f110 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59d01015d0 0x7f59d019bdf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 --2- 172.21.15.31:0/3317068592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59d0100cb0 0x7f59d019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 >> 172.21.15.31:0/3317068592 conn(0x7f59d00758a0 msgr2=0x7f59d0101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:54.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 shutdown_connections 2024-09-16T08:53:54.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:51.706+0000 7f59d587d640 1 -- 172.21.15.31:0/3317068592 wait complete. 2024-09-16T08:53:54.598 INFO:teuthology.orchestra.run.smithi031.stdout:115964117012 2024-09-16T08:53:54.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 -- 172.21.15.31:0/1370611097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb864104a30 msgr2=0x7fb864104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1370611097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb864104a30 0x7fb864104e10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb84c009880 tx=0x7fb84c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 -- 172.21.15.31:0/1370611097 shutdown_connections 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1370611097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8640fea30 0x7fb8640fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1370611097 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb864104a30 0x7fb864104e10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 -- 172.21.15.31:0/1370611097 >> 172.21.15.31:0/1370611097 conn(0x7fb8640fa730 msgr2=0x7fb8640fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 -- 172.21.15.31:0/1370611097 shutdown_connections 2024-09-16T08:53:54.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.786+0000 7fb86acaa640 1 -- 172.21.15.31:0/1370611097 wait complete. 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb86acaa640 1 Processor -- start 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb86acaa640 1 -- start start 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb86acaa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8640fea30 0x7fb864196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb86acaa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb864104a30 0x7fb864196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb86acaa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb864197380 con 0x7fb8640fea30 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb86acaa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8641974f0 con 0x7fb864104a30 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb868a46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8640fea30 0x7fb864196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb868a46640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8640fea30 0x7fb864196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60426/0 (socket says 172.21.15.31:60426) 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb868a46640 1 -- 172.21.15.31:0/1113576085 learned_addr learned my addr 172.21.15.31:0/1113576085 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb868a46640 1 -- 172.21.15.31:0/1113576085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb864104a30 msgr2=0x7fb864196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb868a46640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb864104a30 0x7fb864196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb868a46640 1 -- 172.21.15.31:0/1113576085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c009560 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.787+0000 7fb868a46640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8640fea30 0x7fb864196900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb84c02a9c0 tx=0x7fb84c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.788+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb84c004aa0 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.788+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb84c004c00 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.788+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb864197770 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.788+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb84c003ab0 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.788+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86419fc50 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.789+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb86410bfa0 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.789+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb84c031d20 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb861ffb640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb85006cc50 0x7fb85006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb84c0ad3d0 con 0x7fb8640fea30 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb863fff640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb85006cc50 0x7fb85006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb861ffb640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] conn(0x7fb8500729e0 0x7fb850074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 --> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb8500754b0 con 0x7fb8500729e0 2024-09-16T08:53:54.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb84c0ad830 con 0x7fb8640fea30 2024-09-16T08:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb869247640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] conn(0x7fb8500729e0 0x7fb850074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.790+0000 7fb863fff640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb85006cc50 0x7fb85006f110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb864068c90 tx=0x7fb854005c70 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.791+0000 7fb869247640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] conn(0x7fb8500729e0 0x7fb850074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:53:54.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.791+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== osd.2 v2:172.21.15.31:6818/1861226285 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fb8500754b0 con 0x7fb8500729e0 2024-09-16T08:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 --> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb864060b30 con 0x7fb8500729e0 2024-09-16T08:53:54.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb861ffb640 1 -- 172.21.15.31:0/1113576085 <== osd.2 v2:172.21.15.31:6818/1861226285 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb864060b30 con 0x7fb8500729e0 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] conn(0x7fb8500729e0 msgr2=0x7fb850074e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] conn(0x7fb8500729e0 0x7fb850074e00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb85006cc50 msgr2=0x7fb85006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb85006cc50 0x7fb85006f110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb864068c90 tx=0x7fb854005c70 comp rx=0 tx=0).stop 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8640fea30 msgr2=0x7fb864196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.806+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8640fea30 0x7fb864196900 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb84c02a9c0 tx=0x7fb84c0055a0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 shutdown_connections 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb85006cc50 0x7fb85006f110 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:6818/1861226285,v1:172.21.15.31:6819/1861226285] conn(0x7fb8500729e0 0x7fb850074e00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb864104a30 0x7fb864196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 --2- 172.21.15.31:0/1113576085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8640fea30 0x7fb864196900 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:53:54.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 >> 172.21.15.31:0/1113576085 conn(0x7fb8640fa730 msgr2=0x7fb8640fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:53:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 shutdown_connections 2024-09-16T08:53:54.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:53:54.807+0000 7fb86acaa640 1 -- 172.21.15.31:0/1113576085 wait complete. 2024-09-16T08:53:55.090 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:55.097 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:55.098 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:53:55.445 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:55 smithi031 ceph-mon[27844]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:55.455 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:55 smithi097 ceph-mon[35478]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:57.380 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:57 smithi031 ceph-mon[27844]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:57.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:57 smithi097 ceph-mon[35478]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:57.827 INFO:teuthology.orchestra.run.smithi031.stdout:81604378649 2024-09-16T08:53:57.827 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T08:53:59.385 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:53:59 smithi031 ceph-mon[27844]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:53:59.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:53:59 smithi097 ceph-mon[35478]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:00.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 -- 172.21.15.31:0/630033088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 msgr2=0x7f44d0109a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 --2- 172.21.15.31:0/630033088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d0109a50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009880 tx=0x7f44b802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 -- 172.21.15.31:0/630033088 shutdown_connections 2024-09-16T08:54:00.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 --2- 172.21.15.31:0/630033088 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d0109a50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 --2- 172.21.15.31:0/630033088 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d0100c40 0x7f44d0101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 -- 172.21.15.31:0/630033088 >> 172.21.15.31:0/630033088 conn(0x7f44d0075830 msgr2=0x7f44d00fcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 -- 172.21.15.31:0/630033088 shutdown_connections 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.048+0000 7f44d6680640 1 -- 172.21.15.31:0/630033088 wait complete. 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44d6680640 1 Processor -- start 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44d6680640 1 -- start start 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44d6680640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d0100c40 0x7f44d00fe8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44d6680640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d00fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44d6680640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00ff320 con 0x7f44d0101560 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44d6680640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00ff490 con 0x7f44d0100c40 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d00fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d00fede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60436/0 (socket says 172.21.15.31:60436) 2024-09-16T08:54:00.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44c7fff640 1 -- 172.21.15.31:0/2610926845 learned_addr learned my addr 172.21.15.31:0/2610926845 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44cffff640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d0100c40 0x7f44d00fe8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44c7fff640 1 -- 172.21.15.31:0/2610926845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d0100c40 msgr2=0x7f44d00fe8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44c7fff640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d0100c40 0x7f44d00fe8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.049+0000 7f44c7fff640 1 -- 172.21.15.31:0/2610926845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8009560 con 0x7f44d0101560 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.050+0000 7f44c7fff640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d00fede0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f44b8032bb0 tx=0x7f44b8032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.050+0000 7f44cdffb640 1 -- 172.21.15.31:0/2610926845 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44b80037c0 con 0x7f44d0101560 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.050+0000 7f44cdffb640 1 -- 172.21.15.31:0/2610926845 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44b8003de0 con 0x7f44d0101560 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.050+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d00ff710 con 0x7f44d0101560 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.050+0000 7f44cdffb640 1 -- 172.21.15.31:0/2610926845 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44b803be80 con 0x7f44d0101560 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.050+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d01a1f10 con 0x7f44d0101560 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.051+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d00684a0 con 0x7f44d0101560 2024-09-16T08:54:00.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.051+0000 7f44cdffb640 1 -- 172.21.15.31:0/2610926845 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f44b803b6f0 con 0x7f44d0101560 2024-09-16T08:54:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.052+0000 7f44cdffb640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f44bc06cc60 0x7f44bc06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.052+0000 7f44cdffb640 1 -- 172.21.15.31:0/2610926845 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f44b80aca40 con 0x7f44d0101560 2024-09-16T08:54:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.052+0000 7f44cffff640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f44bc06cc60 0x7f44bc06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.052+0000 7f44cffff640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f44bc06cc60 0x7f44bc06f120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44c0005fd0 tx=0x7f44c0005850 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:00.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.056+0000 7f44cdffb640 1 -- 172.21.15.31:0/2610926845 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f44b80771d0 con 0x7f44d0101560 2024-09-16T08:54:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.236+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f44d0101020 con 0x7f44d0101560 2024-09-16T08:54:00.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.237+0000 7f44cdffb640 1 -- 172.21.15.31:0/2610926845 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44d0101020 con 0x7f44d0101560 2024-09-16T08:54:00.238 INFO:teuthology.orchestra.run.smithi031.stdout:171798691851 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f44bc06cc60 msgr2=0x7f44bc06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f44bc06cc60 0x7f44bc06f120 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44c0005fd0 tx=0x7f44c0005850 comp rx=0 tx=0).stop 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 msgr2=0x7f44d00fede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d00fede0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f44b8032bb0 tx=0x7f44b8032be0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 shutdown_connections 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f44bc06cc60 0x7f44bc06f120 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44d0101560 0x7f44d00fede0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 --2- 172.21.15.31:0/2610926845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d0100c40 0x7f44d00fe8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.239+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 >> 172.21.15.31:0/2610926845 conn(0x7f44d0075830 msgr2=0x7f44d0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.240+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 shutdown_connections 2024-09-16T08:54:00.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.240+0000 7f44d6680640 1 -- 172.21.15.31:0/2610926845 wait complete. 2024-09-16T08:54:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.752+0000 7f038a663640 1 -- 172.21.15.31:0/3686294061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384104a30 msgr2=0x7f0384104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.752+0000 7f038a663640 1 --2- 172.21.15.31:0/3686294061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384104a30 0x7f0384104e10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0374009880 tx=0x7f037402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 -- 172.21.15.31:0/3686294061 shutdown_connections 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 --2- 172.21.15.31:0/3686294061 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03840fea30 0x7f03840fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 --2- 172.21.15.31:0/3686294061 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0384104a30 0x7f0384104e10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 -- 172.21.15.31:0/3686294061 >> 172.21.15.31:0/3686294061 conn(0x7f03840fa730 msgr2=0x7f03840fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 -- 172.21.15.31:0/3686294061 shutdown_connections 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 -- 172.21.15.31:0/3686294061 wait complete. 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 Processor -- start 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.753+0000 7f038a663640 1 -- start start 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f038a663640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03840fea30 0x7f0384197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f038a663640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0384104a30 0x7f03841982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f038a663640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384198960 con 0x7f03840fea30 2024-09-16T08:54:00.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f038a663640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038419c6d0 con 0x7f0384104a30 2024-09-16T08:54:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f0383fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03840fea30 0x7f0384197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f0383fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03840fea30 0x7f0384197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60480/0 (socket says 172.21.15.31:60480) 2024-09-16T08:54:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f0383fff640 1 -- 172.21.15.31:0/3484721497 learned_addr learned my addr 172.21.15.31:0/3484721497 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f03837fe640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0384104a30 0x7f03841982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f0383fff640 1 -- 172.21.15.31:0/3484721497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0384104a30 msgr2=0x7f03841982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f0383fff640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0384104a30 0x7f03841982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f0383fff640 1 -- 172.21.15.31:0/3484721497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0374009560 con 0x7f03840fea30 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.754+0000 7f0383fff640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03840fea30 0x7f0384197d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f037402a9c0 tx=0x7f03740055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.755+0000 7f03817fa640 1 -- 172.21.15.31:0/3484721497 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0374004aa0 con 0x7f03840fea30 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.755+0000 7f03817fa640 1 -- 172.21.15.31:0/3484721497 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0374004c00 con 0x7f03840fea30 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.755+0000 7f03817fa640 1 -- 172.21.15.31:0/3484721497 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0374003a20 con 0x7f03840fea30 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.755+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038419c950 con 0x7f03840fea30 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.755+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f038419ce40 con 0x7f03840fea30 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.756+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0384100170 con 0x7f03840fea30 2024-09-16T08:54:00.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.757+0000 7f03817fa640 1 -- 172.21.15.31:0/3484721497 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0374031c90 con 0x7f03840fea30 2024-09-16T08:54:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.757+0000 7f03817fa640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f036c06ccb0 0x7f036c06f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.758+0000 7f03837fe640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f036c06ccb0 0x7f036c06f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.758+0000 7f03817fa640 1 -- 172.21.15.31:0/3484721497 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f03740ace50 con 0x7f03840fea30 2024-09-16T08:54:00.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.758+0000 7f03837fe640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f036c06ccb0 0x7f036c06f170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0384199340 tx=0x7f0378009110 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:00.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.762+0000 7f03817fa640 1 -- 172.21.15.31:0/3484721497 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0374077680 con 0x7f03840fea30 2024-09-16T08:54:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.939+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03840fef10 con 0x7f03840fea30 2024-09-16T08:54:00.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.939+0000 7f03817fa640 1 -- 172.21.15.31:0/3484721497 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f037407aca0 con 0x7f03840fea30 2024-09-16T08:54:00.940 INFO:teuthology.orchestra.run.smithi031.stdout:60129542176 2024-09-16T08:54:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f036c06ccb0 msgr2=0x7f036c06f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f036c06ccb0 0x7f036c06f170 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0384199340 tx=0x7f0378009110 comp rx=0 tx=0).stop 2024-09-16T08:54:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03840fea30 msgr2=0x7f0384197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03840fea30 0x7f0384197d90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f037402a9c0 tx=0x7f03740055a0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 shutdown_connections 2024-09-16T08:54:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f036c06ccb0 0x7f036c06f170 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0384104a30 0x7f03841982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 --2- 172.21.15.31:0/3484721497 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03840fea30 0x7f0384197d90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 >> 172.21.15.31:0/3484721497 conn(0x7f03840fa730 msgr2=0x7f0384068ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 shutdown_connections 2024-09-16T08:54:00.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.941+0000 7f038a663640 1 -- 172.21.15.31:0/3484721497 wait complete. 2024-09-16T08:54:01.358 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:01 smithi031 ceph-mon[27844]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:01.358 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:01 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2610926845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T08:54:01.359 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:01 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1887166502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T08:54:01.359 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:01 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3484721497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T08:54:01.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:01 smithi097 ceph-mon[35478]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:01 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2610926845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T08:54:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:01 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1887166502' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T08:54:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:01 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3484721497' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T08:54:01.613 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:02.002 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542174 got 60129542176 for osd.1 2024-09-16T08:54:02.003 DEBUG:teuthology.parallel:result is None 2024-09-16T08:54:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/4014866484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 msgr2=0x7f5e380fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/4014866484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e380fee50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009880 tx=0x7f5e2c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/4014866484 shutdown_connections 2024-09-16T08:54:02.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/4014866484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e380fee50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/4014866484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e381049f0 0x7f5e38104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/4014866484 >> 172.21.15.31:0/4014866484 conn(0x7f5e380fa730 msgr2=0x7f5e380fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/4014866484 shutdown_connections 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.269+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/4014866484 wait complete. 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3f9a5640 1 Processor -- start 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3f9a5640 1 -- start start 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3f9a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e38072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3f9a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e381049f0 0x7f5e380fb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3f9a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380fd020 con 0x7f5e380fe9f0 2024-09-16T08:54:02.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3f9a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380fb650 con 0x7f5e381049f0 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3d741640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e38072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3d741640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e38072060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60452/0 (socket says 172.21.15.31:60452) 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3d741640 1 -- 172.21.15.31:0/1887166502 learned_addr learned my addr 172.21.15.31:0/1887166502 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3cf40640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e381049f0 0x7f5e380fb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3d741640 1 -- 172.21.15.31:0/1887166502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e381049f0 msgr2=0x7f5e380fb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3d741640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e381049f0 0x7f5e380fb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.270+0000 7f5e3d741640 1 -- 172.21.15.31:0/1887166502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009560 con 0x7f5e380fe9f0 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.271+0000 7f5e3cf40640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e381049f0 0x7f5e380fb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.271+0000 7f5e3d741640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e38072060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400cc60 tx=0x7f5e3400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:02.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.271+0000 7f5e2a7fc640 1 -- 172.21.15.31:0/1887166502 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3400f450 con 0x7f5e380fe9f0 2024-09-16T08:54:02.099 INFO:teuthology.orchestra.run.smithi031.stdout:150323855377 2024-09-16T08:54:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.271+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e380fb930 con 0x7f5e380fe9f0 2024-09-16T08:54:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.271+0000 7f5e2a7fc640 1 -- 172.21.15.31:0/1887166502 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e3400fa70 con 0x7f5e380fe9f0 2024-09-16T08:54:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.271+0000 7f5e2a7fc640 1 -- 172.21.15.31:0/1887166502 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e34018b50 con 0x7f5e380fe9f0 2024-09-16T08:54:02.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.271+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e380fbe80 con 0x7f5e380fe9f0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.272+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38100130 con 0x7f5e380fe9f0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.273+0000 7f5e2a7fc640 1 -- 172.21.15.31:0/1887166502 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5e3400fbe0 con 0x7f5e380fe9f0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.273+0000 7f5e2a7fc640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e2406cc60 0x7f5e2406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.273+0000 7f5e2a7fc640 1 -- 172.21.15.31:0/1887166502 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5e3408c6b0 con 0x7f5e380fe9f0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.273+0000 7f5e3cf40640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e2406cc60 0x7f5e2406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.274+0000 7f5e3cf40640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e2406cc60 0x7f5e2406f120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c000c00 tx=0x7f5e2c004280 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.277+0000 7f5e2a7fc640 1 -- 172.21.15.31:0/1887166502 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5e34056dc0 con 0x7f5e380fe9f0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.449+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5e380fc640 con 0x7f5e380fe9f0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.450+0000 7f5e2a7fc640 1 -- 172.21.15.31:0/1887166502 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e3405a3e0 con 0x7f5e380fe9f0 2024-09-16T08:54:02.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e2406cc60 msgr2=0x7f5e2406f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e2406cc60 0x7f5e2406f120 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c000c00 tx=0x7f5e2c004280 comp rx=0 tx=0).stop 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 msgr2=0x7f5e38072060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e38072060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400cc60 tx=0x7f5e3400a620 comp rx=0 tx=0).stop 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 shutdown_connections 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e2406cc60 0x7f5e2406f120 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e381049f0 0x7f5e380fb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 --2- 172.21.15.31:0/1887166502 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e380fe9f0 0x7f5e38072060 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 >> 172.21.15.31:0/1887166502 conn(0x7f5e380fa730 msgr2=0x7f5e38068c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 shutdown_connections 2024-09-16T08:54:02.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:00.452+0000 7f5e3f9a5640 1 -- 172.21.15.31:0/1887166502 wait complete. 2024-09-16T08:54:02.708 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855375 got 150323855377 for osd.4 2024-09-16T08:54:02.709 DEBUG:teuthology.parallel:result is None 2024-09-16T08:54:03.087 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:03 smithi031 ceph-mon[27844]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:03.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:03 smithi097 ceph-mon[35478]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:03.559 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691851 for osd.5 2024-09-16T08:54:03.560 DEBUG:teuthology.parallel:result is None 2024-09-16T08:54:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 -- 172.21.15.31:0/3847350162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 msgr2=0x7fa3741047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 --2- 172.21.15.31:0/3847350162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa3741047a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009880 tx=0x7fa35c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 -- 172.21.15.31:0/3847350162 shutdown_connections 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 --2- 172.21.15.31:0/3847350162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa3741047a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 --2- 172.21.15.31:0/3847350162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3740ffda0 0x7fa374100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 -- 172.21.15.31:0/3847350162 >> 172.21.15.31:0/3847350162 conn(0x7fa3740fb840 msgr2=0x7fa3740fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 -- 172.21.15.31:0/3847350162 shutdown_connections 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.742+0000 7fa379afe640 1 -- 172.21.15.31:0/3847350162 wait complete. 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa379afe640 1 Processor -- start 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa379afe640 1 -- start start 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa379afe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3740ffda0 0x7fa37419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa379afe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa37419be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa379afe640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37419c480 con 0x7fa374100750 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa379afe640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374195a10 con 0x7fa3740ffda0 2024-09-16T08:54:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa372ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa37419be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa372ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa37419be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50556/0 (socket says 172.21.15.31:50556) 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa372ffd640 1 -- 172.21.15.31:0/904183932 learned_addr learned my addr 172.21.15.31:0/904183932 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.743+0000 7fa3737fe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3740ffda0 0x7fa37419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa372ffd640 1 -- 172.21.15.31:0/904183932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3740ffda0 msgr2=0x7fa37419b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa372ffd640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3740ffda0 0x7fa37419b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa372ffd640 1 -- 172.21.15.31:0/904183932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c009560 con 0x7fa374100750 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa372ffd640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa37419be60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa35c02a9c0 tx=0x7fa35c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa370ff9640 1 -- 172.21.15.31:0/904183932 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35c004aa0 con 0x7fa374100750 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa370ff9640 1 -- 172.21.15.31:0/904183932 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35c004c00 con 0x7fa374100750 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa374195c90 con 0x7fa374100750 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa370ff9640 1 -- 172.21.15.31:0/904183932 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35c003ab0 con 0x7fa374100750 2024-09-16T08:54:03.774 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.744+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa374196180 con 0x7fa374100750 2024-09-16T08:54:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.745+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa374101280 con 0x7fa374100750 2024-09-16T08:54:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.745+0000 7fa370ff9640 1 -- 172.21.15.31:0/904183932 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fa35c003ce0 con 0x7fa374100750 2024-09-16T08:54:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.746+0000 7fa370ff9640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fa36006cc50 0x7fa36006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.746+0000 7fa370ff9640 1 -- 172.21.15.31:0/904183932 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa35c0ad020 con 0x7fa374100750 2024-09-16T08:54:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.746+0000 7fa3737fe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fa36006cc50 0x7fa36006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.746+0000 7fa3737fe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fa36006cc50 0x7fa36006f110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa364009890 tx=0x7fa364008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:03.775 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.750+0000 7fa370ff9640 1 -- 172.21.15.31:0/904183932 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fa35c077730 con 0x7fa374100750 2024-09-16T08:54:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.965+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa3741080b0 con 0x7fa374100750 2024-09-16T08:54:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.965+0000 7fa370ff9640 1 -- 172.21.15.31:0/904183932 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa35c07ad50 con 0x7fa374100750 2024-09-16T08:54:03.966 INFO:teuthology.orchestra.run.smithi031.stdout:38654705700 2024-09-16T08:54:03.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fa36006cc50 msgr2=0x7fa36006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fa36006cc50 0x7fa36006f110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa364009890 tx=0x7fa364008040 comp rx=0 tx=0).stop 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 msgr2=0x7fa37419be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa37419be60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa35c02a9c0 tx=0x7fa35c0055a0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 shutdown_connections 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fa36006cc50 0x7fa36006f110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa374100750 0x7fa37419be60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 --2- 172.21.15.31:0/904183932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa3740ffda0 0x7fa37419b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 >> 172.21.15.31:0/904183932 conn(0x7fa3740fb840 msgr2=0x7fa3740fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 shutdown_connections 2024-09-16T08:54:03.969 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:03.968+0000 7fa379afe640 1 -- 172.21.15.31:0/904183932 wait complete. 2024-09-16T08:54:04.399 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:04 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/904183932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T08:54:04.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:04 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/904183932' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T08:54:05.341 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:05 smithi031 ceph-mon[27844]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:05.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:05 smithi097 ceph-mon[35478]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:06.105 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705697 got 38654705700 for osd.0 2024-09-16T08:54:06.106 DEBUG:teuthology.parallel:result is None 2024-09-16T08:54:06.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:06 smithi031 ceph-mon[27844]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:06.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:06 smithi097 ceph-mon[35478]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:07.223 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 -- 172.21.15.31:0/2935987470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 msgr2=0x7f3a10100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 --2- 172.21.15.31:0/2935987470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a10100180 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009880 tx=0x7f3a0002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 -- 172.21.15.31:0/2935987470 shutdown_connections 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 --2- 172.21.15.31:0/2935987470 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10100750 0x7f3a101047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 --2- 172.21.15.31:0/2935987470 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a10100180 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 -- 172.21.15.31:0/2935987470 >> 172.21.15.31:0/2935987470 conn(0x7f3a100fb840 msgr2=0x7f3a100fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 -- 172.21.15.31:0/2935987470 shutdown_connections 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.205+0000 7f3a15972640 1 -- 172.21.15.31:0/2935987470 wait complete. 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a15972640 1 Processor -- start 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a15972640 1 -- start start 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a15972640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a1019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:07.224 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a15972640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10100750 0x7f3a1019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a15972640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10197b40 con 0x7f3a100ffda0 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a15972640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10197cb0 con 0x7f3a10100750 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a0effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a1019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a0effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a1019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50574/0 (socket says 172.21.15.31:50574) 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.206+0000 7f3a0effd640 1 -- 172.21.15.31:0/4167606917 learned_addr learned my addr 172.21.15.31:0/4167606917 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a0e7fc640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10100750 0x7f3a1019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a0effd640 1 -- 172.21.15.31:0/4167606917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10100750 msgr2=0x7f3a1019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a0effd640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10100750 0x7f3a1019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a0effd640 1 -- 172.21.15.31:0/4167606917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a00009560 con 0x7f3a100ffda0 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a0e7fc640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10100750 0x7f3a1019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a0effd640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a1019da50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a0002a9c0 tx=0x7f3a000055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a14970640 1 -- 172.21.15.31:0/4167606917 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a00004aa0 con 0x7f3a100ffda0 2024-09-16T08:54:07.225 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a14970640 1 -- 172.21.15.31:0/4167606917 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a00004c00 con 0x7f3a100ffda0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a10197f30 con 0x7f3a100ffda0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a14970640 1 -- 172.21.15.31:0/4167606917 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a00003ab0 con 0x7f3a100ffda0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.207+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a10198420 con 0x7f3a100ffda0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.208+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a10101280 con 0x7f3a100ffda0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.208+0000 7f3a14970640 1 -- 172.21.15.31:0/4167606917 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3a00031d20 con 0x7f3a100ffda0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.209+0000 7f3a14970640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f39f806cc50 0x7f39f806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.209+0000 7f3a14970640 1 -- 172.21.15.31:0/4167606917 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3a000ad020 con 0x7f3a100ffda0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.209+0000 7f3a0e7fc640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f39f806cc50 0x7f39f806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.209+0000 7f3a0e7fc640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f39f806cc50 0x7f39f806f110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a10072700 tx=0x7f3a04005d30 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:07.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.213+0000 7f3a14970640 1 -- 172.21.15.31:0/4167606917 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3a00077730 con 0x7f3a100ffda0 2024-09-16T08:54:07.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.387+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a10198fe0 con 0x7f3a100ffda0 2024-09-16T08:54:07.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.388+0000 7f3a14970640 1 -- 172.21.15.31:0/4167606917 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a0007ad50 con 0x7f3a100ffda0 2024-09-16T08:54:07.389 INFO:teuthology.orchestra.run.smithi031.stdout:115964117015 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f39f806cc50 msgr2=0x7f39f806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f39f806cc50 0x7f39f806f110 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3a10072700 tx=0x7f3a04005d30 comp rx=0 tx=0).stop 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 msgr2=0x7f3a1019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a1019da50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a0002a9c0 tx=0x7f3a000055a0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 shutdown_connections 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f39f806cc50 0x7f39f806f110 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a10100750 0x7f3a1019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 --2- 172.21.15.31:0/4167606917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a100ffda0 0x7f3a1019da50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 >> 172.21.15.31:0/4167606917 conn(0x7f3a100fb840 msgr2=0x7f3a100fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 shutdown_connections 2024-09-16T08:54:07.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.390+0000 7f3a15972640 1 -- 172.21.15.31:0/4167606917 wait complete. 2024-09-16T08:54:07.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.451+0000 7faf53921640 1 -- 172.21.15.31:0/3613687901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c0fe9f0 msgr2=0x7faf4c0fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.451+0000 7faf53921640 1 --2- 172.21.15.31:0/3613687901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c0fe9f0 0x7faf4c0fee50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009880 tx=0x7faf3c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.451+0000 7faf53921640 1 -- 172.21.15.31:0/3613687901 shutdown_connections 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.451+0000 7faf53921640 1 --2- 172.21.15.31:0/3613687901 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c0fe9f0 0x7faf4c0fee50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.451+0000 7faf53921640 1 --2- 172.21.15.31:0/3613687901 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c1049f0 0x7faf4c104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.451+0000 7faf53921640 1 -- 172.21.15.31:0/3613687901 >> 172.21.15.31:0/3613687901 conn(0x7faf4c0758a0 msgr2=0x7faf4c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 -- 172.21.15.31:0/3613687901 shutdown_connections 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 -- 172.21.15.31:0/3613687901 wait complete. 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 Processor -- start 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 -- start start 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0fe9f0 0x7faf4c196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1049f0 0x7faf4c196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c197490 con 0x7faf4c1049f0 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.452+0000 7faf53921640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c197600 con 0x7faf4c0fe9f0 2024-09-16T08:54:07.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf50ebc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1049f0 0x7faf4c196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf50ebc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1049f0 0x7faf4c196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50592/0 (socket says 172.21.15.31:50592) 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf50ebc640 1 -- 172.21.15.31:0/3656210651 learned_addr learned my addr 172.21.15.31:0/3656210651 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf516bd640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0fe9f0 0x7faf4c196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf50ebc640 1 -- 172.21.15.31:0/3656210651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0fe9f0 msgr2=0x7faf4c196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf50ebc640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0fe9f0 0x7faf4c196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf50ebc640 1 -- 172.21.15.31:0/3656210651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c009560 con 0x7faf4c1049f0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf516bd640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0fe9f0 0x7faf4c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf50ebc640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1049f0 0x7faf4c196ec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faf3c02a9c0 tx=0x7faf3c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf427fc640 1 -- 172.21.15.31:0/3656210651 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c004aa0 con 0x7faf4c1049f0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf427fc640 1 -- 172.21.15.31:0/3656210651 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf3c004c00 con 0x7faf4c1049f0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c19f9f0 con 0x7faf4c1049f0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf427fc640 1 -- 172.21.15.31:0/3656210651 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c003ab0 con 0x7faf4c1049f0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.453+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c19fd50 con 0x7faf4c1049f0 2024-09-16T08:54:07.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.454+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c100130 con 0x7faf4c1049f0 2024-09-16T08:54:07.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.455+0000 7faf427fc640 1 -- 172.21.15.31:0/3656210651 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7faf3c031d20 con 0x7faf4c1049f0 2024-09-16T08:54:07.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.455+0000 7faf427fc640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faf3806cc50 0x7faf3806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:07.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.455+0000 7faf427fc640 1 -- 172.21.15.31:0/3656210651 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7faf3c0acf20 con 0x7faf4c1049f0 2024-09-16T08:54:07.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.455+0000 7faf516bd640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faf3806cc50 0x7faf3806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:07.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.456+0000 7faf516bd640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faf3806cc50 0x7faf3806f110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf48005fd0 tx=0x7faf48005ea0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:07.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.459+0000 7faf427fc640 1 -- 172.21.15.31:0/3656210651 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7faf3c0e0e20 con 0x7faf4c1049f0 2024-09-16T08:54:07.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:07 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/4167606917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T08:54:07.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.634+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faf4c10bc80 con 0x7faf4c1049f0 2024-09-16T08:54:07.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.634+0000 7faf427fc640 1 -- 172.21.15.31:0/3656210651 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf3c07b140 con 0x7faf4c1049f0 2024-09-16T08:54:07.635 INFO:teuthology.orchestra.run.smithi031.stdout:81604378652 2024-09-16T08:54:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faf3806cc50 msgr2=0x7faf3806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faf3806cc50 0x7faf3806f110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faf48005fd0 tx=0x7faf48005ea0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1049f0 msgr2=0x7faf4c196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1049f0 0x7faf4c196ec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faf3c02a9c0 tx=0x7faf3c0055a0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 shutdown_connections 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7faf3806cc50 0x7faf3806f110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faf4c1049f0 0x7faf4c196ec0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 --2- 172.21.15.31:0/3656210651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf4c0fe9f0 0x7faf4c196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 >> 172.21.15.31:0/3656210651 conn(0x7faf4c0758a0 msgr2=0x7faf4c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 shutdown_connections 2024-09-16T08:54:07.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:07.636+0000 7faf53921640 1 -- 172.21.15.31:0/3656210651 wait complete. 2024-09-16T08:54:07.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:07 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/4167606917' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T08:54:08.128 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117012 got 115964117015 for osd.3 2024-09-16T08:54:08.128 DEBUG:teuthology.parallel:result is None 2024-09-16T08:54:08.752 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:08 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3656210651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T08:54:08.752 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:08 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:08.752 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:08 smithi031 ceph-mon[27844]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:08.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:08 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3656210651' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T08:54:08.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:08 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:08.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:08 smithi097 ceph-mon[35478]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:08.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378652 for osd.2 2024-09-16T08:54:08.776 DEBUG:teuthology.parallel:result is None 2024-09-16T08:54:08.777 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T08:54:08.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T08:54:08.990 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 -- 172.21.15.31:0/526184747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 msgr2=0x7f00c80fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 --2- 172.21.15.31:0/526184747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80fff90 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009880 tx=0x7f00b002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 -- 172.21.15.31:0/526184747 shutdown_connections 2024-09-16T08:54:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 --2- 172.21.15.31:0/526184747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80fff90 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 --2- 172.21.15.31:0/526184747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c8105be0 0x7f00c8105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 -- 172.21.15.31:0/526184747 >> 172.21.15.31:0/526184747 conn(0x7f00c80fb810 msgr2=0x7f00c80fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 -- 172.21.15.31:0/526184747 shutdown_connections 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.580+0000 7f00cd655640 1 -- 172.21.15.31:0/526184747 wait complete. 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00cd655640 1 Processor -- start 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00cd655640 1 -- start start 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00cd655640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00cd655640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c8105be0 0x7f00c8072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00cd655640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c8073230 con 0x7f00c80ffb30 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00cd655640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c80733a0 con 0x7f00c8105be0 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00c6ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50598/0 (socket says 172.21.15.31:50598) 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00c6ffd640 1 -- 172.21.15.31:0/1149852350 learned_addr learned my addr 172.21.15.31:0/1149852350 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.581+0000 7f00c67fc640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c8105be0 0x7f00c8072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:10.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00c6ffd640 1 -- 172.21.15.31:0/1149852350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c8105be0 msgr2=0x7f00c8072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00c6ffd640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c8105be0 0x7f00c8072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00c6ffd640 1 -- 172.21.15.31:0/1149852350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b0009560 con 0x7f00c80ffb30 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00c6ffd640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80727b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f00b800e970 tx=0x7f00b800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00bffff640 1 -- 172.21.15.31:0/1149852350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00b800cc90 con 0x7f00c80ffb30 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c81a1c00 con 0x7f00c80ffb30 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00bffff640 1 -- 172.21.15.31:0/1149852350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00b800cdf0 con 0x7f00c80ffb30 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00bffff640 1 -- 172.21.15.31:0/1149852350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00b8018810 con 0x7f00c80ffb30 2024-09-16T08:54:10.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.582+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c81a1fe0 con 0x7f00c80ffb30 2024-09-16T08:54:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.583+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c81011e0 con 0x7f00c80ffb30 2024-09-16T08:54:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.584+0000 7f00bffff640 1 -- 172.21.15.31:0/1149852350 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f00b8018a80 con 0x7f00c80ffb30 2024-09-16T08:54:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.584+0000 7f00bffff640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f00b406cc50 0x7f00b406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.584+0000 7f00bffff640 1 -- 172.21.15.31:0/1149852350 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f00b8014070 con 0x7f00c80ffb30 2024-09-16T08:54:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.584+0000 7f00c67fc640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f00b406cc50 0x7f00b406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:10.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.585+0000 7f00c67fc640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f00b406cc50 0x7f00b406f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009230 tx=0x7f00b00058d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:10.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.588+0000 7f00bffff640 1 -- 172.21.15.31:0/1149852350 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f00b8052e20 con 0x7f00c80ffb30 2024-09-16T08:54:10.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.757+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00c8073df0 con 0x7f00b406cc50 2024-09-16T08:54:10.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.759+0000 7f00bffff640 1 -- 172.21.15.31:0/1149852350 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27871 (secure 0 0 0) 0x7f00c8073df0 con 0x7f00b406cc50 2024-09-16T08:54:10.760 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:10.760 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f00b406cc50 msgr2=0x7f00b406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f00b406cc50 0x7f00b406f110 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009230 tx=0x7f00b00058d0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 msgr2=0x7f00c80727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80727b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f00b800e970 tx=0x7f00b800ee30 comp rx=0 tx=0).stop 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 shutdown_connections 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f00b406cc50 0x7f00b406f110 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00c8105be0 0x7f00c8072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 --2- 172.21.15.31:0/1149852350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00c80ffb30 0x7f00c80727b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 >> 172.21.15.31:0/1149852350 conn(0x7f00c80fb810 msgr2=0x7f00c80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 shutdown_connections 2024-09-16T08:54:10.762 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:10.761+0000 7f00cd655640 1 -- 172.21.15.31:0/1149852350 wait complete. 2024-09-16T08:54:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:11 smithi031 ceph-mon[27844]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:11.398 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-16T08:54:10.056736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":174256,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":562296656,"statfs":{"total":575970213888,"available":575791775744,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002085"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-16T08:53:16.142714+0000","last_change":"2024-09-16T08:52:52.176837+0000","last_active":"2024-09-16T08:53:16.142714+0000","last_peered":"2024-09-16T08:53:16.142714+0000","last_clean":"2024-09-16T08:53:16.142714+0000","last_became_active":"2024-09-16T08:52:52.176589+0000","last_became_peered":"2024-09-16T08:52:52.176589+0000","last_unstale":"2024-09-16T08:53:16.142714+0000","last_undegraded":"2024-09-16T08:53:16.142714+0000","last_fullsized":"2024-09-16T08:53:16.142714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:52:06.568879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:52:06.568879+0000","last_clean_scrub_stamp":"2024-09-16T08:52:06.568879+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:45:13.429558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63200000000000001}]},{"osd":1,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.38},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Sep 16 08:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Sep 16 08:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.499}]},{"osd":4,"last update":"Mon Sep 16 08:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27388,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717764,"statfs":{"total":95995035648,"available":95966990336,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:53:06 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.26900000000000002}]},{"osd":1,"last update":"Mon Sep 16 08:53:06 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.247},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.28499999999999998}]},{"osd":3,"last update":"Mon Sep 16 08:53:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.41999999999999998}]},{"osd":4,"last update":"Mon Sep 16 08:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"min":{"1min":0.20599999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.27100000000000002}]},{"osd":2,"last update":"Mon Sep 16 08:54:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51700000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.48899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.309}]},{"osd":3,"last update":"Mon Sep 16 08:53:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Sep 16 08:53:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 08:53:38 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.32700000000000001},"min":{"1min":0.23799999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35299999999999998,"15min":0.35199999999999998},"min":{"1min":0.24199999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.317}]},{"osd":2,"last update":"Mon Sep 16 08:53:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.27500000000000002}]},{"osd":3,"last update":"Mon Sep 16 08:53:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Sep 16 08:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.624},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T08:54:11.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T08:54:11.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:11 smithi097 ceph-mon[35478]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:11.605 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:12 smithi031 ceph-mon[27844]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:54:12.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:12 smithi097 ceph-mon[35478]: from='client.14490 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:54:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.350+0000 7f9db7762640 1 -- 172.21.15.31:0/1005127899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 msgr2=0x7f9db01047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.350+0000 7f9db7762640 1 --2- 172.21.15.31:0/1005127899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db01047a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9da4009880 tx=0x7f9da402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.350+0000 7f9db7762640 1 -- 172.21.15.31:0/1005127899 shutdown_connections 2024-09-16T08:54:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.350+0000 7f9db7762640 1 --2- 172.21.15.31:0/1005127899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db01047a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.350+0000 7f9db7762640 1 --2- 172.21.15.31:0/1005127899 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db00ffda0 0x7f9db0100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.350+0000 7f9db7762640 1 -- 172.21.15.31:0/1005127899 >> 172.21.15.31:0/1005127899 conn(0x7f9db00fb840 msgr2=0x7f9db00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.351+0000 7f9db7762640 1 -- 172.21.15.31:0/1005127899 shutdown_connections 2024-09-16T08:54:13.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.351+0000 7f9db7762640 1 -- 172.21.15.31:0/1005127899 wait complete. 2024-09-16T08:54:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.351+0000 7f9db7762640 1 Processor -- start 2024-09-16T08:54:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.351+0000 7f9db7762640 1 -- start start 2024-09-16T08:54:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.351+0000 7f9db7762640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db00ffda0 0x7f9db00727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.351+0000 7f9db7762640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db0072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:13.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.351+0000 7f9db7762640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0073230 con 0x7f9db0100750 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db7762640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00733a0 con 0x7f9db00ffda0 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db4cfd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db0072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db4cfd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db0072cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51960/0 (socket says 172.21.15.31:51960) 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db4cfd640 1 -- 172.21.15.31:0/3135113895 learned_addr learned my addr 172.21.15.31:0/3135113895 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db54fe640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db00ffda0 0x7f9db00727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db4cfd640 1 -- 172.21.15.31:0/3135113895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db00ffda0 msgr2=0x7f9db00727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db4cfd640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db00ffda0 0x7f9db00727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db4cfd640 1 -- 172.21.15.31:0/3135113895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4009560 con 0x7f9db0100750 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db4cfd640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db0072cf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9da402a9c0 tx=0x7f9da40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9dae7fc640 1 -- 172.21.15.31:0/3135113895 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9da4004aa0 con 0x7f9db0100750 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9dae7fc640 1 -- 172.21.15.31:0/3135113895 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9da4004c00 con 0x7f9db0100750 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db0073620 con 0x7f9db0100750 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.352+0000 7f9dae7fc640 1 -- 172.21.15.31:0/3135113895 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9da4003ab0 con 0x7f9db0100750 2024-09-16T08:54:13.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.353+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db01a2010 con 0x7f9db0100750 2024-09-16T08:54:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.354+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db0101280 con 0x7f9db0100750 2024-09-16T08:54:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.354+0000 7f9dae7fc640 1 -- 172.21.15.31:0/3135113895 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f9da4003ce0 con 0x7f9db0100750 2024-09-16T08:54:13.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.354+0000 7f9dae7fc640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9d9806cc50 0x7f9d9806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.354+0000 7f9dae7fc640 1 -- 172.21.15.31:0/3135113895 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9da40ace20 con 0x7f9db0100750 2024-09-16T08:54:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.354+0000 7f9db54fe640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9d9806cc50 0x7f9d9806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:13.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.355+0000 7f9db54fe640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9d9806cc50 0x7f9d9806f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9da000a400 tx=0x7f9da000d040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:13.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.359+0000 7f9dae7fc640 1 -- 172.21.15.31:0/3135113895 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9da40775e0 con 0x7f9db0100750 2024-09-16T08:54:13.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:13 smithi031 ceph-mon[27844]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:13.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:13 smithi097 ceph-mon[35478]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:13.530 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.529+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9db0073620 con 0x7f9d9806cc50 2024-09-16T08:54:13.531 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.531+0000 7f9dae7fc640 1 -- 172.21.15.31:0/3135113895 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27940 (secure 0 0 0) 0x7f9db0073620 con 0x7f9d9806cc50 2024-09-16T08:54:13.532 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9d9806cc50 msgr2=0x7f9d9806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9d9806cc50 0x7f9d9806f110 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9da000a400 tx=0x7f9da000d040 comp rx=0 tx=0).stop 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 msgr2=0x7f9db0072cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db0072cf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9da402a9c0 tx=0x7f9da40055a0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 shutdown_connections 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f9d9806cc50 0x7f9d9806f110 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9db0100750 0x7f9db0072cf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 --2- 172.21.15.31:0/3135113895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9db00ffda0 0x7f9db00727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 >> 172.21.15.31:0/3135113895 conn(0x7f9db00fb840 msgr2=0x7f9db00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 shutdown_connections 2024-09-16T08:54:13.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:13.533+0000 7f9db7762640 1 -- 172.21.15.31:0/3135113895 wait complete. 2024-09-16T08:54:14.159 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-16T08:54:12.057114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":174256,"kb_used_data":3924,"kb_used_omap":9,"kb_used_meta":170230,"kb_avail":562296656,"statfs":{"total":575970213888,"available":575791775744,"internally_reserved":0,"allocated":4018176,"data_stored":2627442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":174316198},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002186"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":186,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-16T08:53:16.142714+0000","last_change":"2024-09-16T08:52:52.176837+0000","last_active":"2024-09-16T08:53:16.142714+0000","last_peered":"2024-09-16T08:53:16.142714+0000","last_clean":"2024-09-16T08:53:16.142714+0000","last_became_active":"2024-09-16T08:52:52.176589+0000","last_became_peered":"2024-09-16T08:52:52.176589+0000","last_unstale":"2024-09-16T08:53:16.142714+0000","last_undegraded":"2024-09-16T08:53:16.142714+0000","last_fullsized":"2024-09-16T08:53:16.142714+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T08:52:06.568879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T08:52:06.568879+0000","last_clean_scrub_stamp":"2024-09-16T08:52:06.568879+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:45:13.429558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]}]},{"osd":4,"up_from":35,"seq":150323855379,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717768,"statfs":{"total":95995035648,"available":95966994432,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.63200000000000001}]},{"osd":1,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.55700000000000005}]},{"osd":3,"last update":"Mon Sep 16 08:53:55 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.38},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]}]},{"osd":3,"up_from":27,"seq":115964117016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32068,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713084,"statfs":{"total":95995035648,"available":95962198016,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.43099999999999999}]},{"osd":1,"last update":"Mon Sep 16 08:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.51000000000000001}]},{"osd":2,"last update":"Mon Sep 16 08:53:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.499}]},{"osd":4,"last update":"Mon Sep 16 08:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27388,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717764,"statfs":{"total":95995035648,"available":95966990336,"internally_reserved":0,"allocated":372736,"data_stored":142723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:54:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.36299999999999999,"15min":0.36899999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.45100000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46700000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.31900000000000001}]},{"osd":1,"last update":"Mon Sep 16 08:54:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.35599999999999998,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.39500000000000002,"15min":0.39900000000000002},"min":{"1min":0.27700000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48299999999999998,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.39600000000000002}]},{"osd":3,"last update":"Mon Sep 16 08:53:28 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.624}]},{"osd":4,"last update":"Mon Sep 16 08:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 08:53:39 2024","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"min":{"1min":0.20599999999999999,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.27100000000000002}]},{"osd":2,"last update":"Mon Sep 16 08:54:07 2024","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.375,"15min":0.38300000000000001},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51700000000000002,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.40000000000000002,"15min":0.40799999999999997},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.48899999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.309}]},{"osd":3,"last update":"Mon Sep 16 08:53:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Mon Sep 16 08:53:56 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.629},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.63800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":733091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 08:53:38 2024","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.33100000000000002,"15min":0.32700000000000001},"min":{"1min":0.23799999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.35299999999999998,"15min":0.35199999999999998},"min":{"1min":0.24199999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.317}]},{"osd":2,"last update":"Mon Sep 16 08:53:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.27500000000000002}]},{"osd":3,"last update":"Mon Sep 16 08:53:26 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.49399999999999999}]},{"osd":4,"last update":"Mon Sep 16 08:53:53 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.624},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T08:54:14.160 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T08:54:14.160 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T08:54:14.161 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T08:54:14.161 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T08:54:14.364 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:15 smithi031 ceph-mon[27844]: from='client.14494 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:54:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:15 smithi031 ceph-mon[27844]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:15.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:15 smithi097 ceph-mon[35478]: from='client.14494 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T08:54:15.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:15 smithi097 ceph-mon[35478]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:15.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 -- 172.21.15.31:0/2062609123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac1049f0 msgr2=0x7fc2ac104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:15.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 --2- 172.21.15.31:0/2062609123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac1049f0 0x7fc2ac104dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009880 tx=0x7fc2a802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:15.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 -- 172.21.15.31:0/2062609123 shutdown_connections 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 --2- 172.21.15.31:0/2062609123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2ac0fe9f0 0x7fc2ac0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 --2- 172.21.15.31:0/2062609123 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac1049f0 0x7fc2ac104dd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 -- 172.21.15.31:0/2062609123 >> 172.21.15.31:0/2062609123 conn(0x7fc2ac0758a0 msgr2=0x7fc2ac0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 -- 172.21.15.31:0/2062609123 shutdown_connections 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 -- 172.21.15.31:0/2062609123 wait complete. 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 Processor -- start 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.987+0000 7fc2b3354640 1 -- start start 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b3354640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac0fe9f0 0x7fc2ac196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b3354640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2ac1049f0 0x7fc2ac196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b3354640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac197490 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b3354640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac197600 con 0x7fc2ac1049f0 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b10f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac0fe9f0 0x7fc2ac196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b10f0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac0fe9f0 0x7fc2ac196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51984/0 (socket says 172.21.15.31:51984) 2024-09-16T08:54:15.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b10f0640 1 -- 172.21.15.31:0/935485285 learned_addr learned my addr 172.21.15.31:0/935485285 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b08ef640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2ac1049f0 0x7fc2ac196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b10f0640 1 -- 172.21.15.31:0/935485285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2ac1049f0 msgr2=0x7fc2ac196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b10f0640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2ac1049f0 0x7fc2ac196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b10f0640 1 -- 172.21.15.31:0/935485285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a8009560 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b08ef640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2ac1049f0 0x7fc2ac196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b10f0640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac0fe9f0 0x7fc2ac196980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802a9c0 tx=0x7fc2a80055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2a27fc640 1 -- 172.21.15.31:0/935485285 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a8004aa0 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac19f9f0 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2a27fc640 1 -- 172.21.15.31:0/935485285 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2a8004c00 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.988+0000 7fc2a27fc640 1 -- 172.21.15.31:0/935485285 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2a8003ab0 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.989+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac19fd50 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.989+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac100130 con 0x7fc2ac0fe9f0 2024-09-16T08:54:15.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.990+0000 7fc2a27fc640 1 -- 172.21.15.31:0/935485285 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc2a8003ce0 con 0x7fc2ac0fe9f0 2024-09-16T08:54:16.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.990+0000 7fc2a27fc640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc29806cc50 0x7fc29806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:16.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.991+0000 7fc2b08ef640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc29806cc50 0x7fc29806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:16.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.991+0000 7fc2a27fc640 1 -- 172.21.15.31:0/935485285 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc2a80ad0a0 con 0x7fc2ac0fe9f0 2024-09-16T08:54:16.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.991+0000 7fc2b08ef640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc29806cc50 0x7fc29806f110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005fd0 tx=0x7fc29c005ea0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:16.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:15.995+0000 7fc2a27fc640 1 -- 172.21.15.31:0/935485285 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc2a8077710 con 0x7fc2ac0fe9f0 2024-09-16T08:54:16.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.207+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc2ac068220 con 0x7fc2ac0fe9f0 2024-09-16T08:54:16.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.207+0000 7fc2a27fc640 1 -- 172.21.15.31:0/935485285 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc2a807ad30 con 0x7fc2ac0fe9f0 2024-09-16T08:54:16.209 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:16.210 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T08:54:16.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc29806cc50 msgr2=0x7fc29806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:16.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc29806cc50 0x7fc29806f110 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc29c005fd0 tx=0x7fc29c005ea0 comp rx=0 tx=0).stop 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac0fe9f0 msgr2=0x7fc2ac196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac0fe9f0 0x7fc2ac196980 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc2a802a9c0 tx=0x7fc2a80055a0 comp rx=0 tx=0).stop 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 shutdown_connections 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc29806cc50 0x7fc29806f110 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2ac1049f0 0x7fc2ac196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 --2- 172.21.15.31:0/935485285 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2ac0fe9f0 0x7fc2ac196980 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 >> 172.21.15.31:0/935485285 conn(0x7fc2ac0758a0 msgr2=0x7fc2ac0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 shutdown_connections 2024-09-16T08:54:16.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:16.210+0000 7fc2b3354640 1 -- 172.21.15.31:0/935485285 wait complete. 2024-09-16T08:54:16.882 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T08:54:16.883 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T08:54:16.883 INFO:teuthology.run_tasks:Running task print... 2024-09-16T08:54:16.896 INFO:teuthology.task.print:**** done end installing reef cephadm ... 2024-09-16T08:54:16.896 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:54:16.907 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:54:16.907 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-09-16T08:54:17.114 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:17.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:17 smithi031 ceph-mon[27844]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:17.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:17 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/935485285' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T08:54:17.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:17 smithi097 ceph-mon[35478]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:17.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:17 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/935485285' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.774+0000 7f8906571640 1 -- 172.21.15.31:0/4166042403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 msgr2=0x7f8900101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.774+0000 7f8906571640 1 --2- 172.21.15.31:0/4166042403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900101210 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f88f4009880 tx=0x7f88f402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.774+0000 7f8906571640 1 -- 172.21.15.31:0/4166042403 shutdown_connections 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.774+0000 7f8906571640 1 --2- 172.21.15.31:0/4166042403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900101210 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.774+0000 7f8906571640 1 --2- 172.21.15.31:0/4166042403 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89000ffb90 0x7f89000fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.774+0000 7f8906571640 1 -- 172.21.15.31:0/4166042403 >> 172.21.15.31:0/4166042403 conn(0x7f89000fb320 msgr2=0x7f89000fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.775+0000 7f8906571640 1 -- 172.21.15.31:0/4166042403 shutdown_connections 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.775+0000 7f8906571640 1 -- 172.21.15.31:0/4166042403 wait complete. 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.775+0000 7f8906571640 1 Processor -- start 2024-09-16T08:54:18.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.775+0000 7f8906571640 1 -- start start 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f8906571640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89000ffb90 0x7f8900195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f8906571640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f8906571640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8900196120 con 0x7f8900100d90 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f8906571640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8900196290 con 0x7f89000ffb90 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88ff7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51996/0 (socket says 172.21.15.31:51996) 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88ff7fe640 1 -- 172.21.15.31:0/4048708734 learned_addr learned my addr 172.21.15.31:0/4048708734 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88fffff640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89000ffb90 0x7f8900195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88ff7fe640 1 -- 172.21.15.31:0/4048708734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89000ffb90 msgr2=0x7f8900195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88ff7fe640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89000ffb90 0x7f8900195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88ff7fe640 1 -- 172.21.15.31:0/4048708734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4009560 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88fffff640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89000ffb90 0x7f8900195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88ff7fe640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900195b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f88f40055a0 tx=0x7f88f40055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88fd7fa640 1 -- 172.21.15.31:0/4048708734 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f4004aa0 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890019ac80 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88fd7fa640 1 -- 172.21.15.31:0/4048708734 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88f4004c00 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.776+0000 7f88fd7fa640 1 -- 172.21.15.31:0/4048708734 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88f4003b10 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.777+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890019b1f0 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.777+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890018f710 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.778+0000 7f88fd7fa640 1 -- 172.21.15.31:0/4048708734 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f88f4003d40 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.778+0000 7f88fd7fa640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f88e806cc50 0x7f88e806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.779+0000 7f88fd7fa640 1 -- 172.21.15.31:0/4048708734 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f88f40acf80 con 0x7f8900100d90 2024-09-16T08:54:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.779+0000 7f88fffff640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f88e806cc50 0x7f88e806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:18.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.779+0000 7f88fffff640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f88e806cc50 0x7f88e806f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8900100bf0 tx=0x7f88f0005f90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:18.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.783+0000 7f88fd7fa640 1 -- 172.21.15.31:0/4048708734 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f88f4077800 con 0x7f8900100d90 2024-09-16T08:54:18.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.958+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f890002c3e0 con 0x7f8900100d90 2024-09-16T08:54:18.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.963+0000 7f88fd7fa640 1 -- 172.21.15.31:0/4048708734 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v40)=0 v40) v1 ==== 143+0+0 (secure 0 0 0) 0x7f88f407ae20 con 0x7f8900100d90 2024-09-16T08:54:18.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f88e806cc50 msgr2=0x7f88e806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:18.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f88e806cc50 0x7f88e806f110 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8900100bf0 tx=0x7f88f0005f90 comp rx=0 tx=0).stop 2024-09-16T08:54:18.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 msgr2=0x7f8900195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:18.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900195b50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f88f40055a0 tx=0x7f88f40055d0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 shutdown_connections 2024-09-16T08:54:18.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f88e806cc50 0x7f88e806f110 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8900100d90 0x7f8900195b50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 --2- 172.21.15.31:0/4048708734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f89000ffb90 0x7f8900195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:18.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 >> 172.21.15.31:0/4048708734 conn(0x7f89000fb320 msgr2=0x7f8900104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:18.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 shutdown_connections 2024-09-16T08:54:18.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:18.965+0000 7f8906571640 1 -- 172.21.15.31:0/4048708734 wait complete. 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/4048708734' entity='client.admin' 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:54:19.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:19 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:19.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/4048708734' entity='client.admin' 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:54:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:19 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:19.762 INFO:teuthology.run_tasks:Running task print... 2024-09-16T08:54:19.772 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-09-16T08:54:19.772 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:54:19.795 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:54:19.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T08:54:20.009 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:21.381 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:21 smithi031 ceph-mon[27844]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:21.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:21 smithi097 ceph-mon[35478]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 -- 172.21.15.31:0/3964251660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c100d90 msgr2=0x7f5b5c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3964251660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c100d90 0x7f5b5c101210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009880 tx=0x7f5b5002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 -- 172.21.15.31:0/3964251660 shutdown_connections 2024-09-16T08:54:21.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3964251660 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c100d90 0x7f5b5c101210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3964251660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b5c0ffb90 0x7f5b5c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 -- 172.21.15.31:0/3964251660 >> 172.21.15.31:0/3964251660 conn(0x7f5b5c0fb320 msgr2=0x7f5b5c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 -- 172.21.15.31:0/3964251660 shutdown_connections 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.701+0000 7f5b61c27640 1 -- 172.21.15.31:0/3964251660 wait complete. 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b61c27640 1 Processor -- start 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b61c27640 1 -- start start 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b61c27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c0ffb90 0x7f5b5c195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b61c27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b5c100d90 0x7f5b5c195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b61c27640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c196140 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b61c27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c1962b0 con 0x7f5b5c100d90 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c0ffb90 0x7f5b5c195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c0ffb90 0x7f5b5c195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39088/0 (socket says 172.21.15.31:39088) 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b5b7fe640 1 -- 172.21.15.31:0/3804595289 learned_addr learned my addr 172.21.15.31:0/3804595289 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:21.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.702+0000 7f5b5affd640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b5c100d90 0x7f5b5c195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b5b7fe640 1 -- 172.21.15.31:0/3804595289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b5c100d90 msgr2=0x7f5b5c195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b5b7fe640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b5c100d90 0x7f5b5c195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b5b7fe640 1 -- 172.21.15.31:0/3804595289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b50009560 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b5b7fe640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c0ffb90 0x7f5b5c195630 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00d900 tx=0x7f5b4c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b58ff9640 1 -- 172.21.15.31:0/3804595289 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b4c009c00 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c19ad50 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b58ff9640 1 -- 172.21.15.31:0/3804595289 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b4c009d60 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b58ff9640 1 -- 172.21.15.31:0/3804595289 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b4c00f660 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.703+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c19b290 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.704+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c18f730 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.705+0000 7f5b58ff9640 1 -- 172.21.15.31:0/3804595289 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5b4c010460 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.705+0000 7f5b58ff9640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5b4406cc50 0x7f5b4406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:21.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.705+0000 7f5b58ff9640 1 -- 172.21.15.31:0/3804595289 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5b4c08bd30 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.705+0000 7f5b5affd640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5b4406cc50 0x7f5b4406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:21.708 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.706+0000 7f5b5affd640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5b4406cc50 0x7f5b4406f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000c00 tx=0x7f5b500314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:21.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.709+0000 7f5b58ff9640 1 -- 172.21.15.31:0/3804595289 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5b4c0564f0 con 0x7f5b5c0ffb90 2024-09-16T08:54:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.892+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b5c02c560 con 0x7f5b4406cc50 2024-09-16T08:54:21.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.893+0000 7f5b58ff9640 1 -- 172.21.15.31:0/3804595289 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5b5c02c560 con 0x7f5b4406cc50 2024-09-16T08:54:21.895 INFO:teuthology.orchestra.run.smithi031.stdout:Backend: cephadm 2024-09-16T08:54:21.895 INFO:teuthology.orchestra.run.smithi031.stdout:Available: Yes 2024-09-16T08:54:21.895 INFO:teuthology.orchestra.run.smithi031.stdout:Paused: No 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5b4406cc50 msgr2=0x7f5b4406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5b4406cc50 0x7f5b4406f110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5b50000c00 tx=0x7f5b500314e0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c0ffb90 msgr2=0x7f5b5c195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c0ffb90 0x7f5b5c195630 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c00d900 tx=0x7f5b4c00ddc0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 shutdown_connections 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5b4406cc50 0x7f5b4406f110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b5c100d90 0x7f5b5c195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 --2- 172.21.15.31:0/3804595289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b5c0ffb90 0x7f5b5c195630 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 >> 172.21.15.31:0/3804595289 conn(0x7f5b5c0fb320 msgr2=0x7f5b5c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 shutdown_connections 2024-09-16T08:54:21.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:21.895+0000 7f5b61c27640 1 -- 172.21.15.31:0/3804595289 wait complete. 2024-09-16T08:54:22.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T08:54:22.878 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:23.277 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:23 smithi031 ceph-mon[27844]: from='client.14506 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:23.277 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:23 smithi031 ceph-mon[27844]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:23.277 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:23.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:23 smithi097 ceph-mon[35478]: from='client.14506 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:23.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:23 smithi097 ceph-mon[35478]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:23.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:23 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:24.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 -- 172.21.15.31:0/3533250678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 msgr2=0x7f60400fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:24.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 --2- 172.21.15.31:0/3533250678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f60400fff90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6030009880 tx=0x7f603002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:24.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 -- 172.21.15.31:0/3533250678 shutdown_connections 2024-09-16T08:54:24.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 --2- 172.21.15.31:0/3533250678 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6040100d90 0x7f6040101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:24.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 --2- 172.21.15.31:0/3533250678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f60400fff90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:24.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 -- 172.21.15.31:0/3533250678 >> 172.21.15.31:0/3533250678 conn(0x7f60400fb320 msgr2=0x7f60400fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 -- 172.21.15.31:0/3533250678 shutdown_connections 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.560+0000 7f604506c640 1 -- 172.21.15.31:0/3533250678 wait complete. 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.561+0000 7f604506c640 1 Processor -- start 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.561+0000 7f604506c640 1 -- start start 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.561+0000 7f604506c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f6040195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.561+0000 7f604506c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6040100d90 0x7f6040195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.561+0000 7f604506c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6040196220 con 0x7f60400ffb90 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.561+0000 7f604506c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6040196390 con 0x7f6040100d90 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f6040195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603ed9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f6040195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39104/0 (socket says 172.21.15.31:39104) 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603ed9d640 1 -- 172.21.15.31:0/3606040876 learned_addr learned my addr 172.21.15.31:0/3606040876 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:24.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603e59c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6040100d90 0x7f6040195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603ed9d640 1 -- 172.21.15.31:0/3606040876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6040100d90 msgr2=0x7f6040195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603ed9d640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6040100d90 0x7f6040195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603ed9d640 1 -- 172.21.15.31:0/3606040876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009560 con 0x7f60400ffb90 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f603ed9d640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f6040195710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6030005610 tx=0x7f6030005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f6027fff640 1 -- 172.21.15.31:0/3606040876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6030004ba0 con 0x7f60400ffb90 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f6027fff640 1 -- 172.21.15.31:0/3606040876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6030004d00 con 0x7f60400ffb90 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f604019add0 con 0x7f60400ffb90 2024-09-16T08:54:24.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f6027fff640 1 -- 172.21.15.31:0/3606040876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6030003bf0 con 0x7f60400ffb90 2024-09-16T08:54:24.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.562+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f604019b2e0 con 0x7f60400ffb90 2024-09-16T08:54:24.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.564+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6040108800 con 0x7f60400ffb90 2024-09-16T08:54:24.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.564+0000 7f6027fff640 1 -- 172.21.15.31:0/3606040876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6030031640 con 0x7f60400ffb90 2024-09-16T08:54:24.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.565+0000 7f6027fff640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f602806cc60 0x7f602806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:24.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.565+0000 7f603e59c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f602806cc60 0x7f602806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:24.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.565+0000 7f6027fff640 1 -- 172.21.15.31:0/3606040876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f60300ad270 con 0x7f60400ffb90 2024-09-16T08:54:24.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.565+0000 7f603e59c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f602806cc60 0x7f602806f120 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6040196c30 tx=0x7f6034008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:24.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.569+0000 7f6027fff640 1 -- 172.21.15.31:0/3606040876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6030077980 con 0x7f60400ffb90 2024-09-16T08:54:24.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.747+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f604019b500 con 0x7f602806cc60 2024-09-16T08:54:24.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.755+0000 7f6027fff640 1 -- 172.21.15.31:0/3606040876 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3080 (secure 0 0 0) 0x7f604019b500 con 0x7f602806cc60 2024-09-16T08:54:24.757 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:54:24.757 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (3m) 2m ago 5m 26.5M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (6m) 2m ago 6m 7063k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 66d87845da99 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (5m) 69s ago 5m 7033k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 edd14d48bff9 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (6m) 2m ago 6m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (5m) 69s ago 5m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (3m) 2m ago 5m 80.4M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:9283,8765,8443 running (7m) 2m ago 7m 476M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 1c78052ec7c9 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (4m) 69s ago 4m 440M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b82e4c86f40c 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (7m) 2m ago 7m 45.5M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (4m) 69s ago 4m 40.9M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (6m) 2m ago 6m 15.4M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (4m) 69s ago 4m 16.4M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 2m ago 3m 37.4M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:54:24.758 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 2m ago 2m 46.6M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:54:24.759 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 2m ago 2m 12.4M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:54:24.759 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (2m) 69s ago 2m 47.5M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:54:24.759 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (99s) 69s ago 99s 39.2M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:54:24.759 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (75s) 69s ago 74s 11.6M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:54:24.759 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 2m ago 5m 42.7M - 2.43.0 a07b618ecd1d b0b74488871e 2024-09-16T08:54:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f602806cc60 msgr2=0x7f602806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:24.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f602806cc60 0x7f602806f120 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6040196c30 tx=0x7f6034008040 comp rx=0 tx=0).stop 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 msgr2=0x7f6040195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f6040195710 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6030005610 tx=0x7f6030005640 comp rx=0 tx=0).stop 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 shutdown_connections 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f602806cc60 0x7f602806f120 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6040100d90 0x7f6040195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 --2- 172.21.15.31:0/3606040876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60400ffb90 0x7f6040195710 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.757+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 >> 172.21.15.31:0/3606040876 conn(0x7f60400fb320 msgr2=0x7f6040103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.758+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 shutdown_connections 2024-09-16T08:54:24.760 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:24.758+0000 7f604506c640 1 -- 172.21.15.31:0/3606040876 wait complete. 2024-09-16T08:54:25.408 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:25 smithi031 ceph-mon[27844]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:25.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T08:54:25.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:25 smithi097 ceph-mon[35478]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:25.636 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:26.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:26 smithi031 ceph-mon[27844]: from='client.14510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:26.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:26 smithi097 ceph-mon[35478]: from='client.14510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 -- 172.21.15.31:0/2007448107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c100d90 msgr2=0x7fb16c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 --2- 172.21.15.31:0/2007448107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c100d90 0x7fb16c101210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb160009880 tx=0x7fb16002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 -- 172.21.15.31:0/2007448107 shutdown_connections 2024-09-16T08:54:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 --2- 172.21.15.31:0/2007448107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c100d90 0x7fb16c101210 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 --2- 172.21.15.31:0/2007448107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c0ffb90 0x7fb16c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 -- 172.21.15.31:0/2007448107 >> 172.21.15.31:0/2007448107 conn(0x7fb16c0fb320 msgr2=0x7fb16c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 -- 172.21.15.31:0/2007448107 shutdown_connections 2024-09-16T08:54:27.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 -- 172.21.15.31:0/2007448107 wait complete. 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 Processor -- start 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.280+0000 7fb172a36640 1 -- start start 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb172a36640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c0ffb90 0x7fb16c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb172a36640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c100d90 0x7fb16c0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb172a36640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c072de0 con 0x7fb16c0ffb90 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb172a36640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c072f50 con 0x7fb16c100d90 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c0ffb90 0x7fb16c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c0ffb90 0x7fb16c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39122/0 (socket says 172.21.15.31:39122) 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16bfff640 1 -- 172.21.15.31:0/886327858 learned_addr learned my addr 172.21.15.31:0/886327858 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:27.283 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16b7fe640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c100d90 0x7fb16c0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16bfff640 1 -- 172.21.15.31:0/886327858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c100d90 msgr2=0x7fb16c0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:27.284 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16bfff640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c100d90 0x7fb16c0728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16bfff640 1 -- 172.21.15.31:0/886327858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb160009560 con 0x7fb16c0ffb90 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16b7fe640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c100d90 0x7fb16c0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.281+0000 7fb16bfff640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c0ffb90 0x7fb16c074250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b3d0 tx=0x7fb15c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.282+0000 7fb1697fa640 1 -- 172.21.15.31:0/886327858 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb15c011800 con 0x7fb16c0ffb90 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.282+0000 7fb1697fa640 1 -- 172.21.15.31:0/886327858 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb15c011e20 con 0x7fb16c0ffb90 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.282+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c073230 con 0x7fb16c0ffb90 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.282+0000 7fb1697fa640 1 -- 172.21.15.31:0/886327858 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb15c00f510 con 0x7fb16c0ffb90 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.282+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c19d790 con 0x7fb16c0ffb90 2024-09-16T08:54:27.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.283+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c1091f0 con 0x7fb16c0ffb90 2024-09-16T08:54:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.283+0000 7fb1697fa640 1 -- 172.21.15.31:0/886327858 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb15c010380 con 0x7fb16c0ffb90 2024-09-16T08:54:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.284+0000 7fb1697fa640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb15406cc50 0x7fb15406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.284+0000 7fb1697fa640 1 -- 172.21.15.31:0/886327858 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb15c08bc40 con 0x7fb16c0ffb90 2024-09-16T08:54:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.284+0000 7fb16b7fe640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb15406cc50 0x7fb15406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:27.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.284+0000 7fb16b7fe640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb15406cc50 0x7fb15406f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb160000c00 tx=0x7fb1600314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:27.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.288+0000 7fb1697fa640 1 -- 172.21.15.31:0/886327858 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb16c1091f0 con 0x7fb16c0ffb90 2024-09-16T08:54:27.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:27 smithi031 ceph-mon[27844]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:27.472 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.471+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb16c109180 con 0x7fb15406cc50 2024-09-16T08:54:27.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.475+0000 7fb1697fa640 1 -- 172.21.15.31:0/886327858 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1290 (secure 0 0 0) 0x7fb16c109180 con 0x7fb15406cc50 2024-09-16T08:54:27.476 INFO:teuthology.orchestra.run.smithi031.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T08:54:27.476 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-09-16T08:54:27.476 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter 2/2 2m ago 6m * 2024-09-16T08:54:27.477 INFO:teuthology.orchestra.run.smithi031.stdout:crash 2/2 2m ago 6m * 2024-09-16T08:54:27.477 INFO:teuthology.orchestra.run.smithi031.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-09-16T08:54:27.477 INFO:teuthology.orchestra.run.smithi031.stdout:mgr 2/2 2m ago 6m count:2 2024-09-16T08:54:27.477 INFO:teuthology.orchestra.run.smithi031.stdout:mon 2/2 2m ago 5m smithi031:172.21.15.31=smithi031;smithi097:172.21.15.97=smithi097;count:2 2024-09-16T08:54:27.477 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-09-16T08:54:27.477 INFO:teuthology.orchestra.run.smithi031.stdout:osd 6 2m ago - 2024-09-16T08:54:27.477 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb15406cc50 msgr2=0x7fb15406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb15406cc50 0x7fb15406f110 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb160000c00 tx=0x7fb1600314e0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c0ffb90 msgr2=0x7fb16c074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c0ffb90 0x7fb16c074250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00b3d0 tx=0x7fb15c00b890 comp rx=0 tx=0).stop 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 shutdown_connections 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb15406cc50 0x7fb15406f110 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb16c100d90 0x7fb16c0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 --2- 172.21.15.31:0/886327858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb16c0ffb90 0x7fb16c074250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:27.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 >> 172.21.15.31:0/886327858 conn(0x7fb16c0fb320 msgr2=0x7fb16c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 shutdown_connections 2024-09-16T08:54:27.479 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:27.477+0000 7fb172a36640 1 -- 172.21.15.31:0/886327858 wait complete. 2024-09-16T08:54:27.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:27 smithi097 ceph-mon[35478]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:28.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T08:54:28.410 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:29.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:29 smithi031 ceph-mon[27844]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:29.480 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:29 smithi031 ceph-mon[27844]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:29.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:29 smithi097 ceph-mon[35478]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:29.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:29 smithi097 ceph-mon[35478]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:30.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 -- 172.21.15.31:0/828181353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800feaa0 msgr2=0x7f93800feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:30.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 --2- 172.21.15.31:0/828181353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800feaa0 0x7f93800feea0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9370009880 tx=0x7f937002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 -- 172.21.15.31:0/828181353 shutdown_connections 2024-09-16T08:54:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 --2- 172.21.15.31:0/828181353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93800ffca0 0x7f9380100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 --2- 172.21.15.31:0/828181353 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800feaa0 0x7f93800feea0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 -- 172.21.15.31:0/828181353 >> 172.21.15.31:0/828181353 conn(0x7f93800fa250 msgr2=0x7f93800fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 -- 172.21.15.31:0/828181353 shutdown_connections 2024-09-16T08:54:30.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.025+0000 7f9384ce5640 1 -- 172.21.15.31:0/828181353 wait complete. 2024-09-16T08:54:30.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.026+0000 7f9384ce5640 1 Processor -- start 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.026+0000 7f9384ce5640 1 -- start start 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f9384ce5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93800feaa0 0x7f9380068fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f9384ce5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800ffca0 0x7f938019a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f9384ce5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938019a950 con 0x7f93800ffca0 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f9384ce5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938019aa90 con 0x7f93800feaa0 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800ffca0 0x7f938019a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800ffca0 0x7f938019a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39150/0 (socket says 172.21.15.31:39150) 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937dd9b640 1 -- 172.21.15.31:0/3346365580 learned_addr learned my addr 172.21.15.31:0/3346365580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937e59c640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93800feaa0 0x7f9380068fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937dd9b640 1 -- 172.21.15.31:0/3346365580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93800feaa0 msgr2=0x7f9380068fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937dd9b640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93800feaa0 0x7f9380068fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937dd9b640 1 -- 172.21.15.31:0/3346365580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370009560 con 0x7f93800ffca0 2024-09-16T08:54:30.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f937dd9b640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800ffca0 0x7f938019a380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f937400d8d0 tx=0x7f937400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f936f7fe640 1 -- 172.21.15.31:0/3346365580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937400bea0 con 0x7f93800ffca0 2024-09-16T08:54:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.027+0000 7f936f7fe640 1 -- 172.21.15.31:0/3346365580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9374009c00 con 0x7f93800ffca0 2024-09-16T08:54:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.028+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938019ad40 con 0x7f93800ffca0 2024-09-16T08:54:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.028+0000 7f936f7fe640 1 -- 172.21.15.31:0/3346365580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937400f710 con 0x7f93800ffca0 2024-09-16T08:54:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.028+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938019b310 con 0x7f93800ffca0 2024-09-16T08:54:30.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.029+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f938018e620 con 0x7f93800ffca0 2024-09-16T08:54:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.029+0000 7f936f7fe640 1 -- 172.21.15.31:0/3346365580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f937400f990 con 0x7f93800ffca0 2024-09-16T08:54:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.030+0000 7f936f7fe640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f936806cc50 0x7f936806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.030+0000 7f936f7fe640 1 -- 172.21.15.31:0/3346365580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f937408bbc0 con 0x7f93800ffca0 2024-09-16T08:54:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.030+0000 7f937e59c640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f936806cc50 0x7f936806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:30.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.030+0000 7f937e59c640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f936806cc50 0x7f936806f110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9370009230 tx=0x7f93700314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:30.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.034+0000 7f936f7fe640 1 -- 172.21.15.31:0/3346365580 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f9374056380 con 0x7f93800ffca0 2024-09-16T08:54:30.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.216+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f938002c4f0 con 0x7f936806cc50 2024-09-16T08:54:30.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.217+0000 7f936f7fe640 1 -- 172.21.15.31:0/3346365580 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+145 (secure 0 0 0) 0x7f938002c4f0 con 0x7f936806cc50 2024-09-16T08:54:30.218 INFO:teuthology.orchestra.run.smithi031.stdout:HOST ADDR LABELS STATUS 2024-09-16T08:54:30.218 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 172.21.15.31 2024-09-16T08:54:30.218 INFO:teuthology.orchestra.run.smithi031.stdout:smithi097 172.21.15.97 2024-09-16T08:54:30.218 INFO:teuthology.orchestra.run.smithi031.stdout:2 hosts in cluster 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f936806cc50 msgr2=0x7f936806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f936806cc50 0x7f936806f110 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9370009230 tx=0x7f93700314e0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800ffca0 msgr2=0x7f938019a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800ffca0 0x7f938019a380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f937400d8d0 tx=0x7f937400dd90 comp rx=0 tx=0).stop 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 shutdown_connections 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f936806cc50 0x7f936806f110 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93800ffca0 0x7f938019a380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 --2- 172.21.15.31:0/3346365580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93800feaa0 0x7f9380068fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 >> 172.21.15.31:0/3346365580 conn(0x7f93800fa250 msgr2=0x7f9380103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:30.221 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 shutdown_connections 2024-09-16T08:54:30.222 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:30.220+0000 7f9384ce5640 1 -- 172.21.15.31:0/3346365580 wait complete. 2024-09-16T08:54:30.876 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T08:54:31.081 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:31.457 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:31 smithi031 ceph-mon[27844]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:31.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:31 smithi097 ceph-mon[35478]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:32.219 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:32 smithi031 ceph-mon[27844]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:32.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:32 smithi097 ceph-mon[35478]: from='client.14518 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:32.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 -- 172.21.15.31:0/587627328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 msgr2=0x7fb318101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:32.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/587627328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318101210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb300009880 tx=0x7fb30002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:32.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 -- 172.21.15.31:0/587627328 shutdown_connections 2024-09-16T08:54:32.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/587627328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318101210 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:32.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/587627328 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180ffb90 0x7fb3180fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 -- 172.21.15.31:0/587627328 >> 172.21.15.31:0/587627328 conn(0x7fb3180fb320 msgr2=0x7fb3180fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 -- 172.21.15.31:0/587627328 shutdown_connections 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.717+0000 7fb31d6ba640 1 -- 172.21.15.31:0/587627328 wait complete. 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb31d6ba640 1 Processor -- start 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb31d6ba640 1 -- start start 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb31d6ba640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180ffb90 0x7fb318195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb31d6ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb31d6ba640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318196140 con 0x7fb318100d90 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb31d6ba640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3181962b0 con 0x7fb3180ffb90 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb30ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb30ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59144/0 (socket says 172.21.15.31:59144) 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb30ffff640 1 -- 172.21.15.31:0/1607265128 learned_addr learned my addr 172.21.15.31:0/1607265128 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:32.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.718+0000 7fb316ffd640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180ffb90 0x7fb318195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb30ffff640 1 -- 172.21.15.31:0/1607265128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180ffb90 msgr2=0x7fb318195630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb30ffff640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180ffb90 0x7fb318195630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:32.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb30ffff640 1 -- 172.21.15.31:0/1607265128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300009560 con 0x7fb318100d90 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb30ffff640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318195b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb300005520 tx=0x7fb300005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb314ff9640 1 -- 172.21.15.31:0/1607265128 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb300004aa0 con 0x7fb318100d90 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb314ff9640 1 -- 172.21.15.31:0/1607265128 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb300004c00 con 0x7fb318100d90 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb314ff9640 1 -- 172.21.15.31:0/1607265128 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb300003b10 con 0x7fb318100d90 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31819acf0 con 0x7fb318100d90 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.719+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31819b260 con 0x7fb318100d90 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.720+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31818f730 con 0x7fb318100d90 2024-09-16T08:54:32.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.721+0000 7fb314ff9640 1 -- 172.21.15.31:0/1607265128 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb300031d20 con 0x7fb318100d90 2024-09-16T08:54:32.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.722+0000 7fb314ff9640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb30406cc50 0x7fb30406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:32.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.722+0000 7fb314ff9640 1 -- 172.21.15.31:0/1607265128 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb3000acf90 con 0x7fb318100d90 2024-09-16T08:54:32.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.722+0000 7fb316ffd640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb30406cc50 0x7fb30406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:32.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.722+0000 7fb316ffd640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb30406cc50 0x7fb30406f110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb308005fd0 tx=0x7fb30800b390 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:32.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.726+0000 7fb314ff9640 1 -- 172.21.15.31:0/1607265128 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb3000777d0 con 0x7fb318100d90 2024-09-16T08:54:32.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.905+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fb31802c4f0 con 0x7fb30406cc50 2024-09-16T08:54:32.907 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.906+0000 7fb314ff9640 1 -- 172.21.15.31:0/1607265128 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7fb31802c4f0 con 0x7fb30406cc50 2024-09-16T08:54:32.907 INFO:teuthology.orchestra.run.smithi031.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T08:54:32.907 INFO:teuthology.orchestra.run.smithi031.stdout:smithi031 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53310008400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T08:54:32.907 INFO:teuthology.orchestra.run.smithi031.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 74s ago Has a FileSystem, LVM detected 2024-09-16T08:54:32.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb30406cc50 msgr2=0x7fb30406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:32.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb30406cc50 0x7fb30406f110 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb308005fd0 tx=0x7fb30800b390 comp rx=0 tx=0).stop 2024-09-16T08:54:32.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 msgr2=0x7fb318195b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:32.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318195b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb300005520 tx=0x7fb300005550 comp rx=0 tx=0).stop 2024-09-16T08:54:32.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 shutdown_connections 2024-09-16T08:54:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb30406cc50 0x7fb30406f110 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb318100d90 0x7fb318195b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 --2- 172.21.15.31:0/1607265128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3180ffb90 0x7fb318195630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 >> 172.21.15.31:0/1607265128 conn(0x7fb3180fb320 msgr2=0x7fb318104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 shutdown_connections 2024-09-16T08:54:32.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:32.908+0000 7fb31d6ba640 1 -- 172.21.15.31:0/1607265128 wait complete. 2024-09-16T08:54:33.206 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:33 smithi031 ceph-mon[27844]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:33.206 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:33 smithi031 ceph-mon[27844]: from='client.14522 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:33.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:33 smithi097 ceph-mon[35478]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:33.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:33 smithi097 ceph-mon[35478]: from='client.14522 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:33.732 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:54:33.745 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:54:33.746 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-09-16T08:54:33.951 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:34.339 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:34 smithi031 ceph-mon[27844]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:34.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:34 smithi097 ceph-mon[35478]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:35.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 -- 172.21.15.31:0/2441307422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 msgr2=0x7f3798101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:35.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 --2- 172.21.15.31:0/2441307422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798101210 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3780009880 tx=0x7f378002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:35.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 -- 172.21.15.31:0/2441307422 shutdown_connections 2024-09-16T08:54:35.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 --2- 172.21.15.31:0/2441307422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798101210 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:35.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 --2- 172.21.15.31:0/2441307422 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37980ffb90 0x7f37980fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 -- 172.21.15.31:0/2441307422 >> 172.21.15.31:0/2441307422 conn(0x7f37980fb320 msgr2=0x7f37980fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 -- 172.21.15.31:0/2441307422 shutdown_connections 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.592+0000 7f379d31d640 1 -- 172.21.15.31:0/2441307422 wait complete. 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f379d31d640 1 Processor -- start 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f379d31d640 1 -- start start 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f379d31d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37980ffb90 0x7f37981955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f379d31d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f379d31d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3798196100 con 0x7f3798100d90 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f379d31d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3798196270 con 0x7f37980ffb90 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f378ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f378ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59168/0 (socket says 172.21.15.31:59168) 2024-09-16T08:54:35.605 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f378ffff640 1 -- 172.21.15.31:0/1382558606 learned_addr learned my addr 172.21.15.31:0/1382558606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f3796ffd640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37980ffb90 0x7f37981955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f378ffff640 1 -- 172.21.15.31:0/1382558606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37980ffb90 msgr2=0x7f37981955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f378ffff640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37980ffb90 0x7f37981955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.593+0000 7f378ffff640 1 -- 172.21.15.31:0/1382558606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3780009560 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.594+0000 7f378ffff640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798195b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3780005640 tx=0x7f3780005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.594+0000 7f3794ff9640 1 -- 172.21.15.31:0/1382558606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3780004ba0 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.594+0000 7f3794ff9640 1 -- 172.21.15.31:0/1382558606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3780004d00 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.594+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379819acb0 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.594+0000 7f3794ff9640 1 -- 172.21.15.31:0/1382558606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3780003c30 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.594+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379819b220 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.595+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379818f710 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.595+0000 7f3794ff9640 1 -- 172.21.15.31:0/1382558606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3780031640 con 0x7f3798100d90 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.596+0000 7f3794ff9640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f378406cc50 0x7f378406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:35.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.596+0000 7f3794ff9640 1 -- 172.21.15.31:0/1382558606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f37800ad0e0 con 0x7f3798100d90 2024-09-16T08:54:35.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.596+0000 7f3796ffd640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f378406cc50 0x7f378406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:35.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.596+0000 7f3796ffd640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f378406cc50 0x7f378406f110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3788005db0 tx=0x7f3788005d40 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:35.607 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.600+0000 7f3794ff9640 1 -- 172.21.15.31:0/1382558606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3780077870 con 0x7f3798100d90 2024-09-16T08:54:35.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:35.799+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f379802c4f0 con 0x7f378406cc50 2024-09-16T08:54:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:36 smithi097 ceph-mon[35478]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:36 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-16T08:54:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:36 smithi097 ceph-mon[35478]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:37.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:36 smithi031 ceph-mon[27844]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:54:37.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:36 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-16T08:54:37.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:36 smithi031 ceph-mon[27844]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:37.309 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.308+0000 7f3794ff9640 1 -- 172.21.15.31:0/1382558606 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f379802c4f0 con 0x7f378406cc50 2024-09-16T08:54:37.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f378406cc50 msgr2=0x7f378406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:37.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f378406cc50 0x7f378406f110 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3788005db0 tx=0x7f3788005d40 comp rx=0 tx=0).stop 2024-09-16T08:54:37.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 msgr2=0x7f3798195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:37.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798195b30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3780005640 tx=0x7f3780005670 comp rx=0 tx=0).stop 2024-09-16T08:54:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 shutdown_connections 2024-09-16T08:54:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f378406cc50 0x7f378406f110 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3788005db0 tx=0x7f3788005d40 comp rx=0 tx=0).stop 2024-09-16T08:54:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3798100d90 0x7f3798195b30 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 --2- 172.21.15.31:0/1382558606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37980ffb90 0x7f37981955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 >> 172.21.15.31:0/1382558606 conn(0x7f37980fb320 msgr2=0x7f3798104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 shutdown_connections 2024-09-16T08:54:37.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:37.312+0000 7f379d31d640 1 -- 172.21.15.31:0/1382558606 wait complete. 2024-09-16T08:54:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[27840]: 2024-09-16T08:54:37.289+0000 7f11eebd5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T08:54:37.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: fsmap cephfs:0 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:37.985 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:37 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:38.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: osdmap e42: 6 total, 6 up, 6 in 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: osdmap e43: 6 total, 6 up, 6 in 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T08:54:38.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: osdmap e44: 6 total, 6 up, 6 in 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: fsmap cephfs:0 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:38.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:37 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:39.182 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:39 smithi031 ceph-mon[27844]: Saving service mds.cephfs spec with placement count:4 2024-09-16T08:54:39.182 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:39 smithi031 ceph-mon[27844]: Deploying daemon mds.cephfs.smithi031.lcfutm on smithi031 2024-09-16T08:54:39.182 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:39.182 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:39.182 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:39 smithi031 ceph-mon[27844]: pgmap v197: 65 pgs: 17 active+clean, 16 creating+peering, 32 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:39.182 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:39 smithi031 ceph-mon[27844]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T08:54:39.283 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:39.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:39 smithi097 ceph-mon[35478]: Saving service mds.cephfs spec with placement count:4 2024-09-16T08:54:39.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:39 smithi097 ceph-mon[35478]: Deploying daemon mds.cephfs.smithi031.lcfutm on smithi031 2024-09-16T08:54:39.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:39.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:39.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:39 smithi097 ceph-mon[35478]: pgmap v197: 65 pgs: 17 active+clean, 16 creating+peering, 32 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:39.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:39 smithi097 ceph-mon[35478]: osdmap e45: 6 total, 6 up, 6 in 2024-09-16T08:54:40.526 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:40 smithi031 ceph-mon[27844]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T08:54:40.526 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:40 smithi031 ceph-mon[27844]: pgmap v200: 65 pgs: 29 active+clean, 19 creating+peering, 17 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:40.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:40 smithi097 ceph-mon[35478]: osdmap e46: 6 total, 6 up, 6 in 2024-09-16T08:54:40.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:40 smithi097 ceph-mon[35478]: pgmap v200: 65 pgs: 29 active+clean, 19 creating+peering, 17 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:41.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.061+0000 7f40eb99b640 1 -- 172.21.15.31:0/203639760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 msgr2=0x7f40e4102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.061+0000 7f40eb99b640 1 --2- 172.21.15.31:0/203639760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4102a20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f40e00098b0 tx=0x7f40e002a4d0 comp rx=0 tx=0).stop 2024-09-16T08:54:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.061+0000 7f40eb99b640 1 -- 172.21.15.31:0/203639760 shutdown_connections 2024-09-16T08:54:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.061+0000 7f40eb99b640 1 --2- 172.21.15.31:0/203639760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e4102f60 0x7f40e4105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.061+0000 7f40eb99b640 1 --2- 172.21.15.31:0/203639760 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4102a20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.061+0000 7f40eb99b640 1 -- 172.21.15.31:0/203639760 >> 172.21.15.31:0/203639760 conn(0x7f40e40fa250 msgr2=0x7f40e40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.062+0000 7f40eb99b640 1 -- 172.21.15.31:0/203639760 shutdown_connections 2024-09-16T08:54:41.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.062+0000 7f40eb99b640 1 -- 172.21.15.31:0/203639760 wait complete. 2024-09-16T08:54:41.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.062+0000 7f40eb99b640 1 Processor -- start 2024-09-16T08:54:41.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.062+0000 7f40eb99b640 1 -- start start 2024-09-16T08:54:41.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40eb99b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:41.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40eb99b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e4102f60 0x7f40e4195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:41.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40eb99b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e4196150 con 0x7f40e4100630 2024-09-16T08:54:41.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40eb99b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e41962c0 con 0x7f40e4102f60 2024-09-16T08:54:41.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e9737640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e9737640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59194/0 (socket says 172.21.15.31:59194) 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e9737640 1 -- 172.21.15.31:0/2099491400 learned_addr learned my addr 172.21.15.31:0/2099491400 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e8f36640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e4102f60 0x7f40e4195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e9737640 1 -- 172.21.15.31:0/2099491400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e4102f60 msgr2=0x7f40e4195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e9737640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e4102f60 0x7f40e4195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e9737640 1 -- 172.21.15.31:0/2099491400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e0009560 con 0x7f40e4100630 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40e9737640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4195640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40e00054f0 tx=0x7f40e0005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40da7fc640 1 -- 172.21.15.31:0/2099491400 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e0004aa0 con 0x7f40e4100630 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40da7fc640 1 -- 172.21.15.31:0/2099491400 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40e0004c00 con 0x7f40e4100630 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e419ad00 con 0x7f40e4100630 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.063+0000 7f40da7fc640 1 -- 172.21.15.31:0/2099491400 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40e0003b10 con 0x7f40e4100630 2024-09-16T08:54:41.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.064+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e419b220 con 0x7f40e4100630 2024-09-16T08:54:41.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.065+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e41087c0 con 0x7f40e4100630 2024-09-16T08:54:41.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.065+0000 7f40da7fc640 1 -- 172.21.15.31:0/2099491400 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f40e0003d80 con 0x7f40e4100630 2024-09-16T08:54:41.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.066+0000 7f40da7fc640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f40d006cc50 0x7f40d006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:41.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.066+0000 7f40da7fc640 1 -- 172.21.15.31:0/2099491400 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f40e00ad2a0 con 0x7f40e4100630 2024-09-16T08:54:41.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.066+0000 7f40e8f36640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f40d006cc50 0x7f40d006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:41.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.066+0000 7f40e8f36640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f40d006cc50 0x7f40d006f110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40e4196b60 tx=0x7f40d400b390 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:41.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.070+0000 7f40da7fc640 1 -- 172.21.15.31:0/2099491400 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f40e00777a0 con 0x7f40e4100630 2024-09-16T08:54:41.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.288+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f40e419b460 con 0x7f40e4100630 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.288+0000 7f40da7fc640 1 -- 172.21.15.31:0/2099491400 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1114 (secure 0 0 0) 0x7f40e007adc0 con 0x7f40e4100630 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 2 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stdout:e2 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:54:41.290 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 2 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:37.289844+0000 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 0 2024-09-16T08:54:41.291 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:in 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:up {} 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 0 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:41.292 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f40d006cc50 msgr2=0x7f40d006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f40d006cc50 0x7f40d006f110 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f40e4196b60 tx=0x7f40d400b390 comp rx=0 tx=0).stop 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 msgr2=0x7f40e4195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4195640 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f40e00054f0 tx=0x7f40e0005520 comp rx=0 tx=0).stop 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 shutdown_connections 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f40d006cc50 0x7f40d006f110 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40e4102f60 0x7f40e4195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 --2- 172.21.15.31:0/2099491400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e4100630 0x7f40e4195640 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:41.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 >> 172.21.15.31:0/2099491400 conn(0x7f40e40fa250 msgr2=0x7f40e40fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:41.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 shutdown_connections 2024-09-16T08:54:41.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:41.290+0000 7f40eb99b640 1 -- 172.21.15.31:0/2099491400 wait complete. 2024-09-16T08:54:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:41 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2099491400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:54:41.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:41 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2099491400' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:54:41.908 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:54:41.917 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:54:41.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-09-16T08:54:42.242 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:42.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:42 smithi031 ceph-mon[27844]: pgmap v201: 65 pgs: 38 active+clean, 19 creating+peering, 8 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:42.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:42 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:42.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:42 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:42.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:42 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:42.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:42 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:42.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:42 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:42.364 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:42 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:42.610 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:42 smithi097 ceph-mon[35478]: pgmap v201: 65 pgs: 38 active+clean, 19 creating+peering, 8 unknown; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:54:42.611 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:42 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:42.611 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:42 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:42.611 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:42 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:42.611 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:42 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:42.611 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:42 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:42.611 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:42 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: Deploying daemon mds.cephfs.smithi097.cagmus on smithi097 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6826/2048602032,v1:172.21.15.31:6827/2048602032] up:boot 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: daemon mds.cephfs.smithi031.lcfutm assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: Cluster is now healthy 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: fsmap cephfs:0 1 up:standby 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:creating} 2024-09-16T08:54:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:43 smithi031 ceph-mon[27844]: daemon mds.cephfs.smithi031.lcfutm is now active in filesystem cephfs as rank 0 2024-09-16T08:54:43.613 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: Deploying daemon mds.cephfs.smithi097.cagmus on smithi097 2024-09-16T08:54:43.613 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6826/2048602032,v1:172.21.15.31:6827/2048602032] up:boot 2024-09-16T08:54:43.613 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: daemon mds.cephfs.smithi031.lcfutm assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-16T08:54:43.613 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T08:54:43.613 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T08:54:43.614 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: Cluster is now healthy 2024-09-16T08:54:43.614 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: fsmap cephfs:0 1 up:standby 2024-09-16T08:54:43.614 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:54:43.614 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:creating} 2024-09-16T08:54:43.614 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:43 smithi097 ceph-mon[35478]: daemon mds.cephfs.smithi031.lcfutm is now active in filesystem cephfs as rank 0 2024-09-16T08:54:43.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.950+0000 7fe50c5f6640 1 -- 172.21.15.31:0/4019015499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 msgr2=0x7fe504101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:43.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.950+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/4019015499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504101210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe4f4009880 tx=0x7fe4f402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.950+0000 7fe50c5f6640 1 -- 172.21.15.31:0/4019015499 shutdown_connections 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.950+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/4019015499 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504101210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.950+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/4019015499 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5040ffb90 0x7fe5040fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.950+0000 7fe50c5f6640 1 -- 172.21.15.31:0/4019015499 >> 172.21.15.31:0/4019015499 conn(0x7fe5040fb320 msgr2=0x7fe5040fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 -- 172.21.15.31:0/4019015499 shutdown_connections 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 -- 172.21.15.31:0/4019015499 wait complete. 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 Processor -- start 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 -- start start 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5040ffb90 0x7fe504195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe504196150 con 0x7fe504100d90 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.951+0000 7fe50c5f6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5041962c0 con 0x7fe5040ffb90 2024-09-16T08:54:43.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe509b91640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe509b91640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44238/0 (socket says 172.21.15.31:44238) 2024-09-16T08:54:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe50a392640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5040ffb90 0x7fe504195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe509b91640 1 -- 172.21.15.31:0/2064726885 learned_addr learned my addr 172.21.15.31:0/2064726885 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe509b91640 1 -- 172.21.15.31:0/2064726885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5040ffb90 msgr2=0x7fe504195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe509b91640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5040ffb90 0x7fe504195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe509b91640 1 -- 172.21.15.31:0/2064726885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f4009560 con 0x7fe504100d90 2024-09-16T08:54:43.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe50a392640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5040ffb90 0x7fe504195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:54:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe509b91640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504195b80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40055a0 tx=0x7fe4f40055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/2064726885 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f4004aa0 con 0x7fe504100d90 2024-09-16T08:54:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/2064726885 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4f4004c00 con 0x7fe504100d90 2024-09-16T08:54:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe50419acb0 con 0x7fe504100d90 2024-09-16T08:54:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/2064726885 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4f4003b10 con 0x7fe504100d90 2024-09-16T08:54:43.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.952+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe50419b220 con 0x7fe504100d90 2024-09-16T08:54:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.953+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe50418f730 con 0x7fe504100d90 2024-09-16T08:54:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.954+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/2064726885 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fe4f4031d20 con 0x7fe504100d90 2024-09-16T08:54:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.954+0000 7fe4fb7fe640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe4f006cc50 0x7fe4f006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.954+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/2064726885 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fe4f40ad2b0 con 0x7fe504100d90 2024-09-16T08:54:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.954+0000 7fe50a392640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe4f006cc50 0x7fe4f006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:43.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.955+0000 7fe50a392640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe4f006cc50 0x7fe4f006f110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe504100bf0 tx=0x7fe500005f90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:43.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:43.959+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/2064726885 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fe4f40777d0 con 0x7fe504100d90 2024-09-16T08:54:44.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.165+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fe504108a20 con 0x7fe504100d90 2024-09-16T08:54:44.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.326+0000 7fe4fb7fe640 1 -- 172.21.15.31:0/2064726885 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7fe4f407adf0 con 0x7fe504100d90 2024-09-16T08:54:44.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe4f006cc50 msgr2=0x7fe4f006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:44.329 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe4f006cc50 0x7fe4f006f110 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe504100bf0 tx=0x7fe500005f90 comp rx=0 tx=0).stop 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 msgr2=0x7fe504195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504195b80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe4f40055a0 tx=0x7fe4f40055d0 comp rx=0 tx=0).stop 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 shutdown_connections 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fe4f006cc50 0x7fe4f006f110 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe504100d90 0x7fe504195b80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 --2- 172.21.15.31:0/2064726885 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5040ffb90 0x7fe504195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.328+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 >> 172.21.15.31:0/2064726885 conn(0x7fe5040fb320 msgr2=0x7fe504104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.329+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 shutdown_connections 2024-09-16T08:54:44.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:44.329+0000 7fe50c5f6640 1 -- 172.21.15.31:0/2064726885 wait complete. 2024-09-16T08:54:44.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:44 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6826/2048602032,v1:172.21.15.31:6827/2048602032] up:active 2024-09-16T08:54:44.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:44 smithi031 ceph-mon[27844]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 2024-09-16T08:54:44.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:44 smithi031 ceph-mon[27844]: pgmap v202: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 4 op/s 2024-09-16T08:54:44.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:44 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2064726885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-16T08:54:44.638 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:44 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6826/2048602032,v1:172.21.15.31:6827/2048602032] up:active 2024-09-16T08:54:44.638 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:44 smithi097 ceph-mon[35478]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 2024-09-16T08:54:44.638 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:44 smithi097 ceph-mon[35478]: pgmap v202: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 4 op/s 2024-09-16T08:54:44.638 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:44 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2064726885' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-16T08:54:45.138 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:54:45.150 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:54:45.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-09-16T08:54:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T08:54:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2064726885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T08:54:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 2024-09-16T08:54:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:45.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:45.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:45.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:45 smithi031 ceph-mon[27844]: Deploying daemon mds.cephfs.smithi031.mjhxgi on smithi031 2024-09-16T08:54:45.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2064726885' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:45.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:45 smithi097 ceph-mon[35478]: Deploying daemon mds.cephfs.smithi031.mjhxgi on smithi031 2024-09-16T08:54:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] up:boot 2024-09-16T08:54:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: daemon mds.cephfs.smithi097.cagmus assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T08:54:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T08:54:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T08:54:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 1 up:standby 2024-09-16T08:54:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"}]: dispatch 2024-09-16T08:54:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:creating} 2024-09-16T08:54:46.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: daemon mds.cephfs.smithi097.cagmus is now active in filesystem cephfs as rank 1 2024-09-16T08:54:46.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:46 smithi031 ceph-mon[27844]: pgmap v203: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-16T08:54:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:46.690 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] up:boot 2024-09-16T08:54:46.690 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: daemon mds.cephfs.smithi097.cagmus assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T08:54:46.690 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-16T08:54:46.690 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T08:54:46.691 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 1 up:standby 2024-09-16T08:54:46.691 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"}]: dispatch 2024-09-16T08:54:46.691 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:creating} 2024-09-16T08:54:46.691 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: daemon mds.cephfs.smithi097.cagmus is now active in filesystem cephfs as rank 1 2024-09-16T08:54:46.691 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:46 smithi097 ceph-mon[35478]: pgmap v203: 65 pgs: 65 active+clean; 579 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-16T08:54:47.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:47 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] up:active 2024-09-16T08:54:47.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:47 smithi097 ceph-mon[35478]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 2024-09-16T08:54:47.575 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:47 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] up:active 2024-09-16T08:54:47.575 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:47 smithi031 ceph-mon[27844]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 2024-09-16T08:54:48.443 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:48 smithi031 ceph-mon[27844]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1482545357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a480ffb90 msgr2=0x7f7a480fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1482545357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a480ffb90 0x7f7a480fff90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009880 tx=0x7f7a3802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1482545357 shutdown_connections 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1482545357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a48100d90 0x7f7a48101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1482545357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a480ffb90 0x7f7a480fff90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1482545357 >> 172.21.15.31:0/1482545357 conn(0x7f7a480fb320 msgr2=0x7f7a480fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1482545357 shutdown_connections 2024-09-16T08:54:48.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.618+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1482545357 wait complete. 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a4dbc4640 1 Processor -- start 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a4dbc4640 1 -- start start 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a4dbc4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480ffb90 0x7f7a48195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a4dbc4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48100d90 0x7f7a48195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a4dbc4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48196120 con 0x7f7a48100d90 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a4dbc4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a48196290 con 0x7f7a480ffb90 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48100d90 0x7f7a48195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:48.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a46ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48100d90 0x7f7a48195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:44270/0 (socket says 172.21.15.31:44270) 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a46ffd640 1 -- 172.21.15.31:0/1789019270 learned_addr learned my addr 172.21.15.31:0/1789019270 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a477fe640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480ffb90 0x7f7a48195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a46ffd640 1 -- 172.21.15.31:0/1789019270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480ffb90 msgr2=0x7f7a48195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a46ffd640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480ffb90 0x7f7a48195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a46ffd640 1 -- 172.21.15.31:0/1789019270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38009560 con 0x7f7a48100d90 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.619+0000 7f7a46ffd640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48100d90 0x7f7a48195b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00d900 tx=0x7f7a3c00ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.620+0000 7f7a44ff9640 1 -- 172.21.15.31:0/1789019270 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3c009c00 con 0x7f7a48100d90 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.620+0000 7f7a44ff9640 1 -- 172.21.15.31:0/1789019270 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a3c00b660 con 0x7f7a48100d90 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.620+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4819ad30 con 0x7f7a48100d90 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.620+0000 7f7a44ff9640 1 -- 172.21.15.31:0/1789019270 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a3c00f680 con 0x7f7a48100d90 2024-09-16T08:54:48.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.620+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4819b300 con 0x7f7a48100d90 2024-09-16T08:54:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.621+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4818f710 con 0x7f7a48100d90 2024-09-16T08:54:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.621+0000 7f7a44ff9640 1 -- 172.21.15.31:0/1789019270 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7a3c010ab0 con 0x7f7a48100d90 2024-09-16T08:54:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.622+0000 7f7a44ff9640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7a3006cc50 0x7f7a3006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.622+0000 7f7a44ff9640 1 -- 172.21.15.31:0/1789019270 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f7a3c08c000 con 0x7f7a48100d90 2024-09-16T08:54:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.622+0000 7f7a477fe640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7a3006cc50 0x7f7a3006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:48.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.622+0000 7f7a477fe640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7a3006cc50 0x7f7a3006f110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009230 tx=0x7f7a38005d50 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:48.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.626+0000 7f7a44ff9640 1 -- 172.21.15.31:0/1789019270 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7a3c0563d0 con 0x7f7a48100d90 2024-09-16T08:54:48.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:48 smithi097 ceph-mon[35478]: pgmap v204: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s wr, 7 op/s 2024-09-16T08:54:48.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:48.841+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7a48108780 con 0x7f7a48100d90 2024-09-16T08:54:49.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.353+0000 7f7a44ff9640 1 -- 172.21.15.31:0/1789019270 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7a3c0599f0 con 0x7f7a48100d90 2024-09-16T08:54:49.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7a3006cc50 msgr2=0x7f7a3006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7a3006cc50 0x7f7a3006f110 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009230 tx=0x7f7a38005d50 comp rx=0 tx=0).stop 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48100d90 msgr2=0x7f7a48195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48100d90 0x7f7a48195b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c00d900 tx=0x7f7a3c00ddc0 comp rx=0 tx=0).stop 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 shutdown_connections 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7a3006cc50 0x7f7a3006f110 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a48100d90 0x7f7a48195b50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 --2- 172.21.15.31:0/1789019270 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a480ffb90 0x7f7a48195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.356+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 >> 172.21.15.31:0/1789019270 conn(0x7f7a480fb320 msgr2=0x7f7a48104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.357+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 shutdown_connections 2024-09-16T08:54:49.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:49.357+0000 7f7a4dbc4640 1 -- 172.21.15.31:0/1789019270 wait complete. 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1789019270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:49.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:49 smithi097 ceph-mon[35478]: Deploying daemon mds.cephfs.smithi097.sbwmfq on smithi097 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1789019270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:54:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:49 smithi031 ceph-mon[27844]: Deploying daemon mds.cephfs.smithi097.sbwmfq on smithi097 2024-09-16T08:54:50.142 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:54:50.153 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:54:50.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-09-16T08:54:50.366 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:50.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:50 smithi031 ceph-mon[27844]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T08:54:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:50 smithi031 ceph-mon[27844]: Cluster is now healthy 2024-09-16T08:54:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:50 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:boot 2024-09-16T08:54:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:50 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1789019270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-09-16T08:54:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:50 smithi031 ceph-mon[27844]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 1 up:standby 2024-09-16T08:54:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:50 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"}]: dispatch 2024-09-16T08:54:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:50 smithi031 ceph-mon[27844]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-16T08:54:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:50 smithi097 ceph-mon[35478]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-16T08:54:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:50 smithi097 ceph-mon[35478]: Cluster is now healthy 2024-09-16T08:54:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:50 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:boot 2024-09-16T08:54:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:50 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1789019270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-09-16T08:54:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:50 smithi097 ceph-mon[35478]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 1 up:standby 2024-09-16T08:54:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:50 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"}]: dispatch 2024-09-16T08:54:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:50 smithi097 ceph-mon[35478]: pgmap v205: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-09-16T08:54:52.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 -- 172.21.15.31:0/1414353287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0100720 msgr2=0x7f1be0102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:52.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 --2- 172.21.15.31:0/1414353287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0100720 0x7f1be0102b10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009c20 tx=0x7f1bd002a970 comp rx=0 tx=0).stop 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 -- 172.21.15.31:0/1414353287 shutdown_connections 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 --2- 172.21.15.31:0/1414353287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be0103050 0x7f1be0105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 --2- 172.21.15.31:0/1414353287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0100720 0x7f1be0102b10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 -- 172.21.15.31:0/1414353287 >> 172.21.15.31:0/1414353287 conn(0x7f1be00fa280 msgr2=0x7f1be00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 -- 172.21.15.31:0/1414353287 shutdown_connections 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 -- 172.21.15.31:0/1414353287 wait complete. 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.042+0000 7f1be5a14640 1 Processor -- start 2024-09-16T08:54:52.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be5a14640 1 -- start start 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be5a14640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be0100720 0x7f1be01956a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be5a14640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0103050 0x7f1be0195be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be5a14640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be01961b0 con 0x7f1be0103050 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be5a14640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0196320 con 0x7f1be0100720 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bde7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0103050 0x7f1be0195be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bde7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0103050 0x7f1be0195be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48700/0 (socket says 172.21.15.31:48700) 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bde7fc640 1 -- 172.21.15.31:0/785193002 learned_addr learned my addr 172.21.15.31:0/785193002 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bdeffd640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be0100720 0x7f1be01956a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bde7fc640 1 -- 172.21.15.31:0/785193002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be0100720 msgr2=0x7f1be01956a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bde7fc640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be0100720 0x7f1be01956a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:52.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bde7fc640 1 -- 172.21.15.31:0/785193002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd0009900 con 0x7f1be0103050 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1bde7fc640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0103050 0x7f1be0195be0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800e9a0 tx=0x7f1bc800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be4a12640 1 -- 172.21.15.31:0/785193002 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc800cc90 con 0x7f1be0103050 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be4a12640 1 -- 172.21.15.31:0/785193002 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bc800cdf0 con 0x7f1be0103050 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.043+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be019adc0 con 0x7f1be0103050 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.044+0000 7f1be4a12640 1 -- 172.21.15.31:0/785193002 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bc8010430 con 0x7f1be0103050 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.044+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be019b2e0 con 0x7f1be0103050 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.045+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be0100b00 con 0x7f1be0103050 2024-09-16T08:54:52.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.045+0000 7f1be4a12640 1 -- 172.21.15.31:0/785193002 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f1bc8005020 con 0x7f1be0103050 2024-09-16T08:54:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.046+0000 7f1be4a12640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1bcc06cc50 0x7f1bcc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.046+0000 7f1be4a12640 1 -- 172.21.15.31:0/785193002 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f1bc8014070 con 0x7f1be0103050 2024-09-16T08:54:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.046+0000 7f1bdeffd640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1bcc06cc50 0x7f1bcc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.046+0000 7f1bdeffd640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1bcc06cc50 0x7f1bcc06f110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0004af0 tx=0x7f1bd003d040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:52.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.050+0000 7f1be4a12640 1 -- 172.21.15.31:0/785193002 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f1bc8056aa0 con 0x7f1be0103050 2024-09-16T08:54:52.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.272+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f1be0108820 con 0x7f1be0103050 2024-09-16T08:54:52.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.385+0000 7f1be4a12640 1 -- 172.21.15.31:0/785193002 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f1bc805a0c0 con 0x7f1be0103050 2024-09-16T08:54:52.387 INFO:teuthology.orchestra.run.smithi031.stderr:inline data disabled 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1bcc06cc50 msgr2=0x7f1bcc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1bcc06cc50 0x7f1bcc06f110 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0004af0 tx=0x7f1bd003d040 comp rx=0 tx=0).stop 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0103050 msgr2=0x7f1be0195be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0103050 0x7f1be0195be0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1bc800e9a0 tx=0x7f1bc800ee60 comp rx=0 tx=0).stop 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 shutdown_connections 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f1bcc06cc50 0x7f1bcc06f110 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1be0103050 0x7f1be0195be0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:52.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 --2- 172.21.15.31:0/785193002 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be0100720 0x7f1be01956a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 >> 172.21.15.31:0/785193002 conn(0x7f1be00fa280 msgr2=0x7f1be00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 shutdown_connections 2024-09-16T08:54:52.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:52.388+0000 7f1be5a14640 1 -- 172.21.15.31:0/785193002 wait complete. 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:boot 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"}]: dispatch 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:52.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: pgmap v206: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-16T08:54:52.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:52 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/785193002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-09-16T08:54:52.688 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:boot 2024-09-16T08:54:52.688 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"}]: dispatch 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: pgmap v206: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-09-16T08:54:52.689 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:52 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/785193002' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-09-16T08:54:53.093 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T08:54:53.680 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:54:53.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T08:54:53.706 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/785193002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-09-16T08:54:53.706 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:53.706 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: Dropping low affinity active daemon mds.cephfs.smithi031.lcfutm in favor of higher affinity standby. 2024-09-16T08:54:53.706 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: Replacing daemon mds.cephfs.smithi031.lcfutm as rank 0 with standby daemon mds.cephfs.smithi031.mjhxgi 2024-09-16T08:54:53.706 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T08:54:53.707 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T08:54:53.707 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:standby 2024-09-16T08:54:53.707 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:replay,1=cephfs.smithi097.cagmus=up:active} 1 up:standby 2024-09-16T08:54:53.707 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:53.707 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:53.707 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:53.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/785193002' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-09-16T08:54:53.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:53.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: Dropping low affinity active daemon mds.cephfs.smithi031.lcfutm in favor of higher affinity standby. 2024-09-16T08:54:53.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: Replacing daemon mds.cephfs.smithi031.lcfutm as rank 0 with standby daemon mds.cephfs.smithi031.mjhxgi 2024-09-16T08:54:53.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T08:54:53.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: osdmap e47: 6 total, 6 up, 6 in 2024-09-16T08:54:53.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:standby 2024-09-16T08:54:53.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:replay,1=cephfs.smithi097.cagmus=up:active} 1 up:standby 2024-09-16T08:54:53.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:53.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:54:53.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:54.443 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:54.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:54.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:54.700 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:resolve 2024-09-16T08:54:54.701 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] up:boot 2024-09-16T08:54:54.701 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] up:active 2024-09-16T08:54:54.701 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:resolve,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:54.701 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:54:54.701 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:54 smithi031 ceph-mon[27844]: pgmap v208: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-16T08:54:54.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:54.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:54:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:resolve 2024-09-16T08:54:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] up:boot 2024-09-16T08:54:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] up:active 2024-09-16T08:54:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:resolve,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:54:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:54 smithi097 ceph-mon[35478]: pgmap v208: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-16T08:54:56.216 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:55 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:reconnect 2024-09-16T08:54:56.217 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:55 smithi097 ceph-mon[35478]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:reconnect,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:56.288 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:55 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:reconnect 2024-09-16T08:54:56.288 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:55 smithi031 ceph-mon[27844]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:reconnect,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:56 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:rejoin 2024-09-16T08:54:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:56 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:standby 2024-09-16T08:54:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:56 smithi097 ceph-mon[35478]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:rejoin,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:56 smithi097 ceph-mon[35478]: daemon mds.cephfs.smithi031.mjhxgi is now active in filesystem cephfs as rank 0 2024-09-16T08:54:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:56 smithi097 ceph-mon[35478]: pgmap v209: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-16T08:54:57.320 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:56 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:rejoin 2024-09-16T08:54:57.321 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:56 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:standby 2024-09-16T08:54:57.321 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:56 smithi031 ceph-mon[27844]: fsmap cephfs:2/2 {0=cephfs.smithi031.mjhxgi=up:rejoin,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:57.321 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:56 smithi031 ceph-mon[27844]: daemon mds.cephfs.smithi031.mjhxgi is now active in filesystem cephfs as rank 0 2024-09-16T08:54:57.321 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:56 smithi031 ceph-mon[27844]: pgmap v209: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 -- 172.21.15.31:0/1143710883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 msgr2=0x7f6e70101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/1143710883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70101210 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009880 tx=0x7f6e5802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 -- 172.21.15.31:0/1143710883 shutdown_connections 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/1143710883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70101210 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/1143710883 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e700ffb90 0x7f6e700fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 -- 172.21.15.31:0/1143710883 >> 172.21.15.31:0/1143710883 conn(0x7f6e700fb360 msgr2=0x7f6e700fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 -- 172.21.15.31:0/1143710883 shutdown_connections 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 -- 172.21.15.31:0/1143710883 wait complete. 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 Processor -- start 2024-09-16T08:54:57.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.405+0000 7f6e75bc5640 1 -- start start 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e75bc5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e700ffb90 0x7f6e70195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e75bc5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e75bc5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e70196150 con 0x7f6e70100d90 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e75bc5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e701962c0 con 0x7f6e700ffb90 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e66ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e66ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48742/0 (socket says 172.21.15.31:48742) 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e66ffd640 1 -- 172.21.15.31:0/3058686232 learned_addr learned my addr 172.21.15.31:0/3058686232 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e6f7fe640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e700ffb90 0x7f6e70195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e66ffd640 1 -- 172.21.15.31:0/3058686232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e700ffb90 msgr2=0x7f6e70195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e66ffd640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e700ffb90 0x7f6e70195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e66ffd640 1 -- 172.21.15.31:0/3058686232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e58009560 con 0x7f6e70100d90 2024-09-16T08:54:57.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e6f7fe640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e700ffb90 0x7f6e70195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e66ffd640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70195b80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6e58005520 tx=0x7f6e58005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.406+0000 7f6e6d7fa640 1 -- 172.21.15.31:0/3058686232 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e58004aa0 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.407+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7019acb0 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.407+0000 7f6e6d7fa640 1 -- 172.21.15.31:0/3058686232 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e58004c00 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.407+0000 7f6e6d7fa640 1 -- 172.21.15.31:0/3058686232 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e58003b10 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.407+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7019b220 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.408+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7018f710 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.408+0000 7f6e6d7fa640 1 -- 172.21.15.31:0/3058686232 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6e58031d20 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.408+0000 7f6e6d7fa640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e5c06cc50 0x7f6e5c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.409+0000 7f6e6d7fa640 1 -- 172.21.15.31:0/3058686232 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f6e580ad4b0 con 0x7f6e70100d90 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.409+0000 7f6e6f7fe640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e5c06cc50 0x7f6e5c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.409+0000 7f6e6f7fe640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e5c06cc50 0x7f6e5c06f110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e600058c0 tx=0x7f6e60005850 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:54:57.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.413+0000 7f6e6d7fa640 1 -- 172.21.15.31:0/3058686232 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6e58077820 con 0x7f6e70100d90 2024-09-16T08:54:57.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.626+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6e7002c480 con 0x7f6e70100d90 2024-09-16T08:54:57.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.627+0000 7f6e6d7fa640 1 -- 172.21.15.31:0/3058686232 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f6e5807ae40 con 0x7f6e70100d90 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:54:57.628 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:54:57.629 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:54:57.630 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:57.631 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:57.631 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:54:57.631 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:54:57.631 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:54:57.631 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:54:57.631 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e5c06cc50 msgr2=0x7f6e5c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e5c06cc50 0x7f6e5c06f110 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e600058c0 tx=0x7f6e60005850 comp rx=0 tx=0).stop 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 msgr2=0x7f6e70195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70195b80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6e58005520 tx=0x7f6e58005550 comp rx=0 tx=0).stop 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 shutdown_connections 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f6e5c06cc50 0x7f6e5c06f110 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e70100d90 0x7f6e70195b80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 --2- 172.21.15.31:0/3058686232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e700ffb90 0x7f6e70195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 >> 172.21.15.31:0/3058686232 conn(0x7f6e700fb360 msgr2=0x7f6e701044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 shutdown_connections 2024-09-16T08:54:57.632 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:54:57.629+0000 7f6e75bc5640 1 -- 172.21.15.31:0/3058686232 wait complete. 2024-09-16T08:54:58.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:57 smithi031 ceph-mon[27844]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T08:54:58.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:57 smithi031 ceph-mon[27844]: Cluster is now healthy 2024-09-16T08:54:58.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:57 smithi031 ceph-mon[27844]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:active 2024-09-16T08:54:58.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:57 smithi031 ceph-mon[27844]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:58.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:57 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3058686232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:54:58.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:57 smithi097 ceph-mon[35478]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T08:54:58.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:57 smithi097 ceph-mon[35478]: Cluster is now healthy 2024-09-16T08:54:58.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:57 smithi097 ceph-mon[35478]: mds.? [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] up:active 2024-09-16T08:54:58.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:57 smithi097 ceph-mon[35478]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:54:58.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:57 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3058686232' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:54:58.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-09-16T08:54:59.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:54:58 smithi031 ceph-mon[27844]: pgmap v210: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 3.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T08:54:59.126 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:54:59.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:54:58 smithi097 ceph-mon[35478]: pgmap v210: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 3.6 KiB/s rd, 0 B/s wr, 4 op/s 2024-09-16T08:55:00.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:00 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:00.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:00 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:00.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:00 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:00.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:00 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:00.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:00 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:00.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:00 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:55:00.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:00 smithi097 ceph-mon[35478]: pgmap v211: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:55:00.557 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:00 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:00.558 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:00 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:00.558 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:00 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:00.558 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:00 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:00.558 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:00 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:00.558 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:00 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:55:00.558 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:00 smithi031 ceph-mon[27844]: pgmap v211: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:55:00.864 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 -- 172.21.15.31:0/3611093267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 msgr2=0x7f5e20101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 --2- 172.21.15.31:0/3611093267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20101210 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009880 tx=0x7f5e1402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 -- 172.21.15.31:0/3611093267 shutdown_connections 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 --2- 172.21.15.31:0/3611093267 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20101210 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 --2- 172.21.15.31:0/3611093267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e200ffb90 0x7f5e200fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 -- 172.21.15.31:0/3611093267 >> 172.21.15.31:0/3611093267 conn(0x7f5e200fb320 msgr2=0x7f5e200fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 -- 172.21.15.31:0/3611093267 shutdown_connections 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.858+0000 7f5e26496640 1 -- 172.21.15.31:0/3611093267 wait complete. 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.859+0000 7f5e26496640 1 Processor -- start 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.859+0000 7f5e26496640 1 -- start start 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.859+0000 7f5e26496640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e200ffb90 0x7f5e20195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:00.865 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.859+0000 7f5e26496640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.859+0000 7f5e26496640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e20196140 con 0x7f5e20100d90 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.859+0000 7f5e26496640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e201962b0 con 0x7f5e200ffb90 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20195b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48750/0 (socket says 172.21.15.31:48750) 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1f7fe640 1 -- 172.21.15.31:0/642039772 learned_addr learned my addr 172.21.15.31:0/642039772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1ffff640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e200ffb90 0x7f5e20195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1f7fe640 1 -- 172.21.15.31:0/642039772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e200ffb90 msgr2=0x7f5e20195630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1f7fe640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e200ffb90 0x7f5e20195630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1f7fe640 1 -- 172.21.15.31:0/642039772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e14009560 con 0x7f5e20100d90 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1f7fe640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20195b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5e140055a0 tx=0x7f5e140055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:00.866 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1d7fa640 1 -- 172.21.15.31:0/642039772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e14004aa0 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2019acf0 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1d7fa640 1 -- 172.21.15.31:0/642039772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e14004c00 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e1d7fa640 1 -- 172.21.15.31:0/642039772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e14003b10 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.860+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2019b260 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.861+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2018f730 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.862+0000 7f5e1d7fa640 1 -- 172.21.15.31:0/642039772 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5e14031d20 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.862+0000 7f5e1d7fa640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cc50 0x7f5e0806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.862+0000 7f5e1d7fa640 1 -- 172.21.15.31:0/642039772 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5e140ad340 con 0x7f5e20100d90 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.862+0000 7f5e1ffff640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cc50 0x7f5e0806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:00.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.863+0000 7f5e1ffff640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cc50 0x7f5e0806f110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e20100bf0 tx=0x7f5e10005f90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:00.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:00.867+0000 7f5e1d7fa640 1 -- 172.21.15.31:0/642039772 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5e14077820 con 0x7f5e20100d90 2024-09-16T08:55:01.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.074+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e201087a0 con 0x7f5e20100d90 2024-09-16T08:55:01.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.075+0000 7f5e1d7fa640 1 -- 172.21.15.31:0/642039772 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4831 (secure 0 0 0) 0x7f5e1407ae40 con 0x7f5e20100d90 2024-09-16T08:55:01.076 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:55:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cc50 msgr2=0x7f5e0806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cc50 0x7f5e0806f110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e20100bf0 tx=0x7f5e10005f90 comp rx=0 tx=0).stop 2024-09-16T08:55:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 msgr2=0x7f5e20195b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20195b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5e140055a0 tx=0x7f5e140055d0 comp rx=0 tx=0).stop 2024-09-16T08:55:01.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 shutdown_connections 2024-09-16T08:55:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5e0806cc50 0x7f5e0806f110 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e20100d90 0x7f5e20195b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 --2- 172.21.15.31:0/642039772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e200ffb90 0x7f5e20195630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 >> 172.21.15.31:0/642039772 conn(0x7f5e200fb320 msgr2=0x7f5e20104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 shutdown_connections 2024-09-16T08:55:01.080 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:01.078+0000 7f5e26496640 1 -- 172.21.15.31:0/642039772 wait complete. 2024-09-16T08:55:01.091 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:55:01.471 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:01 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/642039772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T08:55:01.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:01 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/642039772' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T08:55:01.886 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-09-16T08:55:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:02 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:02 smithi031 ceph-mon[27844]: pgmap v212: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:55:02.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:02.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:02 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:02.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:02 smithi097 ceph-mon[35478]: pgmap v212: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 -- 172.21.15.31:0/3190971577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 msgr2=0x7fcef0100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 --2- 172.21.15.31:0/3190971577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef0100170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fced8009880 tx=0x7fced802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 -- 172.21.15.31:0/3190971577 shutdown_connections 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 --2- 172.21.15.31:0/3190971577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef0100170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 --2- 172.21.15.31:0/3190971577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcef00feaf0 0x7fcef00feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 -- 172.21.15.31:0/3190971577 >> 172.21.15.31:0/3190971577 conn(0x7fcef00fa280 msgr2=0x7fcef00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 -- 172.21.15.31:0/3190971577 shutdown_connections 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.380+0000 7fcef698d640 1 -- 172.21.15.31:0/3190971577 wait complete. 2024-09-16T08:55:04.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcef698d640 1 Processor -- start 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcef698d640 1 -- start start 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcef698d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcef00feaf0 0x7fcef0068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcef698d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcef698d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef01962c0 con 0x7fcef00ffcf0 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcef698d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef019cd90 con 0x7fcef00feaf0 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcee7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41114/0 (socket says 172.21.15.31:41114) 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcee7fff640 1 -- 172.21.15.31:0/521021925 learned_addr learned my addr 172.21.15.31:0/521021925 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fceeffff640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcef00feaf0 0x7fcef0068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcee7fff640 1 -- 172.21.15.31:0/521021925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcef00feaf0 msgr2=0x7fcef0068790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:04.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcee7fff640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcef00feaf0 0x7fcef0068790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.381+0000 7fcee7fff640 1 -- 172.21.15.31:0/521021925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced8009560 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.382+0000 7fcee7fff640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef019c850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fced8006010 tx=0x7fced80055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.382+0000 7fceedffb640 1 -- 172.21.15.31:0/521021925 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced8004aa0 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.382+0000 7fceedffb640 1 -- 172.21.15.31:0/521021925 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fced8004c00 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.382+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef019cfc0 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.382+0000 7fceedffb640 1 -- 172.21.15.31:0/521021925 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced8003a80 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.382+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef019d420 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.383+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef018e690 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.383+0000 7fceedffb640 1 -- 172.21.15.31:0/521021925 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fced8031c90 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.384+0000 7fceedffb640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fcedc06cc50 0x7fcedc06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.384+0000 7fceedffb640 1 -- 172.21.15.31:0/521021925 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fced80ad3b0 con 0x7fcef00ffcf0 2024-09-16T08:55:04.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.384+0000 7fceeffff640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fcedc06cc50 0x7fcedc06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.384+0000 7fceeffff640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fcedc06cc50 0x7fcedc06f110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005fd0 tx=0x7fcee000b390 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:04.389 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.388+0000 7fceedffb640 1 -- 172.21.15.31:0/521021925 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fced8077850 con 0x7fcef00ffcf0 2024-09-16T08:55:04.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.633+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fcef002c480 con 0x7fcef00ffcf0 2024-09-16T08:55:04.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.633+0000 7fceedffb640 1 -- 172.21.15.31:0/521021925 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+97 (secure 0 0 0) 0x7fced807ae70 con 0x7fcef00ffcf0 2024-09-16T08:55:04.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fcedc06cc50 msgr2=0x7fcedc06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:04.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fcedc06cc50 0x7fcedc06f110 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcee0005fd0 tx=0x7fcee000b390 comp rx=0 tx=0).stop 2024-09-16T08:55:04.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 msgr2=0x7fcef019c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:04.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef019c850 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fced8006010 tx=0x7fced80055a0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 shutdown_connections 2024-09-16T08:55:04.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fcedc06cc50 0x7fcedc06f110 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcef00ffcf0 0x7fcef019c850 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 --2- 172.21.15.31:0/521021925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcef00feaf0 0x7fcef0068790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:04.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.635+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 >> 172.21.15.31:0/521021925 conn(0x7fcef00fa280 msgr2=0x7fcef01033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:04.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.636+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 shutdown_connections 2024-09-16T08:55:04.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:04.636+0000 7fcef698d640 1 -- 172.21.15.31:0/521021925 wait complete. 2024-09-16T08:55:04.645 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:55:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:05 smithi031 ceph-mon[27844]: pgmap v213: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:55:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:05 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/521021925' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-16T08:55:05.511 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-09-16T08:55:05.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T08:55:05.525 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:05 smithi097 ceph-mon[35478]: pgmap v213: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 7.3 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-16T08:55:05.526 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:05 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/521021925' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-16T08:55:06.246 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:07 smithi031 ceph-mon[27844]: pgmap v214: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:55:07.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:07 smithi097 ceph-mon[35478]: pgmap v214: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:55:08.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 -- 172.21.15.31:0/2179149237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 msgr2=0x7f5ae8104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:08.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/2179149237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8104e10 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8009880 tx=0x7f5ad802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:08.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 -- 172.21.15.31:0/2179149237 shutdown_connections 2024-09-16T08:55:08.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/2179149237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80fea30 0x7f5ae80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:08.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/2179149237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8104e10 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:08.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 -- 172.21.15.31:0/2179149237 >> 172.21.15.31:0/2179149237 conn(0x7f5ae80fa730 msgr2=0x7f5ae80fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:08.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 -- 172.21.15.31:0/2179149237 shutdown_connections 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 -- 172.21.15.31:0/2179149237 wait complete. 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 Processor -- start 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.511+0000 7f5aeed3a640 1 -- start start 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5aeed3a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80fea30 0x7f5ae8105960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5aeed3a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5aeed3a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae81063e0 con 0x7f5ae8104a30 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5aeed3a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8106520 con 0x7f5ae80fea30 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41130/0 (socket says 172.21.15.31:41130) 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae7fff640 1 -- 172.21.15.31:0/785957 learned_addr learned my addr 172.21.15.31:0/785957 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5aecad6640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80fea30 0x7f5ae8105960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae7fff640 1 -- 172.21.15.31:0/785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80fea30 msgr2=0x7f5ae8105960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:08.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae7fff640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80fea30 0x7f5ae8105960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae7fff640 1 -- 172.21.15.31:0/785957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad8009560 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5aecad6640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80fea30 0x7f5ae8105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae7fff640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8105ea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c8a0 tx=0x7f5adc00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae5ffb640 1 -- 172.21.15.31:0/785957 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5adc007b80 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae81a1c30 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae5ffb640 1 -- 172.21.15.31:0/785957 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5adc007ce0 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.512+0000 7f5ae5ffb640 1 -- 172.21.15.31:0/785957 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5adc018810 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.513+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81a1f90 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.514+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8100170 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.514+0000 7f5ae5ffb640 1 -- 172.21.15.31:0/785957 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5adc018a80 con 0x7f5ae8104a30 2024-09-16T08:55:08.534 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.515+0000 7f5ae5ffb640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5ad006cc50 0x7f5ad006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:08.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.515+0000 7f5ae5ffb640 1 -- 172.21.15.31:0/785957 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5adc08c500 con 0x7f5ae8104a30 2024-09-16T08:55:08.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.515+0000 7f5aecad6640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5ad006cc50 0x7f5ad006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:08.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.515+0000 7f5aecad6640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5ad006cc50 0x7f5ad006f110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad8005a20 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:08.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.519+0000 7f5ae5ffb640 1 -- 172.21.15.31:0/785957 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5adc056870 con 0x7f5ae8104a30 2024-09-16T08:55:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.726+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ae81066e0 con 0x7f5ae8104a30 2024-09-16T08:55:08.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.727+0000 7f5ae5ffb640 1 -- 172.21.15.31:0/785957 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4831 (secure 0 0 0) 0x7f5adc059e90 con 0x7f5ae8104a30 2024-09-16T08:55:08.728 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:55:08.728 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14560,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2549073061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2549073061},{"type":"v1","addr":"172.21.15.31:6827","nonce":2549073061}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T08:54:56.899545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14548,"mds_1":24325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14548":{"gid":14548,"name":"cephfs.smithi031.mjhxgi","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6829/1804644990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1804644990},{"type":"v1","addr":"172.21.15.31:6829","nonce":1804644990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24325":{"gid":24325,"name":"cephfs.smithi097.cagmus","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6825/17261087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":17261087},{"type":"v1","addr":"172.21.15.97:6825","nonce":17261087}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-16T08:55:08.728 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5ad006cc50 msgr2=0x7f5ad006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5ad006cc50 0x7f5ad006f110 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad8005a20 comp rx=0 tx=0).stop 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 msgr2=0x7f5ae8105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8105ea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c8a0 tx=0x7f5adc00cd60 comp rx=0 tx=0).stop 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 shutdown_connections 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5ad006cc50 0x7f5ad006f110 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ae8104a30 0x7f5ae8105ea0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 --2- 172.21.15.31:0/785957 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae80fea30 0x7f5ae8105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 >> 172.21.15.31:0/785957 conn(0x7f5ae80fa730 msgr2=0x7f5ae8068770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 shutdown_connections 2024-09-16T08:55:08.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:08.729+0000 7f5aeed3a640 1 -- 172.21.15.31:0/785957 wait complete. 2024-09-16T08:55:09.314 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:09 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:09.314 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:09 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:09.314 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:09 smithi097 ceph-mon[35478]: pgmap v215: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:55:09.314 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:09 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/785957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T08:55:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:09 smithi031 ceph-mon[27844]: pgmap v215: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-16T08:55:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:09 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/785957' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T08:55:09.619 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-09-16T08:55:09.619 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T08:55:09.653 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T08:55:09.653 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-09-16T08:55:09.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:09.653 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T08:55:09.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:09.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-16T08:55:09.745 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2024-09-16T08:55:09.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:55:09.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:09.748 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-16T08:55:09.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:09.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-09-16T08:55:09.841 INFO:teuthology.orchestra.run.smithi097.stderr:Cannot find device "ceph-brx" 2024-09-16T08:55:09.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T08:55:09.843 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-16T08:55:09.843 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T08:55:09.843 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:09.844 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T08:55:10.062 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:11.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:11 smithi097 ceph-mon[35478]: pgmap v216: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 3.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:55:11.395 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:11 smithi031 ceph-mon[27844]: pgmap v216: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 3.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-16T08:55:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.710+0000 7fb0554e2640 1 -- 172.21.15.31:0/1074053924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 msgr2=0x7fb050100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.710+0000 7fb0554e2640 1 --2- 172.21.15.31:0/1074053924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb050100180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb040009880 tx=0x7fb04002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.710+0000 7fb0554e2640 1 -- 172.21.15.31:0/1074053924 shutdown_connections 2024-09-16T08:55:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.710+0000 7fb0554e2640 1 --2- 172.21.15.31:0/1074053924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050100750 0x7fb0501047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.710+0000 7fb0554e2640 1 --2- 172.21.15.31:0/1074053924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb050100180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.710+0000 7fb0554e2640 1 -- 172.21.15.31:0/1074053924 >> 172.21.15.31:0/1074053924 conn(0x7fb0500fb840 msgr2=0x7fb0500fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:11.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.711+0000 7fb0554e2640 1 -- 172.21.15.31:0/1074053924 shutdown_connections 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.711+0000 7fb0554e2640 1 -- 172.21.15.31:0/1074053924 wait complete. 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.711+0000 7fb0554e2640 1 Processor -- start 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.711+0000 7fb0554e2640 1 -- start start 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb0554e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb05019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb0554e2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050100750 0x7fb05019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb0554e2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb050195a10 con 0x7fb0500ffda0 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb0554e2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb050195b50 con 0x7fb050100750 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb05019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb05019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39534/0 (socket says 172.21.15.31:39534) 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04effd640 1 -- 172.21.15.31:0/2277922556 learned_addr learned my addr 172.21.15.31:0/2277922556 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb0467fc640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050100750 0x7fb05019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:11.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04effd640 1 -- 172.21.15.31:0/2277922556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050100750 msgr2=0x7fb05019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04effd640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050100750 0x7fb05019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04effd640 1 -- 172.21.15.31:0/2277922556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040009560 con 0x7fb0500ffda0 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04effd640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb05019b9b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb04002a9c0 tx=0x7fb0400055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04cff9640 1 -- 172.21.15.31:0/2277922556 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0400048d0 con 0x7fb0500ffda0 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04cff9640 1 -- 172.21.15.31:0/2277922556 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb040031640 con 0x7fb0500ffda0 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb050195dd0 con 0x7fb0500ffda0 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb04cff9640 1 -- 172.21.15.31:0/2277922556 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0400038d0 con 0x7fb0500ffda0 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.712+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0501962c0 con 0x7fb0500ffda0 2024-09-16T08:55:11.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.714+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb050101280 con 0x7fb0500ffda0 2024-09-16T08:55:11.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.714+0000 7fb04cff9640 1 -- 172.21.15.31:0/2277922556 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb040003b90 con 0x7fb0500ffda0 2024-09-16T08:55:11.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.715+0000 7fb04cff9640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb03c06cc50 0x7fb03c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:11.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.715+0000 7fb04cff9640 1 -- 172.21.15.31:0/2277922556 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb0400ad1d0 con 0x7fb0500ffda0 2024-09-16T08:55:11.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.715+0000 7fb0467fc640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb03c06cc50 0x7fb03c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:11.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.715+0000 7fb0467fc640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb03c06cc50 0x7fb03c06f110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb050197260 tx=0x7fb038005f90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:11.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.719+0000 7fb04cff9640 1 -- 172.21.15.31:0/2277922556 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb0400775f0 con 0x7fb0500ffda0 2024-09-16T08:55:11.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.929+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb050196f40 con 0x7fb0500ffda0 2024-09-16T08:55:11.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.929+0000 7fb04cff9640 1 -- 172.21.15.31:0/2277922556 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7fb04007ac10 con 0x7fb0500ffda0 2024-09-16T08:55:11.930 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb03c06cc50 msgr2=0x7fb03c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb03c06cc50 0x7fb03c06f110 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb050197260 tx=0x7fb038005f90 comp rx=0 tx=0).stop 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 msgr2=0x7fb05019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb05019b9b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb04002a9c0 tx=0x7fb0400055a0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 shutdown_connections 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb03c06cc50 0x7fb03c06f110 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb050100750 0x7fb05019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 --2- 172.21.15.31:0/2277922556 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb0500ffda0 0x7fb05019b9b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:11.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 >> 172.21.15.31:0/2277922556 conn(0x7fb0500fb840 msgr2=0x7fb0500fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 shutdown_connections 2024-09-16T08:55:11.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:11.931+0000 7fb0554e2640 1 -- 172.21.15.31:0/2277922556 wait complete. 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T08:55:12.651 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T08:55:12.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:12.652 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T08:55:12.670 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f0 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f0 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42246sec preferred_lft 42246sec 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T08:55:12.671 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:12.672 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp7s0f1 2024-09-16T08:55:12.672 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T08:55:12.672 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:12.672 INFO:teuthology.orchestra.run.smithi031.stdout: altname enp1s0f1 2024-09-16T08:55:12.672 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T08:55:12.672 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:12.673 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T08:55:12.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2024-09-16T08:55:12.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2024-09-16T08:55:12.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2024-09-16T08:55:12.673 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T08:55:12.673 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T08:55:12.761 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:12.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:12.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:12.883 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T08:55:12.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:12.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:12.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:12.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:12.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:12 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:12.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:12 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2277922556' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T08:55:12.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:12 smithi031 ceph-mon[27844]: pgmap v217: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:55:12.910 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-09-16T08:55:12.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:12.911 DEBUG:teuthology.orchestra.run.smithi031:> route 2024-09-16T08:55:12.981 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2024-09-16T08:55:12.981 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T08:55:12.982 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T08:55:12.982 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T08:55:12.982 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T08:55:12.982 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:12.982 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T08:55:12.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T08:55:12.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T08:55:12.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T08:55:12.983 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T08:55:13.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:13.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:12 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:12 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2277922556' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T08:55:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:12 smithi097 ceph-mon[35478]: pgmap v217: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-16T08:55:13.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:13.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:13.284 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T08:55:13.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:13.300 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2024-09-16T08:55:13.355 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:13.356 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T08:55:13.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T08:55:13.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T08:55:13.356 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T08:55:13.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:13.468 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:13.475 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T08:55:13.475 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T08:55:13.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:13.611 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:13.618 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:13.618 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2024-09-16T08:55:13.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2024-09-16T08:55:13.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T08:55:13.618 DEBUG:teuthology.orchestra.run.smithi031:> ') 2024-09-16T08:55:13.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:13.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:13.698 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T08:55:13.698 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:13.713 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T08:55:13.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T08:55:13.714 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:13.768 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T08:55:13.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:13.769 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T08:55:13.879 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T08:55:13.879 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T08:55:13.879 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-09-16T08:55:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T08:55:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T08:55:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-09-16T08:55:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300" 2024-09-16T08:55:14.007 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T08:55:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-16T08:55:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-16T08:55:14.013 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2024-09-16T08:55:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300" 2024-09-16T08:55:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T08:55:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.97:6789" 2024-09-16T08:55:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.97:6789:/ 2024-09-16T08:55:14.014 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=1631ad7a-7408-11ef-bceb-c7b262605968" will pass to kernel 2024-09-16T08:55:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T08:55:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: You just mounted a file system that supports labels which does not 2024-09-16T08:55:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T08:55:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T08:55:14.015 INFO:teuthology.orchestra.run.smithi031.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T08:55:14.016 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T08:55:14.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:14.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:14.044 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T08:55:14.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T08:55:14.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:14.149 INFO:teuthology.orchestra.run.smithi031.stdout:client24365 2024-09-16T08:55:14.149 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:55:14.149 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T08:55:14.167 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T08:55:14.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T08:55:14.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:14.273 INFO:teuthology.orchestra.run.smithi031.stdout:client24365 2024-09-16T08:55:14.273 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T08:55:14.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T08:55:14.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:14.336 INFO:teuthology.orchestra.run.smithi031.stdout:client24365 2024-09-16T08:55:14.336 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1631ad7a-7408-11ef-bceb-c7b262605968.client24365/status') 2024-09-16T08:55:14.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:14.400 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.24365 (3)172.21.15.31:0/3242977461 2024-09-16T08:55:14.400 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2024-09-16T08:55:14.400 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2024-09-16T08:55:14.400 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2024-09-16T08:55:14.400 INFO:teuthology.orchestra.run.smithi031.stderr:71 bytes copied, 2.3923e-05 s, 3.0 MB/s 2024-09-16T08:55:14.401 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:14.407 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-16T08:55:14.407 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T08:55:14.408 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:14.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T08:55:14.613 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:15 smithi031 ceph-mon[27844]: pgmap v218: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-09-16T08:55:15.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:15 smithi097 ceph-mon[35478]: pgmap v218: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-09-16T08:55:16.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 -- 172.21.15.31:0/4257535030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e0100cb0 msgr2=0x7f97e0101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:16.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 --2- 172.21.15.31:0/4257535030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e0100cb0 0x7f97e0101090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009c20 tx=0x7f97d002aa00 comp rx=0 tx=0).stop 2024-09-16T08:55:16.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 -- 172.21.15.31:0/4257535030 shutdown_connections 2024-09-16T08:55:16.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 --2- 172.21.15.31:0/4257535030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97e01015d0 0x7f97e0109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 --2- 172.21.15.31:0/4257535030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e0100cb0 0x7f97e0101090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 -- 172.21.15.31:0/4257535030 >> 172.21.15.31:0/4257535030 conn(0x7f97e00758a0 msgr2=0x7f97e00fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 -- 172.21.15.31:0/4257535030 shutdown_connections 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 -- 172.21.15.31:0/4257535030 wait complete. 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.255+0000 7f97e5242640 1 Processor -- start 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97e5242640 1 -- start start 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97e5242640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97e0100cb0 0x7f97e019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97e5242640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e01015d0 0x7f97e019bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97e5242640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e019c3d0 con 0x7f97e01015d0 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97e5242640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e0195960 con 0x7f97e0100cb0 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97de59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e01015d0 0x7f97e019bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:16.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97de59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e01015d0 0x7f97e019bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39546/0 (socket says 172.21.15.31:39546) 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97de59c640 1 -- 172.21.15.31:0/1644351049 learned_addr learned my addr 172.21.15.31:0/1644351049 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97ded9d640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97e0100cb0 0x7f97e019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97de59c640 1 -- 172.21.15.31:0/1644351049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97e0100cb0 msgr2=0x7f97e019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97de59c640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97e0100cb0 0x7f97e019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.256+0000 7f97de59c640 1 -- 172.21.15.31:0/1644351049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009900 con 0x7f97e01015d0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.257+0000 7f97de59c640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e01015d0 0x7f97e019bdb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f97c800c8f0 tx=0x7f97c800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.257+0000 7f97d7fff640 1 -- 172.21.15.31:0/1644351049 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c8007b80 con 0x7f97e01015d0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.257+0000 7f97d7fff640 1 -- 172.21.15.31:0/1644351049 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97c800ae50 con 0x7f97e01015d0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.257+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e0195c40 con 0x7f97e01015d0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.257+0000 7f97d7fff640 1 -- 172.21.15.31:0/1644351049 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97c8004e50 con 0x7f97e01015d0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.257+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e0196190 con 0x7f97e01015d0 2024-09-16T08:55:16.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.258+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e0068510 con 0x7f97e01015d0 2024-09-16T08:55:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.258+0000 7f97d7fff640 1 -- 172.21.15.31:0/1644351049 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f97c8004ff0 con 0x7f97e01015d0 2024-09-16T08:55:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.259+0000 7f97d7fff640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f97cc06cc00 0x7f97cc06f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.259+0000 7f97d7fff640 1 -- 172.21.15.31:0/1644351049 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f97c808bfc0 con 0x7f97e01015d0 2024-09-16T08:55:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.259+0000 7f97ded9d640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f97cc06cc00 0x7f97cc06f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.259+0000 7f97ded9d640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f97cc06cc00 0x7f97cc06f0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000f80 tx=0x7f97d0003a50 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:16.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.263+0000 7f97d7fff640 1 -- 172.21.15.31:0/1644351049 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f97c80563e0 con 0x7f97e01015d0 2024-09-16T08:55:16.470 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.469+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f97e0104340 con 0x7f97e01015d0 2024-09-16T08:55:16.471 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.469+0000 7f97d7fff640 1 -- 172.21.15.31:0/1644351049 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f97c8059a00 con 0x7f97e01015d0 2024-09-16T08:55:16.471 INFO:teuthology.orchestra.run.smithi031.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f97cc06cc00 msgr2=0x7f97cc06f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f97cc06cc00 0x7f97cc06f0c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97d0000f80 tx=0x7f97d0003a50 comp rx=0 tx=0).stop 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e01015d0 msgr2=0x7f97e019bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e01015d0 0x7f97e019bdb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f97c800c8f0 tx=0x7f97c800cdb0 comp rx=0 tx=0).stop 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 shutdown_connections 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f97cc06cc00 0x7f97cc06f0c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97e01015d0 0x7f97e019bdb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 --2- 172.21.15.31:0/1644351049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f97e0100cb0 0x7f97e019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 >> 172.21.15.31:0/1644351049 conn(0x7f97e00758a0 msgr2=0x7f97e0101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 shutdown_connections 2024-09-16T08:55:16.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:16.471+0000 7f97e5242640 1 -- 172.21.15.31:0/1644351049 wait complete. 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi097.front.sepia.ceph.com 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T08:55:17.271 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-16T08:55:17.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:17.272 DEBUG:teuthology.orchestra.run.smithi097:> ip addr 2024-09-16T08:55:17.292 INFO:teuthology.orchestra.run.smithi097.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T08:55:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T08:55:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T08:55:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-16T08:55:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 ::1/128 scope host 2024-09-16T08:55:17.292 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0e brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f0 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:d8:a2:0f brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: altname enp7s0f1 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:04 brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: link/ether 0c:c4:7a:8f:d8:05 brd ff:ff:ff:ff:ff:ff 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: inet 172.21.15.97/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft 42244sec preferred_lft 42244sec 2024-09-16T08:55:17.293 INFO:teuthology.orchestra.run.smithi097.stdout: inet6 fe80::ec4:7aff:fe8f:d805/64 scope link noprefixroute 2024-09-16T08:55:17.294 INFO:teuthology.orchestra.run.smithi097.stdout: valid_lft forever preferred_lft forever 2024-09-16T08:55:17.294 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T08:55:17.294 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:17.294 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-16T08:55:17.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add name ceph-brx type bridge 2024-09-16T08:55:17.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr flush dev ceph-brx 2024-09-16T08:55:17.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set ceph-brx up 2024-09-16T08:55:17.294 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T08:55:17.294 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-16T08:55:17.379 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:17 smithi031 ceph-mon[27844]: pgmap v219: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:17.379 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:17 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1644351049' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T08:55:17.387 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:17.396 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:17 smithi097 ceph-mon[35478]: pgmap v219: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:17.396 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:17 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1644351049' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T08:55:17.512 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:17.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:17.519 DEBUG:teuthology.orchestra.run.smithi097:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T08:55:17.546 INFO:teuthology.orchestra.run.smithi097.stdout:1 2024-09-16T08:55:17.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:17.547 DEBUG:teuthology.orchestra.run.smithi097:> route 2024-09-16T08:55:17.619 INFO:teuthology.orchestra.run.smithi097.stdout:Kernel IP routing table 2024-09-16T08:55:17.620 INFO:teuthology.orchestra.run.smithi097.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T08:55:17.620 INFO:teuthology.orchestra.run.smithi097.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-16T08:55:17.620 INFO:teuthology.orchestra.run.smithi097.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-16T08:55:17.620 INFO:teuthology.orchestra.run.smithi097.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T08:55:17.620 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:17.620 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-16T08:55:17.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-16T08:55:17.620 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-16T08:55:17.621 DEBUG:teuthology.orchestra.run.smithi097:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-16T08:55:17.621 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-16T08:55:17.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:17.899 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:17.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:17.905 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-16T08:55:17.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:17.921 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list-id 2024-09-16T08:55:17.977 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:17.977 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-16T08:55:17.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T08:55:17.977 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-16T08:55:17.977 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-16T08:55:18.064 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:18.087 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:18.093 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-16T08:55:18.093 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:18.093 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-16T08:55:18.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-16T08:55:18.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T08:55:18.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-16T08:55:18.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-16T08:55:18.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-16T08:55:18.094 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-16T08:55:18.142 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:18.212 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:18.218 DEBUG:teuthology.orchestra.run.smithi097:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T08:55:18.218 DEBUG:teuthology.orchestra.run.smithi097:> set -e 2024-09-16T08:55:18.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set brx.0 up 2024-09-16T08:55:18.218 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T08:55:18.218 DEBUG:teuthology.orchestra.run.smithi097:> ') 2024-09-16T08:55:18.266 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:18.288 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:18.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T08:55:18.294 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:18.310 INFO:teuthology.orchestra.run.smithi097.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-16T08:55:18.310 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T08:55:18.311 DEBUG:teuthology.orchestra.run.smithi097:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:18.366 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-16T08:55:18.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:18.367 DEBUG:teuthology.orchestra.run.smithi097:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T08:55:18.468 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T08:55:18.468 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-16T08:55:18.469 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-16T08:55:18.594 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T08:55:18.594 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-16T08:55:18.595 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-16T08:55:18.595 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300" 2024-09-16T08:55:18.595 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T08:55:18.612 INFO:teuthology.orchestra.run.smithi097.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-16T08:55:18.612 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout:invalid new device string format 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:3300,172.21.15.97:3300" 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: resolved to: "172.21.15.31:6789,172.21.15.97:6789" 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.31:6789,172.21.15.97:6789:/ 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=1631ad7a-7408-11ef-bceb-c7b262605968" will pass to kernel 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-16T08:55:18.613 INFO:teuthology.orchestra.run.smithi097.stdout: You just mounted a file system that supports labels which does not 2024-09-16T08:55:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T08:55:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T08:55:18.614 INFO:teuthology.orchestra.run.smithi097.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T08:55:18.615 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T08:55:18.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T08:55:18.615 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:18.640 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T08:55:18.732 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T08:55:18.734 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:18.740 INFO:teuthology.orchestra.run.smithi097.stdout:client24369 2024-09-16T08:55:18.741 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:55:18.741 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T08:55:18.779 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T08:55:18.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T08:55:18.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:18.883 INFO:teuthology.orchestra.run.smithi097.stdout:client24369 2024-09-16T08:55:18.883 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T08:55:18.931 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T08:55:18.933 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:18.939 INFO:teuthology.orchestra.run.smithi097.stdout:client24369 2024-09-16T08:55:18.940 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/1631ad7a-7408-11ef-bceb-c7b262605968.client24369/status') 2024-09-16T08:55:18.988 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T08:55:19.001 INFO:teuthology.orchestra.run.smithi097.stdout:instance: client.24369 (3)192.168.0.1:0/2254071746 2024-09-16T08:55:19.001 INFO:teuthology.orchestra.run.smithi097.stdout:blocklisted: false 2024-09-16T08:55:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records in 2024-09-16T08:55:19.001 INFO:teuthology.orchestra.run.smithi097.stderr:0+1 records out 2024-09-16T08:55:19.002 INFO:teuthology.orchestra.run.smithi097.stderr:70 bytes copied, 2.1141e-05 s, 3.3 MB/s 2024-09-16T08:55:19.002 INFO:teuthology.orchestra.run.smithi097.stderr:2024-09-16T08:55:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T08:55:19.005 INFO:teuthology.run_tasks:Running task print... 2024-09-16T08:55:19.613 INFO:teuthology.task.print:**** done client 2024-09-16T08:55:19.614 INFO:teuthology.run_tasks:Running task parallel... 2024-09-16T08:55:19.663 INFO:teuthology.task.parallel:starting parallel... 2024-09-16T08:55:19.663 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-16T08:55:19.664 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T08:55:19.664 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:55:19.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-09-16T08:55:19.665 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-16T08:55:19.665 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-09-16T08:55:19.671 INFO:tasks.workunit:Pulling workunits from ref 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:19.672 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T08:55:19.672 INFO:tasks.workunit:timeout=3h 2024-09-16T08:55:19.672 INFO:tasks.workunit:cleanup=True 2024-09-16T08:55:19.672 DEBUG:teuthology.orchestra.run.smithi031:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:19.673 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:19 smithi031 ceph-mon[27844]: pgmap v220: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:55:19.673 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:19 smithi097 ceph-mon[35478]: pgmap v220: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:55:19.694 INFO:teuthology.orchestra.run.smithi031.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:19.695 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-16T08:55:19.695 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 84h/132d Inode: 1 Links: 2 2024-09-16T08:55:19.695 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:55:19.695 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-16T08:55:19.695 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-16T08:55:19.696 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-16 08:54:42.319402232 +0000 2024-09-16T08:55:19.696 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-16 08:55:14.039993263 +0000 2024-09-16T08:55:19.696 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-16 08:54:42.319402232 +0000 2024-09-16T08:55:19.696 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T08:55:19.696 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T08:55:19.769 DEBUG:teuthology.orchestra.run.smithi097:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:19.790 INFO:teuthology.orchestra.run.smithi097.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:19.790 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-09-16T08:55:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 81h/129d Inode: 1 Links: 3 2024-09-16T08:55:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T08:55:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-16T08:55:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-16T08:55:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-09-16 08:55:19.761897454 +0000 2024-09-16T08:55:19.791 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-09-16 08:55:19.761897454 +0000 2024-09-16T08:55:19.792 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-09-16 08:54:42.319402232 +0000 2024-09-16T08:55:19.792 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-16T08:55:19.792 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-16T08:55:19.864 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:19.865 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:19.882 INFO:tasks.workunit.client.0.smithi031.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T08:55:19.901 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:19.924 INFO:tasks.workunit.client.1.smithi097.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-16T08:55:20.366 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:20 smithi031 ceph-mon[27844]: pgmap v221: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:55:20.476 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:20 smithi097 ceph-mon[35478]: pgmap v221: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:55:21.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 -- 172.21.15.31:0/3311764091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fee10 msgr2=0x7fad740ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 --2- 172.21.15.31:0/3311764091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fee10 0x7fad740ff270 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009880 tx=0x7fad5c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 -- 172.21.15.31:0/3311764091 shutdown_connections 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 --2- 172.21.15.31:0/3311764091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fee10 0x7fad740ff270 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 --2- 172.21.15.31:0/3311764091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad740fe4d0 0x7fad740fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 -- 172.21.15.31:0/3311764091 >> 172.21.15.31:0/3311764091 conn(0x7fad740fa280 msgr2=0x7fad740fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 -- 172.21.15.31:0/3311764091 shutdown_connections 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.576+0000 7fad7975c640 1 -- 172.21.15.31:0/3311764091 wait complete. 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad7975c640 1 Processor -- start 2024-09-16T08:55:21.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad7975c640 1 -- start start 2024-09-16T08:55:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad7975c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fe4d0 0x7fad741978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad7975c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad740fee10 0x7fad74197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad7975c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad741983c0 con 0x7fad740fe4d0 2024-09-16T08:55:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad7975c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad74198530 con 0x7fad740fee10 2024-09-16T08:55:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad72ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fe4d0 0x7fad741978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:21.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad72ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fe4d0 0x7fad741978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39400/0 (socket says 172.21.15.31:39400) 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad72ffd640 1 -- 172.21.15.31:0/2033092520 learned_addr learned my addr 172.21.15.31:0/2033092520 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.577+0000 7fad727fc640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad740fee10 0x7fad74197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad72ffd640 1 -- 172.21.15.31:0/2033092520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad740fee10 msgr2=0x7fad74197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad72ffd640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad740fee10 0x7fad74197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad72ffd640 1 -- 172.21.15.31:0/2033092520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c009560 con 0x7fad740fe4d0 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad727fc640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad740fee10 0x7fad74197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad72ffd640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fe4d0 0x7fad741978b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad6400cc90 tx=0x7fad6400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad6bfff640 1 -- 172.21.15.31:0/2033092520 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad64004b20 con 0x7fad740fe4d0 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad6bfff640 1 -- 172.21.15.31:0/2033092520 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad64004c80 con 0x7fad740fe4d0 2024-09-16T08:55:21.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad6bfff640 1 -- 172.21.15.31:0/2033092520 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad6400f660 con 0x7fad740fe4d0 2024-09-16T08:55:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7419cf80 con 0x7fad740fe4d0 2024-09-16T08:55:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.578+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7419d470 con 0x7fad740fe4d0 2024-09-16T08:55:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.579+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad74191950 con 0x7fad740fe4d0 2024-09-16T08:55:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.579+0000 7fad6bfff640 1 -- 172.21.15.31:0/2033092520 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fad6400f920 con 0x7fad740fe4d0 2024-09-16T08:55:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.580+0000 7fad6bfff640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fad6006cc50 0x7fad6006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.580+0000 7fad6bfff640 1 -- 172.21.15.31:0/2033092520 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fad6408c4d0 con 0x7fad740fe4d0 2024-09-16T08:55:21.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.580+0000 7fad727fc640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fad6006cc50 0x7fad6006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:21.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.581+0000 7fad727fc640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fad6006cc50 0x7fad6006f110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad5c031540 tx=0x7fad5c005de0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:21.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.584+0000 7fad6bfff640 1 -- 172.21.15.31:0/2033092520 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fad64056840 con 0x7fad740fe4d0 2024-09-16T08:55:21.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.762+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fad7410a9c0 con 0x7fad740fe4d0 2024-09-16T08:55:21.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.766+0000 7fad6bfff640 1 -- 172.21.15.31:0/2033092520 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v42)=0 v42) v1 ==== 126+0+0 (secure 0 0 0) 0x7fad64059e60 con 0x7fad740fe4d0 2024-09-16T08:55:21.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fad6006cc50 msgr2=0x7fad6006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:21.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fad6006cc50 0x7fad6006f110 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fad5c031540 tx=0x7fad5c005de0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fe4d0 msgr2=0x7fad741978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:21.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fe4d0 0x7fad741978b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fad6400cc90 tx=0x7fad6400a620 comp rx=0 tx=0).stop 2024-09-16T08:55:21.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 shutdown_connections 2024-09-16T08:55:21.772 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fad6006cc50 0x7fad6006f110 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fad740fee10 0x7fad74197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 --2- 172.21.15.31:0/2033092520 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad740fe4d0 0x7fad741978b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:21.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 >> 172.21.15.31:0/2033092520 conn(0x7fad740fa280 msgr2=0x7fad74106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:21.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 shutdown_connections 2024-09-16T08:55:21.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:21.769+0000 7fad7975c640 1 -- 172.21.15.31:0/2033092520 wait complete. 2024-09-16T08:55:22.484 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T08:55:22.485 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:55:22.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-16T08:55:22.738 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:23.099 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:22 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2033092520' entity='client.admin' 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:22 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:22 smithi031 ceph-mon[27844]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:22 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:22 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:23.100 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:22 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:23.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:22 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2033092520' entity='client.admin' 2024-09-16T08:55:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:22 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:55:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:22 smithi097 ceph-mon[35478]: pgmap v222: 65 pgs: 65 active+clean; 580 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-09-16T08:55:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:22 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:22 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:22 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:24.089 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:23 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- 172.21.15.31:0/3320838431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 msgr2=0x7f7ca80feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 --2- 172.21.15.31:0/3320838431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca80feef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c009880 tx=0x7f7c9c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:24.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- 172.21.15.31:0/3320838431 shutdown_connections 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 --2- 172.21.15.31:0/3320838431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca80ffcf0 0x7f7ca8100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 --2- 172.21.15.31:0/3320838431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca80feef0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- 172.21.15.31:0/3320838431 >> 172.21.15.31:0/3320838431 conn(0x7f7ca80fa280 msgr2=0x7f7ca80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- 172.21.15.31:0/3320838431 shutdown_connections 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- 172.21.15.31:0/3320838431 wait complete. 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 Processor -- start 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- start start 2024-09-16T08:55:24.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca8195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca80ffcf0 0x7f7ca8195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8196220 con 0x7f7ca80feaf0 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cb078d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca8196390 con 0x7f7ca80ffcf0 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cae529640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca8195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cae529640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca8195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39416/0 (socket says 172.21.15.31:39416) 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.418+0000 7f7cae529640 1 -- 172.21.15.31:0/647992614 learned_addr learned my addr 172.21.15.31:0/647992614 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7cadd28640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca80ffcf0 0x7f7ca8195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7cae529640 1 -- 172.21.15.31:0/647992614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca80ffcf0 msgr2=0x7f7ca8195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:24.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7cae529640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca80ffcf0 0x7f7ca8195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7cae529640 1 -- 172.21.15.31:0/647992614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c009560 con 0x7f7ca80feaf0 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7cae529640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca8195710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005610 tx=0x7f7c9c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7c9b7fe640 1 -- 172.21.15.31:0/647992614 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c9c004ba0 con 0x7f7ca80feaf0 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca819add0 con 0x7f7ca80feaf0 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7c9b7fe640 1 -- 172.21.15.31:0/647992614 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c9c004d00 con 0x7f7ca80feaf0 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7c9b7fe640 1 -- 172.21.15.31:0/647992614 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c9c003bd0 con 0x7f7ca80feaf0 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.419+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca819b2e0 con 0x7f7ca80feaf0 2024-09-16T08:55:24.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.421+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca8108860 con 0x7f7ca80feaf0 2024-09-16T08:55:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.422+0000 7f7c9b7fe640 1 -- 172.21.15.31:0/647992614 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7c9c031640 con 0x7f7ca80feaf0 2024-09-16T08:55:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.422+0000 7f7c9b7fe640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7c9406cc50 0x7f7c9406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.422+0000 7f7c9b7fe640 1 -- 172.21.15.31:0/647992614 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7c9c0ad5e0 con 0x7f7ca80feaf0 2024-09-16T08:55:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.422+0000 7f7cadd28640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7c9406cc50 0x7f7c9406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.422+0000 7f7cadd28640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7c9406cc50 0x7f7c9406f110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8196c30 tx=0x7f7ca4008040 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:24.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.426+0000 7f7c9b7fe640 1 -- 172.21.15.31:0/647992614 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7c9c077950 con 0x7f7ca80feaf0 2024-09-16T08:55:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.600+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f7ca81089f0 con 0x7f7ca80feaf0 2024-09-16T08:55:24.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.601+0000 7f7c9b7fe640 1 -- 172.21.15.31:0/647992614 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v42)=0 v42) v1 ==== 155+0+0 (secure 0 0 0) 0x7f7c9c07af70 con 0x7f7ca80feaf0 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7c9406cc50 msgr2=0x7f7c9406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7c9406cc50 0x7f7c9406f110 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8196c30 tx=0x7f7ca4008040 comp rx=0 tx=0).stop 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 msgr2=0x7f7ca8195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca8195710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c005610 tx=0x7f7c9c005640 comp rx=0 tx=0).stop 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 shutdown_connections 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7c9406cc50 0x7f7c9406f110 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca80ffcf0 0x7f7ca8195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 --2- 172.21.15.31:0/647992614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ca80feaf0 0x7f7ca8195710 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 >> 172.21.15.31:0/647992614 conn(0x7f7ca80fa280 msgr2=0x7f7ca8102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 shutdown_connections 2024-09-16T08:55:24.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:24.603+0000 7f7cb078d640 1 -- 172.21.15.31:0/647992614 wait complete. 2024-09-16T08:55:25.033 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:24 smithi031 ceph-mon[27844]: pgmap v223: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s wr, 0 op/s 2024-09-16T08:55:25.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:24 smithi097 ceph-mon[35478]: pgmap v223: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s wr, 0 op/s 2024-09-16T08:55:25.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-16T08:55:25.515 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:27.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 -- 172.21.15.31:0/2917002757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 msgr2=0x7f280c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 --2- 172.21.15.31:0/2917002757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c101210 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2800009880 tx=0x7f280002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 -- 172.21.15.31:0/2917002757 shutdown_connections 2024-09-16T08:55:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 --2- 172.21.15.31:0/2917002757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c101210 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 --2- 172.21.15.31:0/2917002757 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f280c0ffb90 0x7f280c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 -- 172.21.15.31:0/2917002757 >> 172.21.15.31:0/2917002757 conn(0x7f280c0fb320 msgr2=0x7f280c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 -- 172.21.15.31:0/2917002757 shutdown_connections 2024-09-16T08:55:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.108+0000 7f2812ae2640 1 -- 172.21.15.31:0/2917002757 wait complete. 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.109+0000 7f2812ae2640 1 Processor -- start 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.109+0000 7f2812ae2640 1 -- start start 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.109+0000 7f2812ae2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f280c0ffb90 0x7f280c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.109+0000 7f2812ae2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.109+0000 7f2812ae2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c196130 con 0x7f280c100d90 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.109+0000 7f2812ae2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1962a0 con 0x7f280c0ffb90 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f280bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f280bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c195b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39428/0 (socket says 172.21.15.31:39428) 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f280bfff640 1 -- 172.21.15.31:0/52321148 learned_addr learned my addr 172.21.15.31:0/52321148 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:27.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f281087e640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f280c0ffb90 0x7f280c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f280bfff640 1 -- 172.21.15.31:0/52321148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f280c0ffb90 msgr2=0x7f280c195620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f280bfff640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f280c0ffb90 0x7f280c195620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f280bfff640 1 -- 172.21.15.31:0/52321148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800009560 con 0x7f280c100d90 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f280bfff640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c195b60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2800005520 tx=0x7f2800005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f2809ffb640 1 -- 172.21.15.31:0/52321148 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2800004aa0 con 0x7f280c100d90 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f2809ffb640 1 -- 172.21.15.31:0/52321148 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2800004c00 con 0x7f280c100d90 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c19ac90 con 0x7f280c100d90 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.110+0000 7f2809ffb640 1 -- 172.21.15.31:0/52321148 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2800003b10 con 0x7f280c100d90 2024-09-16T08:55:27.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.111+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c19b200 con 0x7f280c100d90 2024-09-16T08:55:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.112+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280c18f710 con 0x7f280c100d90 2024-09-16T08:55:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.112+0000 7f2809ffb640 1 -- 172.21.15.31:0/52321148 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2800031d20 con 0x7f280c100d90 2024-09-16T08:55:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.113+0000 7f2809ffb640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f27f406cc50 0x7f27f406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.113+0000 7f2809ffb640 1 -- 172.21.15.31:0/52321148 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f28000ad400 con 0x7f280c100d90 2024-09-16T08:55:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.113+0000 7f281087e640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f27f406cc50 0x7f27f406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:27.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.113+0000 7f281087e640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f27f406cc50 0x7f27f406f110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f280c100bf0 tx=0x7f27fc009670 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:27.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.117+0000 7f2809ffb640 1 -- 172.21.15.31:0/52321148 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2800077820 con 0x7f280c100d90 2024-09-16T08:55:27.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.290+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f280c02c480 con 0x7f280c100d90 2024-09-16T08:55:27.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.291+0000 7f2809ffb640 1 -- 172.21.15.31:0/52321148 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v42)=0 v42) v1 ==== 163+0+0 (secure 0 0 0) 0x7f280007ae40 con 0x7f280c100d90 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f27f406cc50 msgr2=0x7f27f406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f27f406cc50 0x7f27f406f110 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f280c100bf0 tx=0x7f27fc009670 comp rx=0 tx=0).stop 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 msgr2=0x7f280c195b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c195b60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2800005520 tx=0x7f2800005550 comp rx=0 tx=0).stop 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 shutdown_connections 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f27f406cc50 0x7f27f406f110 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f280c100d90 0x7f280c195b60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 --2- 172.21.15.31:0/52321148 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f280c0ffb90 0x7f280c195620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 >> 172.21.15.31:0/52321148 conn(0x7f280c0fb320 msgr2=0x7f280c104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 shutdown_connections 2024-09-16T08:55:27.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:27.293+0000 7f2812ae2640 1 -- 172.21.15.31:0/52321148 wait complete. 2024-09-16T08:55:27.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:27 smithi097 ceph-mon[35478]: pgmap v224: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s wr, 0 op/s 2024-09-16T08:55:27.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:27 smithi031 ceph-mon[27844]: pgmap v224: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 852 B/s wr, 0 op/s 2024-09-16T08:55:28.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-16T08:55:28.253 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:29.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:29 smithi031 ceph-mon[27844]: pgmap v225: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-09-16T08:55:29.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:29 smithi097 ceph-mon[35478]: pgmap v225: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-09-16T08:55:29.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 -- 172.21.15.31:0/1650257118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c100630 msgr2=0x7fdc3c102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1650257118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c100630 0x7fdc3c102a20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fdc380098b0 tx=0x7fdc3802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 -- 172.21.15.31:0/1650257118 shutdown_connections 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1650257118 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc3c102f60 0x7fdc3c105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1650257118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c100630 0x7fdc3c102a20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 -- 172.21.15.31:0/1650257118 >> 172.21.15.31:0/1650257118 conn(0x7fdc3c0fa250 msgr2=0x7fdc3c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 -- 172.21.15.31:0/1650257118 shutdown_connections 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.969+0000 7fdc43e22640 1 -- 172.21.15.31:0/1650257118 wait complete. 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc43e22640 1 Processor -- start 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc43e22640 1 -- start start 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc43e22640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc3c100630 0x7fdc3c195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:29.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc43e22640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c102f60 0x7fdc3c195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc43e22640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c196170 con 0x7fdc3c102f60 2024-09-16T08:55:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc43e22640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1962e0 con 0x7fdc3c100630 2024-09-16T08:55:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc413bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c102f60 0x7fdc3c195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc413bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c102f60 0x7fdc3c195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:39444/0 (socket says 172.21.15.31:39444) 2024-09-16T08:55:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc413bd640 1 -- 172.21.15.31:0/1659065252 learned_addr learned my addr 172.21.15.31:0/1659065252 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc41bbe640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc3c100630 0x7fdc3c195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:29.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc413bd640 1 -- 172.21.15.31:0/1659065252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc3c100630 msgr2=0x7fdc3c195660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc413bd640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc3c100630 0x7fdc3c195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc413bd640 1 -- 172.21.15.31:0/1659065252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38009560 con 0x7fdc3c102f60 2024-09-16T08:55:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.970+0000 7fdc413bd640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c102f60 0x7fdc3c195ba0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00d6f0 tx=0x7fdc2c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.971+0000 7fdc32ffd640 1 -- 172.21.15.31:0/1659065252 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc2c0051f0 con 0x7fdc3c102f60 2024-09-16T08:55:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.971+0000 7fdc32ffd640 1 -- 172.21.15.31:0/1659065252 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc2c009ce0 con 0x7fdc3c102f60 2024-09-16T08:55:29.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.971+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c19ad80 con 0x7fdc3c102f60 2024-09-16T08:55:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.971+0000 7fdc32ffd640 1 -- 172.21.15.31:0/1659065252 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc2c004e80 con 0x7fdc3c102f60 2024-09-16T08:55:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.971+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c19b350 con 0x7fdc3c102f60 2024-09-16T08:55:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.972+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c1087e0 con 0x7fdc3c102f60 2024-09-16T08:55:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.972+0000 7fdc32ffd640 1 -- 172.21.15.31:0/1659065252 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdc2c0046c0 con 0x7fdc3c102f60 2024-09-16T08:55:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.973+0000 7fdc32ffd640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc2806cc50 0x7fdc2806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:29.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.973+0000 7fdc32ffd640 1 -- 172.21.15.31:0/1659065252 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdc2c08bcf0 con 0x7fdc3c102f60 2024-09-16T08:55:29.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.973+0000 7fdc41bbe640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc2806cc50 0x7fdc2806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:29.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.973+0000 7fdc41bbe640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc2806cc50 0x7fdc2806f110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdc38031540 tx=0x7fdc38000bc0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:29.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:29.977+0000 7fdc32ffd640 1 -- 172.21.15.31:0/1659065252 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdc2c056040 con 0x7fdc3c102f60 2024-09-16T08:55:30.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.152+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fdc3c19b590 con 0x7fdc3c102f60 2024-09-16T08:55:30.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.152+0000 7fdc32ffd640 1 -- 172.21.15.31:0/1659065252 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v42)=0 v42) v1 ==== 135+0+0 (secure 0 0 0) 0x7fdc2c0170c0 con 0x7fdc3c102f60 2024-09-16T08:55:30.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.154+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc2806cc50 msgr2=0x7fdc2806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.154+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc2806cc50 0x7fdc2806f110 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdc38031540 tx=0x7fdc38000bc0 comp rx=0 tx=0).stop 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.154+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c102f60 msgr2=0x7fdc3c195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.154+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c102f60 0x7fdc3c195ba0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00d6f0 tx=0x7fdc2c00dbb0 comp rx=0 tx=0).stop 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.154+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 shutdown_connections 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.154+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc2806cc50 0x7fdc2806f110 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.154+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc3c102f60 0x7fdc3c195ba0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.155+0000 7fdc43e22640 1 --2- 172.21.15.31:0/1659065252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc3c100630 0x7fdc3c195660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.155+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 >> 172.21.15.31:0/1659065252 conn(0x7fdc3c0fa250 msgr2=0x7fdc3c0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.155+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 shutdown_connections 2024-09-16T08:55:30.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:30.155+0000 7fdc43e22640 1 -- 172.21.15.31:0/1659065252 wait complete. 2024-09-16T08:55:30.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-16T08:55:31.022 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:31.484 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:31 smithi031 ceph-mon[27844]: pgmap v226: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-16T08:55:31.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:31 smithi097 ceph-mon[35478]: pgmap v226: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-16T08:55:32.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 -- 172.21.15.31:0/1689942641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80ffcf0 msgr2=0x7f0ff8100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 --2- 172.21.15.31:0/1689942641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80ffcf0 0x7f0ff8100170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009880 tx=0x7f0fec02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 -- 172.21.15.31:0/1689942641 shutdown_connections 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 --2- 172.21.15.31:0/1689942641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80ffcf0 0x7f0ff8100170 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 --2- 172.21.15.31:0/1689942641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ff80feaf0 0x7f0ff80feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 -- 172.21.15.31:0/1689942641 >> 172.21.15.31:0/1689942641 conn(0x7f0ff80fa280 msgr2=0x7f0ff80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 -- 172.21.15.31:0/1689942641 shutdown_connections 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 -- 172.21.15.31:0/1689942641 wait complete. 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 Processor -- start 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.619+0000 7f0ffef72640 1 -- start start 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffef72640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80feaf0 0x7f0ff8195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffef72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ff80ffcf0 0x7f0ff8195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:32.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffef72640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8196270 con 0x7f0ff80feaf0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffef72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff81963e0 con 0x7f0ff80ffcf0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffcd0e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80feaf0 0x7f0ff8195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffcd0e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80feaf0 0x7f0ff8195760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35954/0 (socket says 172.21.15.31:35954) 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffcd0e640 1 -- 172.21.15.31:0/4269895526 learned_addr learned my addr 172.21.15.31:0/4269895526 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffcd0e640 1 -- 172.21.15.31:0/4269895526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ff80ffcf0 msgr2=0x7f0ff8195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ff7fff640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ff80ffcf0 0x7f0ff8195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffcd0e640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ff80ffcf0 0x7f0ff8195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffcd0e640 1 -- 172.21.15.31:0/4269895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec009560 con 0x7f0ff80feaf0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ff7fff640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ff80ffcf0 0x7f0ff8195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ffcd0e640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80feaf0 0x7f0ff8195760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800b400 tx=0x7f0fe800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.620+0000 7f0ff5ffb640 1 -- 172.21.15.31:0/4269895526 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe8011800 con 0x7f0ff80feaf0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.621+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff819ae80 con 0x7f0ff80feaf0 2024-09-16T08:55:32.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.621+0000 7f0ff5ffb640 1 -- 172.21.15.31:0/4269895526 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fe8011e20 con 0x7f0ff80feaf0 2024-09-16T08:55:32.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.621+0000 7f0ff5ffb640 1 -- 172.21.15.31:0/4269895526 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe800f2e0 con 0x7f0ff80feaf0 2024-09-16T08:55:32.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.621+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff819b3f0 con 0x7f0ff80feaf0 2024-09-16T08:55:32.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.622+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff8107780 con 0x7f0ff80feaf0 2024-09-16T08:55:32.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.622+0000 7f0ff5ffb640 1 -- 172.21.15.31:0/4269895526 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f0fe8010380 con 0x7f0ff80feaf0 2024-09-16T08:55:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.623+0000 7f0ff5ffb640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0fe006cc50 0x7f0fe006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.623+0000 7f0ff5ffb640 1 -- 172.21.15.31:0/4269895526 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0fe808bfe0 con 0x7f0ff80feaf0 2024-09-16T08:55:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.623+0000 7f0ff7fff640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0fe006cc50 0x7f0fe006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:32.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.623+0000 7f0ff7fff640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0fe006cc50 0x7f0fe006f110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0fec031540 tx=0x7f0fec005de0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:32.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.627+0000 7f0ff5ffb640 1 -- 172.21.15.31:0/4269895526 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f0fe80562f0 con 0x7f0ff80feaf0 2024-09-16T08:55:32.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.815+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ff819b670 con 0x7f0fe006cc50 2024-09-16T08:55:32.822 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.821+0000 7f0ff5ffb640 1 -- 172.21.15.31:0/4269895526 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f0ff819b670 con 0x7f0fe006cc50 2024-09-16T08:55:32.822 INFO:teuthology.orchestra.run.smithi031.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0fe006cc50 msgr2=0x7f0fe006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0fe006cc50 0x7f0fe006f110 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0fec031540 tx=0x7f0fec005de0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80feaf0 msgr2=0x7f0ff8195760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:32.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80feaf0 0x7f0ff8195760 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800b400 tx=0x7f0fe800b8c0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 shutdown_connections 2024-09-16T08:55:32.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f0fe006cc50 0x7f0fe006f110 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0ff80ffcf0 0x7f0ff8195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 --2- 172.21.15.31:0/4269895526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ff80feaf0 0x7f0ff8195760 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:32.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 >> 172.21.15.31:0/4269895526 conn(0x7f0ff80fa280 msgr2=0x7f0ff8067ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:32.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 shutdown_connections 2024-09-16T08:55:32.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:32.823+0000 7f0ffef72640 1 -- 172.21.15.31:0/4269895526 wait complete. 2024-09-16T08:55:33.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:33 smithi097 ceph-mon[35478]: pgmap v227: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-16T08:55:33.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:33 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:33.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:33 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:55:33.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:33 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:33.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:33 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:33.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:33 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:33.538 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:33 smithi031 ceph-mon[27844]: pgmap v227: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-16T08:55:33.538 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:33 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:33.538 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:33 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:55:33.538 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:33 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:55:33.538 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:33 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:55:33.539 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:33 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:55:33.607 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-16T08:55:33.608 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T08:55:33.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-16T08:55:33.824 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T08:55:34.281 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:34 smithi031 ceph-mon[27844]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:34.282 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:34 smithi031 ceph-mon[27844]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:34.282 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:34 smithi031 ceph-mon[27844]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:34.517 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:34 smithi097 ceph-mon[35478]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:35.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:34 smithi097 ceph-mon[35478]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:35.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:34 smithi097 ceph-mon[35478]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T08:55:35.500 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:35 smithi031 ceph-mon[27844]: pgmap v228: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-16T08:55:35.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:35 smithi097 ceph-mon[35478]: pgmap v228: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-09-16T08:55:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.736+0000 7f09b8905640 1 -- 172.21.15.31:0/238836785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40ffcf0 msgr2=0x7f09b4100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.736+0000 7f09b8905640 1 --2- 172.21.15.31:0/238836785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40ffcf0 0x7f09b4100170 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f09a8009880 tx=0x7f09a802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.737+0000 7f09b8905640 1 -- 172.21.15.31:0/238836785 shutdown_connections 2024-09-16T08:55:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.737+0000 7f09b8905640 1 --2- 172.21.15.31:0/238836785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40ffcf0 0x7f09b4100170 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.737+0000 7f09b8905640 1 --2- 172.21.15.31:0/238836785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09b40feaf0 0x7f09b40feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.737+0000 7f09b8905640 1 -- 172.21.15.31:0/238836785 >> 172.21.15.31:0/238836785 conn(0x7f09b40fa280 msgr2=0x7f09b40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.737+0000 7f09b8905640 1 -- 172.21.15.31:0/238836785 shutdown_connections 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.737+0000 7f09b8905640 1 -- 172.21.15.31:0/238836785 wait complete. 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b8905640 1 Processor -- start 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b8905640 1 -- start start 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b8905640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40feaf0 0x7f09b41956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b8905640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09b40ffcf0 0x7f09b4195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b8905640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b41961f0 con 0x7f09b40feaf0 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b8905640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b4196360 con 0x7f09b40ffcf0 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40feaf0 0x7f09b41956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40feaf0 0x7f09b41956e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35962/0 (socket says 172.21.15.31:35962) 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b259c640 1 -- 172.21.15.31:0/2370659315 learned_addr learned my addr 172.21.15.31:0/2370659315 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.738+0000 7f09b1d9b640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09b40ffcf0 0x7f09b4195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:35.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09b259c640 1 -- 172.21.15.31:0/2370659315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09b40ffcf0 msgr2=0x7f09b4195c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09b259c640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09b40ffcf0 0x7f09b4195c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09b259c640 1 -- 172.21.15.31:0/2370659315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8009560 con 0x7f09b40feaf0 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09b259c640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40feaf0 0x7f09b41956e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f09a400d900 tx=0x7f09a400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09a37fe640 1 -- 172.21.15.31:0/2370659315 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a4009c00 con 0x7f09b40feaf0 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09a37fe640 1 -- 172.21.15.31:0/2370659315 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09a4009d60 con 0x7f09b40feaf0 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09a37fe640 1 -- 172.21.15.31:0/2370659315 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a400f660 con 0x7f09b40feaf0 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b419ae00 con 0x7f09b40feaf0 2024-09-16T08:55:35.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.739+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b419b2d0 con 0x7f09b40feaf0 2024-09-16T08:55:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.740+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b40feef0 con 0x7f09b40feaf0 2024-09-16T08:55:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.741+0000 7f09a37fe640 1 -- 172.21.15.31:0/2370659315 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f09a4010460 con 0x7f09b40feaf0 2024-09-16T08:55:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.742+0000 7f09a37fe640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f099c06cc50 0x7f099c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.742+0000 7f09a37fe640 1 -- 172.21.15.31:0/2370659315 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f09a408c0c0 con 0x7f09b40feaf0 2024-09-16T08:55:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.742+0000 7f09b1d9b640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f099c06cc50 0x7f099c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:35.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.742+0000 7f09b1d9b640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f099c06cc50 0x7f099c06f110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000c00 tx=0x7f09a80314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:35.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.747+0000 7f09a37fe640 1 -- 172.21.15.31:0/2370659315 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f09a40564b0 con 0x7f09b40feaf0 2024-09-16T08:55:35.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.958+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09b40683f0 con 0x7f099c06cc50 2024-09-16T08:55:35.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.959+0000 7f09a37fe640 1 -- 172.21.15.31:0/2370659315 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f09b40683f0 con 0x7f099c06cc50 2024-09-16T08:55:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f099c06cc50 msgr2=0x7f099c06f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:35.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f099c06cc50 0x7f099c06f110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000c00 tx=0x7f09a80314e0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40feaf0 msgr2=0x7f09b41956e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40feaf0 0x7f09b41956e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f09a400d900 tx=0x7f09a400ddc0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 shutdown_connections 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f099c06cc50 0x7f099c06f110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09b40ffcf0 0x7f09b4195c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.961+0000 7f09b8905640 1 --2- 172.21.15.31:0/2370659315 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09b40feaf0 0x7f09b41956e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.962+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 >> 172.21.15.31:0/2370659315 conn(0x7f09b40fa280 msgr2=0x7f09b4067ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.962+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 shutdown_connections 2024-09-16T08:55:35.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:35.962+0000 7f09b8905640 1 -- 172.21.15.31:0/2370659315 wait complete. 2024-09-16T08:55:35.972 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 -- 172.21.15.31:0/1464630788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 msgr2=0x7f2c54103400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 --2- 172.21.15.31:0/1464630788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54103400 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2c44009880 tx=0x7f2c4402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 -- 172.21.15.31:0/1464630788 shutdown_connections 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 --2- 172.21.15.31:0/1464630788 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54103400 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 --2- 172.21.15.31:0/1464630788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c54101d80 0x7f2c54102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 -- 172.21.15.31:0/1464630788 >> 172.21.15.31:0/1464630788 conn(0x7f2c540fd510 msgr2=0x7f2c540ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 -- 172.21.15.31:0/1464630788 shutdown_connections 2024-09-16T08:55:36.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.055+0000 7f2c5b434640 1 -- 172.21.15.31:0/1464630788 wait complete. 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c5b434640 1 Processor -- start 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c5b434640 1 -- start start 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c5b434640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c54101d80 0x7f2c54197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c5b434640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c5b434640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54198310 con 0x7f2c54102f80 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c5b434640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c54198480 con 0x7f2c54101d80 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c589cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c589cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35990/0 (socket says 172.21.15.31:35990) 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c589cf640 1 -- 172.21.15.31:0/2476621545 learned_addr learned my addr 172.21.15.31:0/2476621545 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c591d0640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c54101d80 0x7f2c54197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c589cf640 1 -- 172.21.15.31:0/2476621545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c54101d80 msgr2=0x7f2c54197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.058 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c589cf640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c54101d80 0x7f2c54197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c589cf640 1 -- 172.21.15.31:0/2476621545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44009560 con 0x7f2c54102f80 2024-09-16T08:55:36.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.056+0000 7f2c591d0640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c54101d80 0x7f2c54197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:55:36.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.057+0000 7f2c589cf640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54197d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005520 tx=0x7f2c44005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:36.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.057+0000 7f2c4a7fc640 1 -- 172.21.15.31:0/2476621545 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c44004aa0 con 0x7f2c54102f80 2024-09-16T08:55:36.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.057+0000 7f2c4a7fc640 1 -- 172.21.15.31:0/2476621545 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c44004c00 con 0x7f2c54102f80 2024-09-16T08:55:36.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.057+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5419cec0 con 0x7f2c54102f80 2024-09-16T08:55:36.059 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.057+0000 7f2c4a7fc640 1 -- 172.21.15.31:0/2476621545 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c44003b10 con 0x7f2c54102f80 2024-09-16T08:55:36.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.057+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5419d400 con 0x7f2c54102f80 2024-09-16T08:55:36.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.059+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c54102180 con 0x7f2c54102f80 2024-09-16T08:55:36.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.059+0000 7f2c4a7fc640 1 -- 172.21.15.31:0/2476621545 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f2c44031d20 con 0x7f2c54102f80 2024-09-16T08:55:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.060+0000 7f2c4a7fc640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f2c4006cc50 0x7f2c4006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.060+0000 7f2c4a7fc640 1 -- 172.21.15.31:0/2476621545 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2c440ad060 con 0x7f2c54102f80 2024-09-16T08:55:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.060+0000 7f2c591d0640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f2c4006cc50 0x7f2c4006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.061 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.061+0000 7f2c591d0640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f2c4006cc50 0x7f2c4006f110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c500058c0 tx=0x7f2c50005850 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:36.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.065+0000 7f2c4a7fc640 1 -- 172.21.15.31:0/2476621545 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2c440773d0 con 0x7f2c54102f80 2024-09-16T08:55:36.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.280+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c5402c4f0 con 0x7f2c4006cc50 2024-09-16T08:55:36.282 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.281+0000 7f2c4a7fc640 1 -- 172.21.15.31:0/2476621545 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f2c5402c4f0 con 0x7f2c4006cc50 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.283+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f2c4006cc50 msgr2=0x7f2c4006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f2c4006cc50 0x7f2c4006f110 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2c500058c0 tx=0x7f2c50005850 comp rx=0 tx=0).stop 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 msgr2=0x7f2c54197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54197d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005520 tx=0x7f2c44005550 comp rx=0 tx=0).stop 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 shutdown_connections 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f2c4006cc50 0x7f2c4006f110 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c54102f80 0x7f2c54197d40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 --2- 172.21.15.31:0/2476621545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c54101d80 0x7f2c54197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 >> 172.21.15.31:0/2476621545 conn(0x7f2c540fd510 msgr2=0x7f2c54106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:36.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 shutdown_connections 2024-09-16T08:55:36.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.284+0000 7f2c5b434640 1 -- 172.21.15.31:0/2476621545 wait complete. 2024-09-16T08:55:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 -- 172.21.15.31:0/1285536248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0feaf0 msgr2=0x7ffa5c0feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/1285536248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0feaf0 0x7ffa5c0feef0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ffa50009880 tx=0x7ffa5002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 -- 172.21.15.31:0/1285536248 shutdown_connections 2024-09-16T08:55:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/1285536248 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0ffcf0 0x7ffa5c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/1285536248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0feaf0 0x7ffa5c0feef0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 -- 172.21.15.31:0/1285536248 >> 172.21.15.31:0/1285536248 conn(0x7ffa5c0fa280 msgr2=0x7ffa5c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 -- 172.21.15.31:0/1285536248 shutdown_connections 2024-09-16T08:55:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.378+0000 7ffa62d3f640 1 -- 172.21.15.31:0/1285536248 wait complete. 2024-09-16T08:55:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa62d3f640 1 Processor -- start 2024-09-16T08:55:36.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa62d3f640 1 -- start start 2024-09-16T08:55:36.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa62d3f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0feaf0 0x7ffa5c068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa62d3f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0ffcf0 0x7ffa5c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa62d3f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c19a9f0 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa62d3f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5c19ab30 con 0x7ffa5c0feaf0 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0ffcf0 0x7ffa5c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa5bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0ffcf0 0x7ffa5c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36006/0 (socket says 172.21.15.31:36006) 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa5bfff640 1 -- 172.21.15.31:0/87112574 learned_addr learned my addr 172.21.15.31:0/87112574 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa60adb640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0feaf0 0x7ffa5c068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa5bfff640 1 -- 172.21.15.31:0/87112574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0feaf0 msgr2=0x7ffa5c068790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa5bfff640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0feaf0 0x7ffa5c068790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.379+0000 7ffa5bfff640 1 -- 172.21.15.31:0/87112574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa50009560 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.380+0000 7ffa5bfff640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0ffcf0 0x7ffa5c19a420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800c8f0 tx=0x7ffa4800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.380+0000 7ffa59ffb640 1 -- 172.21.15.31:0/87112574 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa48007b80 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.380+0000 7ffa59ffb640 1 -- 172.21.15.31:0/87112574 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa480054e0 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.380+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5c19ade0 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.380+0000 7ffa59ffb640 1 -- 172.21.15.31:0/87112574 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa4800f450 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.380+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5c19b2a0 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.382+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa5c0feef0 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.382+0000 7ffa59ffb640 1 -- 172.21.15.31:0/87112574 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7ffa48004f60 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.383+0000 7ffa59ffb640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffa4c06cc60 0x7ffa4c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.383+0000 7ffa59ffb640 1 -- 172.21.15.31:0/87112574 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ffa4808bf80 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.383+0000 7ffa60adb640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffa4c06cc60 0x7ffa4c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.384+0000 7ffa60adb640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffa4c06cc60 0x7ffa4c06f120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006010 tx=0x7ffa500314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:36.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.387+0000 7ffa59ffb640 1 -- 172.21.15.31:0/87112574 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7ffa48056420 con 0x7ffa5c0ffcf0 2024-09-16T08:55:36.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.603+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa5c02c5a0 con 0x7ffa4c06cc60 2024-09-16T08:55:36.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.616+0000 7ffa59ffb640 1 -- 172.21.15.31:0/87112574 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7ffa5c02c5a0 con 0x7ffa4c06cc60 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 24s ago 7m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (7m) 24s ago 7m 7528k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 66d87845da99 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (6m) 35s ago 6m 7440k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 edd14d48bff9 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (7m) 24s ago 7m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (6m) 35s ago 6m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 24s ago 6m 83.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (55s) 24s ago 54s 11.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (48s) 24s ago 47s 18.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (52s) 35s ago 51s 18.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:55:36.617 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (45s) 35s ago 45s 13.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:9283,8765,8443 running (8m) 24s ago 8m 486M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 1c78052ec7c9 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (6m) 35s ago 6m 441M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b82e4c86f40c 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (8m) 24s ago 8m 47.7M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (6m) 35s ago 6m 35.3M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 24s ago 7m 15.3M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (6m) 35s ago 6m 16.1M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 24s ago 4m 47.3M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 24s ago 4m 54.9M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (3m) 24s ago 3m 46.6M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (3m) 35s ago 3m 51.5M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (2m) 35s ago 2m 44.3M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (2m) 35s ago 2m 44.0M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:55:36.618 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (4m) 24s ago 6m 60.0M - 2.43.0 a07b618ecd1d b0b74488871e 2024-09-16T08:55:36.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.618+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffa4c06cc60 msgr2=0x7ffa4c06f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.618+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffa4c06cc60 0x7ffa4c06f120 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffa50006010 tx=0x7ffa500314e0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.618+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0ffcf0 msgr2=0x7ffa5c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.618+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0ffcf0 0x7ffa5c19a420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800c8f0 tx=0x7ffa4800cdb0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.618+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 shutdown_connections 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.618+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7ffa4c06cc60 0x7ffa4c06f120 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.619+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa5c0ffcf0 0x7ffa5c19a420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.619+0000 7ffa62d3f640 1 --2- 172.21.15.31:0/87112574 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa5c0feaf0 0x7ffa5c068790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.619+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 >> 172.21.15.31:0/87112574 conn(0x7ffa5c0fa280 msgr2=0x7ffa5c1033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.619+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 shutdown_connections 2024-09-16T08:55:36.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.619+0000 7ffa62d3f640 1 -- 172.21.15.31:0/87112574 wait complete. 2024-09-16T08:55:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 -- 172.21.15.31:0/951337649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 msgr2=0x7f6338101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 --2- 172.21.15.31:0/951337649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f6338101210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6328009880 tx=0x7f632802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 -- 172.21.15.31:0/951337649 shutdown_connections 2024-09-16T08:55:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 --2- 172.21.15.31:0/951337649 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f6338101210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 --2- 172.21.15.31:0/951337649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63380ffb90 0x7f63380fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 -- 172.21.15.31:0/951337649 >> 172.21.15.31:0/951337649 conn(0x7f63380fb360 msgr2=0x7f63380fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 -- 172.21.15.31:0/951337649 shutdown_connections 2024-09-16T08:55:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.712+0000 7f6340235640 1 -- 172.21.15.31:0/951337649 wait complete. 2024-09-16T08:55:36.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.713+0000 7f6340235640 1 Processor -- start 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.713+0000 7f6340235640 1 -- start start 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.713+0000 7f6340235640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63380ffb90 0x7f6338074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.713+0000 7f6340235640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f63380728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.713+0000 7f6340235640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6338072de0 con 0x7f6338100d90 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.713+0000 7f6340235640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6338072f50 con 0x7f63380ffb90 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633d7d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f63380728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633d7d0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f63380728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36026/0 (socket says 172.21.15.31:36026) 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633d7d0640 1 -- 172.21.15.31:0/596355749 learned_addr learned my addr 172.21.15.31:0/596355749 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:36.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633dfd1640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63380ffb90 0x7f6338074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633d7d0640 1 -- 172.21.15.31:0/596355749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63380ffb90 msgr2=0x7f6338074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:36.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633d7d0640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63380ffb90 0x7f6338074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:36.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633d7d0640 1 -- 172.21.15.31:0/596355749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6328009560 con 0x7f6338100d90 2024-09-16T08:55:36.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633dfd1640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63380ffb90 0x7f6338074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:55:36.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f633d7d0640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f63380728a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f63280054f0 tx=0x7f6328005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:36.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.714+0000 7f632effd640 1 -- 172.21.15.31:0/596355749 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6328004aa0 con 0x7f6338100d90 2024-09-16T08:55:36.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.715+0000 7f632effd640 1 -- 172.21.15.31:0/596355749 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6328004c00 con 0x7f6338100d90 2024-09-16T08:55:36.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.715+0000 7f632effd640 1 -- 172.21.15.31:0/596355749 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6328003ac0 con 0x7f6338100d90 2024-09-16T08:55:36.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.715+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63380731d0 con 0x7f6338100d90 2024-09-16T08:55:36.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.715+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63380736f0 con 0x7f6338100d90 2024-09-16T08:55:36.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.716+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633818f710 con 0x7f6338100d90 2024-09-16T08:55:36.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.717+0000 7f632effd640 1 -- 172.21.15.31:0/596355749 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f6328031c70 con 0x7f6338100d90 2024-09-16T08:55:36.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.718+0000 7f632effd640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f632406cc50 0x7f632406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:36.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.718+0000 7f632effd640 1 -- 172.21.15.31:0/596355749 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f63280ad510 con 0x7f6338100d90 2024-09-16T08:55:36.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.718+0000 7f633dfd1640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f632406cc50 0x7f632406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:36.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.719+0000 7f633dfd1640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f632406cc50 0x7f632406f110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63340058c0 tx=0x7f6334005850 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:36.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:36.725+0000 7f632effd640 1 -- 172.21.15.31:0/596355749 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f6328077880 con 0x7f6338100d90 2024-09-16T08:55:37.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.013+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6338108780 con 0x7f6338100d90 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.014+0000 7f632effd640 1 -- 172.21.15.31:0/596355749 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f632807aea0 con 0x7f6338100d90 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:55:37.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 14 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:55:37.016 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f632406cc50 msgr2=0x7f632406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f632406cc50 0x7f632406f110 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f63340058c0 tx=0x7f6334005850 comp rx=0 tx=0).stop 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 msgr2=0x7f63380728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f63380728a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f63280054f0 tx=0x7f6328005520 comp rx=0 tx=0).stop 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 shutdown_connections 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f632406cc50 0x7f632406f110 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6338100d90 0x7f63380728a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 --2- 172.21.15.31:0/596355749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63380ffb90 0x7f6338074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 >> 172.21.15.31:0/596355749 conn(0x7f63380fb360 msgr2=0x7f6338104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 shutdown_connections 2024-09-16T08:55:37.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.017+0000 7f6340235640 1 -- 172.21.15.31:0/596355749 wait complete. 2024-09-16T08:55:37.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 -- 172.21.15.31:0/3425908607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0ffcf0 msgr2=0x7f3e7c100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3425908607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0ffcf0 0x7f3e7c100170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009880 tx=0x7f3e6c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 -- 172.21.15.31:0/3425908607 shutdown_connections 2024-09-16T08:55:37.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3425908607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0ffcf0 0x7f3e7c100170 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3425908607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e7c0feaf0 0x7f3e7c0feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 -- 172.21.15.31:0/3425908607 >> 172.21.15.31:0/3425908607 conn(0x7f3e7c0fa280 msgr2=0x7f3e7c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:37.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 -- 172.21.15.31:0/3425908607 shutdown_connections 2024-09-16T08:55:37.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.115+0000 7f3e8415b640 1 -- 172.21.15.31:0/3425908607 wait complete. 2024-09-16T08:55:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.116+0000 7f3e8415b640 1 Processor -- start 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.116+0000 7f3e8415b640 1 -- start start 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.116+0000 7f3e8415b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0feaf0 0x7f3e7c068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e8415b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e7c0ffcf0 0x7f3e7c19a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e8415b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19aa20 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e8415b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c19ab60 con 0x7f3e7c0ffcf0 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e81ef7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0feaf0 0x7f3e7c068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e81ef7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0feaf0 0x7f3e7c068790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36032/0 (socket says 172.21.15.31:36032) 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e81ef7640 1 -- 172.21.15.31:0/3679256498 learned_addr learned my addr 172.21.15.31:0/3679256498 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e816f6640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e7c0ffcf0 0x7f3e7c19a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e81ef7640 1 -- 172.21.15.31:0/3679256498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e7c0ffcf0 msgr2=0x7f3e7c19a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e81ef7640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e7c0ffcf0 0x7f3e7c19a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e81ef7640 1 -- 172.21.15.31:0/3679256498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c009560 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e816f6640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e7c0ffcf0 0x7f3e7c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:55:37.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e81ef7640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0feaf0 0x7f3e7c068790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e940 tx=0x7f3e7800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.117+0000 7f3e72ffd640 1 -- 172.21.15.31:0/3679256498 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e780096e0 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.118+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c19ae10 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.118+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c19b2d0 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.118+0000 7f3e72ffd640 1 -- 172.21.15.31:0/3679256498 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e780044f0 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.118+0000 7f3e72ffd640 1 -- 172.21.15.31:0/3679256498 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e78010430 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.119+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c0feef0 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.119+0000 7f3e72ffd640 1 -- 172.21.15.31:0/3679256498 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f3e7800cc90 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.120+0000 7f3e72ffd640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f3e6806cc60 0x7f3e6806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.120+0000 7f3e72ffd640 1 -- 172.21.15.31:0/3679256498 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3e78014070 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.120+0000 7f3e816f6640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f3e6806cc60 0x7f3e6806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.121+0000 7f3e816f6640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f3e6806cc60 0x7f3e6806f120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c005a20 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:37.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.125+0000 7f3e72ffd640 1 -- 172.21.15.31:0/3679256498 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f3e78056a90 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:37 smithi031 ceph-mon[27844]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:37 smithi031 ceph-mon[27844]: pgmap v229: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:55:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:37 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/596355749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:55:37.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.456+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3e7c02c4e0 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.457+0000 7f3e72ffd640 1 -- 172.21.15.31:0/3679256498 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f3e7805a0b0 con 0x7f3e7c0feaf0 2024-09-16T08:55:37.458 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:55:37.458 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:55:37.458 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:55:37.458 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:55:37.458 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:55:37.458 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:55:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:55:37.460 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:55:37.461 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f3e6806cc60 msgr2=0x7f3e6806f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f3e6806cc60 0x7f3e6806f120 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c005a20 comp rx=0 tx=0).stop 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0feaf0 msgr2=0x7f3e7c068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0feaf0 0x7f3e7c068790 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e940 tx=0x7f3e7800ee00 comp rx=0 tx=0).stop 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 shutdown_connections 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f3e6806cc60 0x7f3e6806f120 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e7c0ffcf0 0x7f3e7c19a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 --2- 172.21.15.31:0/3679256498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e7c0feaf0 0x7f3e7c068790 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.462 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 >> 172.21.15.31:0/3679256498 conn(0x7f3e7c0fa280 msgr2=0x7f3e7c1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:37.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 shutdown_connections 2024-09-16T08:55:37.463 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.459+0000 7f3e8415b640 1 -- 172.21.15.31:0/3679256498 wait complete. 2024-09-16T08:55:37.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:37 smithi097 ceph-mon[35478]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:37 smithi097 ceph-mon[35478]: pgmap v229: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:55:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:37 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/596355749' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.556+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3512022838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 msgr2=0x7f7ee00ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.556+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3512022838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 0x7f7ee00ff270 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009880 tx=0x7f7ed402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.557+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3512022838 shutdown_connections 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.557+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3512022838 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 0x7f7ee00ff270 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.557+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3512022838 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee00fe4d0 0x7f7ee00fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.557+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3512022838 >> 172.21.15.31:0/3512022838 conn(0x7f7ee00fa280 msgr2=0x7f7ee00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.557+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3512022838 shutdown_connections 2024-09-16T08:55:37.558 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.557+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3512022838 wait complete. 2024-09-16T08:55:37.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ee52ec640 1 Processor -- start 2024-09-16T08:55:37.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ee52ec640 1 -- start start 2024-09-16T08:55:37.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ee52ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee00fe4d0 0x7f7ee01978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ee52ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 0x7f7ee0197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ee52ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee01983c0 con 0x7f7ee00fee10 2024-09-16T08:55:37.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ee52ec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0198530 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ede7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 0x7f7ee0197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ede7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 0x7f7ee0197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36050/0 (socket says 172.21.15.31:36050) 2024-09-16T08:55:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7ede7fc640 1 -- 172.21.15.31:0/3857930794 learned_addr learned my addr 172.21.15.31:0/3857930794 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:37.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.558+0000 7f7edeffd640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee00fe4d0 0x7f7ee01978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7edeffd640 1 -- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 msgr2=0x7f7ee0197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7edeffd640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 0x7f7ee0197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7edeffd640 1 -- 172.21.15.31:0/3857930794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed4009560 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7edeffd640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee00fe4d0 0x7f7ee01978b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000e970 tx=0x7f7ed000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7ec7fff640 1 -- 172.21.15.31:0/3857930794 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed0009640 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7ec7fff640 1 -- 172.21.15.31:0/3857930794 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ed000cc90 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee019cf80 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7ec7fff640 1 -- 172.21.15.31:0/3857930794 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ed0018810 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.559+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee019d440 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.561+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ecc005280 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.561+0000 7f7ec7fff640 1 -- 172.21.15.31:0/3857930794 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f7ed0018a80 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.562+0000 7f7ec7fff640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7ec806cc50 0x7f7ec806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.562+0000 7f7ec7fff640 1 -- 172.21.15.31:0/3857930794 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7ed0014070 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.562+0000 7f7ede7fc640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7ec806cc50 0x7f7ec806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.562+0000 7f7ede7fc640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7ec806cc50 0x7f7ec806f110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0198d80 tx=0x7f7ed40314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:37.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.566+0000 7f7ec7fff640 1 -- 172.21.15.31:0/3857930794 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f7ed0052e20 con 0x7f7ee00fe4d0 2024-09-16T08:55:37.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.794+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ecc000bf0 con 0x7f7ec806cc50 2024-09-16T08:55:37.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.796+0000 7f7ec7fff640 1 -- 172.21.15.31:0/3857930794 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f7ecc000bf0 con 0x7f7ec806cc50 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705", 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "", 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 image", 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:55:37.797 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:55:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7ec806cc50 msgr2=0x7f7ec806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7ec806cc50 0x7f7ec806f110 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0198d80 tx=0x7f7ed40314e0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee00fe4d0 msgr2=0x7f7ee01978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee00fe4d0 0x7f7ee01978b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000e970 tx=0x7f7ed000ee30 comp rx=0 tx=0).stop 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 shutdown_connections 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f7ec806cc50 0x7f7ec806f110 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7ee00fee10 0x7f7ee0197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 --2- 172.21.15.31:0/3857930794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ee00fe4d0 0x7f7ee01978b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.798+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 >> 172.21.15.31:0/3857930794 conn(0x7f7ee00fa280 msgr2=0x7f7ee0106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.799+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 shutdown_connections 2024-09-16T08:55:37.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.799+0000 7f7ee52ec640 1 -- 172.21.15.31:0/3857930794 wait complete. 2024-09-16T08:55:37.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 -- 172.21.15.31:0/2872419094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 msgr2=0x7fab78102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 --2- 172.21.15.31:0/2872419094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78102a70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fab740098b0 tx=0x7fab7402a4d0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 -- 172.21.15.31:0/2872419094 shutdown_connections 2024-09-16T08:55:37.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 --2- 172.21.15.31:0/2872419094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab78102fb0 0x7fab781053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 --2- 172.21.15.31:0/2872419094 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78102a70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 -- 172.21.15.31:0/2872419094 >> 172.21.15.31:0/2872419094 conn(0x7fab780fa280 msgr2=0x7fab780fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:37.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 -- 172.21.15.31:0/2872419094 shutdown_connections 2024-09-16T08:55:37.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.899+0000 7fab80310640 1 -- 172.21.15.31:0/2872419094 wait complete. 2024-09-16T08:55:37.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab80310640 1 Processor -- start 2024-09-16T08:55:37.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab80310640 1 -- start start 2024-09-16T08:55:37.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab80310640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab80310640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab78102fb0 0x7fab78195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab80310640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78196220 con 0x7fab78100680 2024-09-16T08:55:37.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab80310640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab78196390 con 0x7fab78102fb0 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab7e0ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab7e0ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36070/0 (socket says 172.21.15.31:36070) 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab7e0ac640 1 -- 172.21.15.31:0/2101557879 learned_addr learned my addr 172.21.15.31:0/2101557879 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.900+0000 7fab7d8ab640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab78102fb0 0x7fab78195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab7e0ac640 1 -- 172.21.15.31:0/2101557879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab78102fb0 msgr2=0x7fab78195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab7e0ac640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab78102fb0 0x7fab78195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab7e0ac640 1 -- 172.21.15.31:0/2101557879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74009560 con 0x7fab78100680 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab7e0ac640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78195710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab74004650 tx=0x7fab740055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab6f7fe640 1 -- 172.21.15.31:0/2101557879 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab74004aa0 con 0x7fab78100680 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab6f7fe640 1 -- 172.21.15.31:0/2101557879 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab74005d50 con 0x7fab78100680 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7819add0 con 0x7fab78100680 2024-09-16T08:55:37.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.901+0000 7fab6f7fe640 1 -- 172.21.15.31:0/2101557879 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab740039f0 con 0x7fab78100680 2024-09-16T08:55:37.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.902+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7819b260 con 0x7fab78100680 2024-09-16T08:55:37.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.903+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab78108830 con 0x7fab78100680 2024-09-16T08:55:37.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.903+0000 7fab6f7fe640 1 -- 172.21.15.31:0/2101557879 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fab74003c60 con 0x7fab78100680 2024-09-16T08:55:37.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.904+0000 7fab6f7fe640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fab6406cc50 0x7fab6406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:55:37.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.904+0000 7fab6f7fe640 1 -- 172.21.15.31:0/2101557879 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fab740ad270 con 0x7fab78100680 2024-09-16T08:55:37.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.904+0000 7fab7d8ab640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fab6406cc50 0x7fab6406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:55:37.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.904+0000 7fab7d8ab640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fab6406cc50 0x7fab6406f110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab78196c30 tx=0x7fab68005f90 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:55:37.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:37.909+0000 7fab6f7fe640 1 -- 172.21.15.31:0/2101557879 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fab74077710 con 0x7fab78100680 2024-09-16T08:55:38.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.202+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab781089c0 con 0x7fab78100680 2024-09-16T08:55:38.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.202+0000 7fab6f7fe640 1 -- 172.21.15.31:0/2101557879 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fab74031c00 con 0x7fab78100680 2024-09-16T08:55:38.203 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:55:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fab6406cc50 msgr2=0x7fab6406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fab6406cc50 0x7fab6406f110 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fab78196c30 tx=0x7fab68005f90 comp rx=0 tx=0).stop 2024-09-16T08:55:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 msgr2=0x7fab78195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:55:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78195710 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fab74004650 tx=0x7fab740055a0 comp rx=0 tx=0).stop 2024-09-16T08:55:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 shutdown_connections 2024-09-16T08:55:38.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fab6406cc50 0x7fab6406f110 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab78102fb0 0x7fab78195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 --2- 172.21.15.31:0/2101557879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fab78100680 0x7fab78195710 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:55:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 >> 172.21.15.31:0/2101557879 conn(0x7fab780fa280 msgr2=0x7fab780fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:55:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.204+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 shutdown_connections 2024-09-16T08:55:38.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:55:38.205+0000 7fab80310640 1 -- 172.21.15.31:0/2101557879 wait complete. 2024-09-16T08:55:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:38 smithi097 ceph-mon[35478]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:38 smithi097 ceph-mon[35478]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:38 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3679256498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:55:38.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:38 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:38.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:38 smithi031 ceph-mon[27844]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:38 smithi031 ceph-mon[27844]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:38 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3679256498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:55:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:38 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:39.412 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:39 smithi097 ceph-mon[35478]: from='client.24399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:39.412 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:39 smithi097 ceph-mon[35478]: pgmap v230: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:55:39.412 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:39 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2101557879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:55:39.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:39 smithi031 ceph-mon[27844]: from='client.24399 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:55:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:39 smithi031 ceph-mon[27844]: pgmap v230: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:55:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:39 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2101557879' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:55:40.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:40 smithi097 ceph-mon[35478]: pgmap v231: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:40.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:40 smithi031 ceph-mon[27844]: pgmap v231: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:43.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:43 smithi097 ceph-mon[35478]: pgmap v232: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:43.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:43 smithi031 ceph-mon[27844]: pgmap v232: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:45.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:45 smithi097 ceph-mon[35478]: pgmap v233: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:45.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:45 smithi031 ceph-mon[27844]: pgmap v233: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:47.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:47 smithi097 ceph-mon[35478]: pgmap v234: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:47.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:47 smithi031 ceph-mon[27844]: pgmap v234: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:49.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:49 smithi097 ceph-mon[35478]: pgmap v235: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:49 smithi031 ceph-mon[27844]: pgmap v235: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:51.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:51 smithi097 ceph-mon[35478]: pgmap v236: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:51.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:51 smithi031 ceph-mon[27844]: pgmap v236: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:53.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:53 smithi031 ceph-mon[27844]: pgmap v237: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:53.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:53.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:53 smithi097 ceph-mon[35478]: pgmap v237: 65 pgs: 65 active+clean; 586 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:55:53.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:55:55.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:55 smithi097 ceph-mon[35478]: pgmap v238: 65 pgs: 65 active+clean; 587 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T08:55:55.610 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:55 smithi031 ceph-mon[27844]: pgmap v238: 65 pgs: 65 active+clean; 587 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T08:55:57.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:57 smithi097 ceph-mon[35478]: pgmap v239: 65 pgs: 65 active+clean; 587 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T08:55:57.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:57 smithi031 ceph-mon[27844]: pgmap v239: 65 pgs: 65 active+clean; 587 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T08:55:59.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:55:59 smithi097 ceph-mon[35478]: pgmap v240: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T08:55:59.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:55:59 smithi031 ceph-mon[27844]: pgmap v240: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T08:56:00.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:00 smithi097 ceph-mon[35478]: pgmap v241: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T08:56:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:00 smithi031 ceph-mon[27844]: pgmap v241: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T08:56:03.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:03 smithi097 ceph-mon[35478]: pgmap v242: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T08:56:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:03 smithi031 ceph-mon[27844]: pgmap v242: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T08:56:05.147 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:05 smithi031 ceph-mon[27844]: pgmap v243: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T08:56:05.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:05 smithi097 ceph-mon[35478]: pgmap v243: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-09-16T08:56:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:07 smithi031 ceph-mon[27844]: Upgrade: Target is version 19.3.0-4917-g8293d73f (unknown) 2024-09-16T08:56:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:07 smithi031 ceph-mon[27844]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce'] 2024-09-16T08:56:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:56:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:07 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:07 smithi031 ceph-mon[27844]: Upgrade: Need to upgrade myself (mgr.smithi031.dcmqpv) 2024-09-16T08:56:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:07 smithi031 ceph-mon[27844]: pgmap v244: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T08:56:07.467 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:07.467 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:07 smithi097 ceph-mon[35478]: Upgrade: Target is version 19.3.0-4917-g8293d73f (unknown) 2024-09-16T08:56:07.467 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:07 smithi097 ceph-mon[35478]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, digests ['quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce'] 2024-09-16T08:56:07.467 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:56:07.467 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:07 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:07.467 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:07 smithi097 ceph-mon[35478]: Upgrade: Need to upgrade myself (mgr.smithi031.dcmqpv) 2024-09-16T08:56:07.467 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:07 smithi097 ceph-mon[35478]: pgmap v244: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T08:56:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.338+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1702722125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 msgr2=0x7fce18102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:08.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.338+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1702722125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 0x7fce18102aa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fce08009880 tx=0x7fce0802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.339 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:08 smithi031 ceph-mon[27844]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce on smithi097 2024-09-16T08:56:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.338+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1702722125 shutdown_connections 2024-09-16T08:56:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.338+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1702722125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 0x7fce181053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.338+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1702722125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 0x7fce18102aa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.338+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1702722125 >> 172.21.15.31:0/1702722125 conn(0x7fce180fa230 msgr2=0x7fce180fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.338+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1702722125 shutdown_connections 2024-09-16T08:56:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.339+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1702722125 wait complete. 2024-09-16T08:56:08.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.339+0000 7fce1d9d3640 1 Processor -- start 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce1d9d3640 1 -- start start 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce1d9d3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 0x7fce18195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce1d9d3640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 0x7fce18195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce1d9d3640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce18195f30 con 0x7fce181006b0 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce1d9d3640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce181960a0 con 0x7fce18102fe0 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce167fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 0x7fce18195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce167fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 0x7fce18195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:33360/0 (socket says 172.21.15.31:33360) 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce167fc640 1 -- 172.21.15.31:0/1149240503 learned_addr learned my addr 172.21.15.31:0/1149240503 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:08.342 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.340+0000 7fce16ffd640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 0x7fce18195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:08.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce167fc640 1 -- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 msgr2=0x7fce18195420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce167fc640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 0x7fce18195420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce167fc640 1 -- 172.21.15.31:0/1149240503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce08009560 con 0x7fce18102fe0 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce16ffd640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 0x7fce18195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce167fc640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 0x7fce18195960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00e970 tx=0x7fce0c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce1c9d1640 1 -- 172.21.15.31:0/1149240503 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce0c00cc90 con 0x7fce18102fe0 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce1c9d1640 1 -- 172.21.15.31:0/1149240503 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce0c00cdf0 con 0x7fce18102fe0 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce180fe500 con 0x7fce18102fe0 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce1c9d1640 1 -- 172.21.15.31:0/1149240503 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce0c010430 con 0x7fce18102fe0 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.341+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce180fead0 con 0x7fce18102fe0 2024-09-16T08:56:08.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.343+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce18108900 con 0x7fce18102fe0 2024-09-16T08:56:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.344+0000 7fce1c9d1640 1 -- 172.21.15.31:0/1149240503 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fce0c005020 con 0x7fce18102fe0 2024-09-16T08:56:08.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.344+0000 7fce1c9d1640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fce0006cc50 0x7fce0006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:08.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.345+0000 7fce16ffd640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fce0006cc50 0x7fce0006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:08.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.345+0000 7fce1c9d1640 1 -- 172.21.15.31:0/1149240503 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fce0c014070 con 0x7fce18102fe0 2024-09-16T08:56:08.346 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.345+0000 7fce16ffd640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fce0006cc50 0x7fce0006f110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce080315d0 tx=0x7fce08031400 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:08.351 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.350+0000 7fce1c9d1640 1 -- 172.21.15.31:0/1149240503 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fce0c056bc0 con 0x7fce18102fe0 2024-09-16T08:56:08.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:08 smithi097 ceph-mon[35478]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce on smithi097 2024-09-16T08:56:08.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.730+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce181039c0 con 0x7fce0006cc50 2024-09-16T08:56:08.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.732+0000 7fce1c9d1640 1 -- 172.21.15.31:0/1149240503 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fce181039c0 con 0x7fce0006cc50 2024-09-16T08:56:08.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fce0006cc50 msgr2=0x7fce0006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:08.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fce0006cc50 0x7fce0006f110 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fce080315d0 tx=0x7fce08031400 comp rx=0 tx=0).stop 2024-09-16T08:56:08.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 msgr2=0x7fce18195960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:08.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 0x7fce18195960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00e970 tx=0x7fce0c00ee30 comp rx=0 tx=0).stop 2024-09-16T08:56:08.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 shutdown_connections 2024-09-16T08:56:08.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fce0006cc50 0x7fce0006f110 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce18102fe0 0x7fce18195960 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00e970 tx=0x7fce0c00ee30 comp rx=0 tx=0).stop 2024-09-16T08:56:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 --2- 172.21.15.31:0/1149240503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fce181006b0 0x7fce18195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 >> 172.21.15.31:0/1149240503 conn(0x7fce180fa230 msgr2=0x7fce180fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 shutdown_connections 2024-09-16T08:56:08.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.735+0000 7fce1d9d3640 1 -- 172.21.15.31:0/1149240503 wait complete. 2024-09-16T08:56:08.749 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:56:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 -- 172.21.15.31:0/3062606886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec102fc0 msgr2=0x7f4eec1053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/3062606886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec102fc0 0x7f4eec1053b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8009880 tx=0x7f4ed802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 -- 172.21.15.31:0/3062606886 shutdown_connections 2024-09-16T08:56:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/3062606886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec102fc0 0x7f4eec1053b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/3062606886 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec100690 0x7f4eec102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 -- 172.21.15.31:0/3062606886 >> 172.21.15.31:0/3062606886 conn(0x7f4eec0fa230 msgr2=0x7f4eec0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:08.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 -- 172.21.15.31:0/3062606886 shutdown_connections 2024-09-16T08:56:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.867+0000 7f4ef2e73640 1 -- 172.21.15.31:0/3062606886 wait complete. 2024-09-16T08:56:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef2e73640 1 Processor -- start 2024-09-16T08:56:08.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef2e73640 1 -- start start 2024-09-16T08:56:08.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef2e73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec100690 0x7f4eec1955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef2e73640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec102fc0 0x7f4eec195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef2e73640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec1960d0 con 0x7f4eec100690 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef2e73640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec196240 con 0x7f4eec102fc0 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef0c0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec100690 0x7f4eec1955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef0c0f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec100690 0x7f4eec1955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55040/0 (socket says 172.21.15.31:55040) 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.869+0000 7f4ef0c0f640 1 -- 172.21.15.31:0/2832800375 learned_addr learned my addr 172.21.15.31:0/2832800375 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.870+0000 7f4ef0c0f640 1 -- 172.21.15.31:0/2832800375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec102fc0 msgr2=0x7f4eec195b00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.870+0000 7f4ef0c0f640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec102fc0 0x7f4eec195b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.870+0000 7f4ef0c0f640 1 -- 172.21.15.31:0/2832800375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8009560 con 0x7f4eec100690 2024-09-16T08:56:08.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.870+0000 7f4ef0c0f640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec100690 0x7f4eec1955c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000e9a0 tx=0x7f4ee000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.870+0000 7f4ee9ffb640 1 -- 172.21.15.31:0/2832800375 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee000cd50 con 0x7f4eec100690 2024-09-16T08:56:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.870+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec19ace0 con 0x7f4eec100690 2024-09-16T08:56:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.871+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec19b200 con 0x7f4eec100690 2024-09-16T08:56:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.871+0000 7f4ee9ffb640 1 -- 172.21.15.31:0/2832800375 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ee000ceb0 con 0x7f4eec100690 2024-09-16T08:56:08.872 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.871+0000 7f4ee9ffb640 1 -- 172.21.15.31:0/2832800375 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ee0010430 con 0x7f4eec100690 2024-09-16T08:56:08.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.873+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec108740 con 0x7f4eec100690 2024-09-16T08:56:08.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.873+0000 7f4ee9ffb640 1 -- 172.21.15.31:0/2832800375 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f4ee0010670 con 0x7f4eec100690 2024-09-16T08:56:08.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.875+0000 7f4ee9ffb640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4edc06cca0 0x7f4edc06f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:08.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.875+0000 7f4ee9ffb640 1 -- 172.21.15.31:0/2832800375 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4ee0014070 con 0x7f4eec100690 2024-09-16T08:56:08.876 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.875+0000 7f4eebfff640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4edc06cca0 0x7f4edc06f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:08.880 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.876+0000 7f4eebfff640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4edc06cca0 0x7f4edc06f160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8031540 tx=0x7f4ed8000bc0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:08.883 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:08.883+0000 7f4ee9ffb640 1 -- 172.21.15.31:0/2832800375 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f4ee005b680 con 0x7f4eec100690 2024-09-16T08:56:09.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.264+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eec19b530 con 0x7f4edc06cca0 2024-09-16T08:56:09.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.266+0000 7f4ee9ffb640 1 -- 172.21.15.31:0/2832800375 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f4eec19b530 con 0x7f4edc06cca0 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4edc06cca0 msgr2=0x7f4edc06f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4edc06cca0 0x7f4edc06f160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8031540 tx=0x7f4ed8000bc0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec100690 msgr2=0x7f4eec1955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec100690 0x7f4eec1955c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000e9a0 tx=0x7f4ee000eda0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 shutdown_connections 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f4edc06cca0 0x7f4edc06f160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec102fc0 0x7f4eec195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 --2- 172.21.15.31:0/2832800375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec100690 0x7f4eec1955c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 >> 172.21.15.31:0/2832800375 conn(0x7f4eec0fa230 msgr2=0x7f4eec0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 shutdown_connections 2024-09-16T08:56:09.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.269+0000 7f4ef2e73640 1 -- 172.21.15.31:0/2832800375 wait complete. 2024-09-16T08:56:09.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.360+0000 7f72e47ce640 1 -- 172.21.15.31:0/3390578468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 msgr2=0x7f72dc0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.360+0000 7f72e47ce640 1 --2- 172.21.15.31:0/3390578468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 0x7f72dc0fff90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009880 tx=0x7f72d802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:09 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:09.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:09 smithi031 ceph-mon[27844]: pgmap v245: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.360+0000 7f72e47ce640 1 -- 172.21.15.31:0/3390578468 shutdown_connections 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.360+0000 7f72e47ce640 1 --2- 172.21.15.31:0/3390578468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 0x7f72dc101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.360+0000 7f72e47ce640 1 --2- 172.21.15.31:0/3390578468 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 0x7f72dc0fff90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.360+0000 7f72e47ce640 1 -- 172.21.15.31:0/3390578468 >> 172.21.15.31:0/3390578468 conn(0x7f72dc0fb320 msgr2=0x7f72dc0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.361+0000 7f72e47ce640 1 -- 172.21.15.31:0/3390578468 shutdown_connections 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.361+0000 7f72e47ce640 1 -- 172.21.15.31:0/3390578468 wait complete. 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.361+0000 7f72e47ce640 1 Processor -- start 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.361+0000 7f72e47ce640 1 -- start start 2024-09-16T08:56:09.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e47ce640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 0x7f72dc197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e47ce640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 0x7f72dc197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e47ce640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc198340 con 0x7f72dc0ffb90 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e256a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 0x7f72dc197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e1d69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 0x7f72dc197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e1d69640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 0x7f72dc197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:33380/0 (socket says 172.21.15.31:33380) 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e1d69640 1 -- 172.21.15.31:0/2543971272 learned_addr learned my addr 172.21.15.31:0/2543971272 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e256a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 0x7f72dc197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55066/0 (socket says 172.21.15.31:55066) 2024-09-16T08:56:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.362+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc1984b0 con 0x7f72dc100d90 2024-09-16T08:56:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72e1d69640 1 -- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 msgr2=0x7f72dc197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72e1d69640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 0x7f72dc197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72e1d69640 1 -- 172.21.15.31:0/2543971272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d8009560 con 0x7f72dc100d90 2024-09-16T08:56:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72e1d69640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 0x7f72dc197d70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00e970 tx=0x7f72cc00ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72d37fe640 1 -- 172.21.15.31:0/2543971272 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72cc00cc90 con 0x7f72dc100d90 2024-09-16T08:56:09.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72d37fe640 1 -- 172.21.15.31:0/2543971272 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72cc00cdf0 con 0x7f72dc100d90 2024-09-16T08:56:09.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc19cf50 con 0x7f72dc100d90 2024-09-16T08:56:09.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.363+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc19d490 con 0x7f72dc100d90 2024-09-16T08:56:09.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.364+0000 7f72d37fe640 1 -- 172.21.15.31:0/2543971272 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72cc010430 con 0x7f72dc100d90 2024-09-16T08:56:09.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.365+0000 7f72d37fe640 1 -- 172.21.15.31:0/2543971272 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f72cc005000 con 0x7f72dc100d90 2024-09-16T08:56:09.366 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.365+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc0fff90 con 0x7f72dc100d90 2024-09-16T08:56:09.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.366+0000 7f72d37fe640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f72c806cc50 0x7f72c806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:09.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.366+0000 7f72e256a640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f72c806cc50 0x7f72c806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:09.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.366+0000 7f72d37fe640 1 -- 172.21.15.31:0/2543971272 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f72cc014070 con 0x7f72dc100d90 2024-09-16T08:56:09.367 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.367+0000 7f72e256a640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f72c806cc50 0x7f72c806f110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009230 tx=0x7f72d80314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:09.372 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.371+0000 7f72d37fe640 1 -- 172.21.15.31:0/2543971272 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f72cc056af0 con 0x7f72dc100d90 2024-09-16T08:56:09.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:09 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:09.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:09 smithi097 ceph-mon[35478]: pgmap v245: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T08:56:09.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.569+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f72dc02c550 con 0x7f72c806cc50 2024-09-16T08:56:09.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.586+0000 7f72d37fe640 1 -- 172.21.15.31:0/2543971272 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f72dc02c550 con 0x7f72c806cc50 2024-09-16T08:56:09.587 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:56:09.587 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (5m) 57s ago 7m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:56:09.587 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (7m) 57s ago 7m 7528k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 66d87845da99 2024-09-16T08:56:09.587 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (6m) 68s ago 6m 7440k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 edd14d48bff9 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (7m) 57s ago 7m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (6m) 68s ago 6m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (5m) 57s ago 7m 83.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (88s) 57s ago 87s 11.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (81s) 57s ago 80s 18.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (85s) 68s ago 84s 18.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (78s) 68s ago 78s 13.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:9283,8765,8443 running (9m) 57s ago 8m 486M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 1c78052ec7c9 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (6m) 68s ago 6m 441M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b82e4c86f40c 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (9m) 57s ago 9m 47.7M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (6m) 68s ago 6m 35.3M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (7m) 57s ago 7m 15.3M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:56:09.588 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (6m) 68s ago 6m 16.1M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:56:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 57s ago 5m 47.3M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:56:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 57s ago 4m 54.9M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:56:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 57s ago 4m 46.6M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:56:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (3m) 68s ago 3m 51.5M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:56:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (3m) 68s ago 3m 44.3M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:56:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (3m) 68s ago 2m 44.0M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:56:09.589 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 57s ago 7m 60.0M - 2.43.0 a07b618ecd1d b0b74488871e 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f72c806cc50 msgr2=0x7f72c806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f72c806cc50 0x7f72c806f110 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009230 tx=0x7f72d80314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 msgr2=0x7f72dc197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 0x7f72dc197d70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00e970 tx=0x7f72cc00ed70 comp rx=0 tx=0).stop 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 shutdown_connections 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f72c806cc50 0x7f72c806f110 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72dc100d90 0x7f72dc197d70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 --2- 172.21.15.31:0/2543971272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72dc0ffb90 0x7f72dc197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 >> 172.21.15.31:0/2543971272 conn(0x7f72dc0fb320 msgr2=0x7f72dc072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 shutdown_connections 2024-09-16T08:56:09.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.589+0000 7f72e47ce640 1 -- 172.21.15.31:0/2543971272 wait complete. 2024-09-16T08:56:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.677+0000 7fb03e4e4640 1 -- 172.21.15.31:0/1440926740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038102fb0 msgr2=0x7fb0381053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.677+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/1440926740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038102fb0 0x7fb0381053a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098b0 tx=0x7fb02c02a4d0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.678+0000 7fb03e4e4640 1 -- 172.21.15.31:0/1440926740 shutdown_connections 2024-09-16T08:56:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.678+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/1440926740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038102fb0 0x7fb0381053a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.678+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/1440926740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb038100680 0x7fb038102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.678+0000 7fb03e4e4640 1 -- 172.21.15.31:0/1440926740 >> 172.21.15.31:0/1440926740 conn(0x7fb0380fa280 msgr2=0x7fb0380fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:09.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.678+0000 7fb03e4e4640 1 -- 172.21.15.31:0/1440926740 shutdown_connections 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.678+0000 7fb03e4e4640 1 -- 172.21.15.31:0/1440926740 wait complete. 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.679+0000 7fb03e4e4640 1 Processor -- start 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.679+0000 7fb03e4e4640 1 -- start start 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.679+0000 7fb03e4e4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038100680 0x7fb0381956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.679+0000 7fb03e4e4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb038102fb0 0x7fb038195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.679+0000 7fb03e4e4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb038196200 con 0x7fb038100680 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.679+0000 7fb03e4e4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb038196370 con 0x7fb038102fb0 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.680+0000 7fb037fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038100680 0x7fb0381956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:09.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.680+0000 7fb0377fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb038102fb0 0x7fb038195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.680+0000 7fb037fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038100680 0x7fb0381956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55088/0 (socket says 172.21.15.31:55088) 2024-09-16T08:56:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.680+0000 7fb037fff640 1 -- 172.21.15.31:0/2846447331 learned_addr learned my addr 172.21.15.31:0/2846447331 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.680+0000 7fb037fff640 1 -- 172.21.15.31:0/2846447331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb038102fb0 msgr2=0x7fb038195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.680+0000 7fb037fff640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb038102fb0 0x7fb038195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.680+0000 7fb037fff640 1 -- 172.21.15.31:0/2846447331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009560 con 0x7fb038100680 2024-09-16T08:56:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.681+0000 7fb037fff640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038100680 0x7fb0381956f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9a0 tx=0x7fb02800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.681+0000 7fb0357fa640 1 -- 172.21.15.31:0/2846447331 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02800cc90 con 0x7fb038100680 2024-09-16T08:56:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.681+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03819ae10 con 0x7fb038100680 2024-09-16T08:56:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.681+0000 7fb0357fa640 1 -- 172.21.15.31:0/2846447331 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb02800cdf0 con 0x7fb038100680 2024-09-16T08:56:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.681+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03819b2d0 con 0x7fb038100680 2024-09-16T08:56:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.681+0000 7fb0357fa640 1 -- 172.21.15.31:0/2846447331 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb02800cc90 con 0x7fb038100680 2024-09-16T08:56:09.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.682+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb024005280 con 0x7fb038100680 2024-09-16T08:56:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.683+0000 7fb0357fa640 1 -- 172.21.15.31:0/2846447331 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb028018690 con 0x7fb038100680 2024-09-16T08:56:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.683+0000 7fb0357fa640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb02006cd10 0x7fb02006f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.683+0000 7fb0357fa640 1 -- 172.21.15.31:0/2846447331 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb028014070 con 0x7fb038100680 2024-09-16T08:56:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.683+0000 7fb0377fe640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb02006cd10 0x7fb02006f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:09.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.684+0000 7fb0377fe640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb02006cd10 0x7fb02006f1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009880 tx=0x7fb02c0314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:09.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.688+0000 7fb0357fa640 1 -- 172.21.15.31:0/2846447331 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb028056df0 con 0x7fb038100680 2024-09-16T08:56:09.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.931+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb024005800 con 0x7fb038100680 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.932+0000 7fb0357fa640 1 -- 172.21.15.31:0/2846447331 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7fb028056f90 con 0x7fb038100680 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:56:09.933 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 14 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:56:09.934 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:56:09.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb02006cd10 msgr2=0x7fb02006f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb02006cd10 0x7fb02006f1d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009880 tx=0x7fb02c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038100680 msgr2=0x7fb0381956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038100680 0x7fb0381956f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb02800e9a0 tx=0x7fb02800ee60 comp rx=0 tx=0).stop 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 shutdown_connections 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb02006cd10 0x7fb02006f1d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb038102fb0 0x7fb038195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 --2- 172.21.15.31:0/2846447331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb038100680 0x7fb0381956f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 >> 172.21.15.31:0/2846447331 conn(0x7fb0380fa280 msgr2=0x7fb0380fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 shutdown_connections 2024-09-16T08:56:09.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:09.935+0000 7fb03e4e4640 1 -- 172.21.15.31:0/2846447331 wait complete. 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.023+0000 7f855aba2640 1 -- 172.21.15.31:0/3751597664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 msgr2=0x7f8554102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.023+0000 7f855aba2640 1 --2- 172.21.15.31:0/3751597664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 0x7f8554102ad0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8548009880 tx=0x7f854802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.024+0000 7f855aba2640 1 -- 172.21.15.31:0/3751597664 shutdown_connections 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.024+0000 7f855aba2640 1 --2- 172.21.15.31:0/3751597664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 0x7f8554105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.024+0000 7f855aba2640 1 --2- 172.21.15.31:0/3751597664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 0x7f8554102ad0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.024+0000 7f855aba2640 1 -- 172.21.15.31:0/3751597664 >> 172.21.15.31:0/3751597664 conn(0x7f85540fa280 msgr2=0x7f85540fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.024+0000 7f855aba2640 1 -- 172.21.15.31:0/3751597664 shutdown_connections 2024-09-16T08:56:10.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.024+0000 7f855aba2640 1 -- 172.21.15.31:0/3751597664 wait complete. 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.024+0000 7f855aba2640 1 Processor -- start 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f855aba2640 1 -- start start 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f855aba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 0x7f8554195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f855aba2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 0x7f8554195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f855aba2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8554196180 con 0x7f85541006e0 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f855aba2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85541962f0 con 0x7f8554103010 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f8553fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 0x7f8554195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f8553fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 0x7f8554195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:33416/0 (socket says 172.21.15.31:33416) 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f8553fff640 1 -- 172.21.15.31:0/3405028902 learned_addr learned my addr 172.21.15.31:0/3405028902 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f855893e640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 0x7f8554195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.027 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f8553fff640 1 -- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 msgr2=0x7f8554195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f8553fff640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 0x7f8554195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f8553fff640 1 -- 172.21.15.31:0/3405028902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548009560 con 0x7f8554103010 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.025+0000 7f855893e640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 0x7f8554195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.026+0000 7f8553fff640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 0x7f8554195bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f854000e970 tx=0x7f854000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.026+0000 7f8551ffb640 1 -- 172.21.15.31:0/3405028902 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854000cc90 con 0x7f8554103010 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.026+0000 7f8551ffb640 1 -- 172.21.15.31:0/3405028902 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f854000cdf0 con 0x7f8554103010 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.026+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855419ad90 con 0x7f8554103010 2024-09-16T08:56:10.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.026+0000 7f8551ffb640 1 -- 172.21.15.31:0/3405028902 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8540010430 con 0x7f8554103010 2024-09-16T08:56:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.029+0000 7f8551ffb640 1 -- 172.21.15.31:0/3405028902 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8540005000 con 0x7f8554103010 2024-09-16T08:56:10.029 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.029+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855419b200 con 0x7f8554103010 2024-09-16T08:56:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.029+0000 7f8551ffb640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f854406cc50 0x7f854406f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.030+0000 7f8551ffb640 1 -- 172.21.15.31:0/3405028902 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8540014070 con 0x7f8554103010 2024-09-16T08:56:10.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.030+0000 7f855893e640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f854406cc50 0x7f854406f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.030+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8554100ac0 con 0x7f8554103010 2024-09-16T08:56:10.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.031+0000 7f855893e640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f854406cc50 0x7f854406f110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f85480314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:10.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.040+0000 7f8551ffb640 1 -- 172.21.15.31:0/3405028902 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8540091060 con 0x7f8554103010 2024-09-16T08:56:10.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.322+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f85541087c0 con 0x7f8554103010 2024-09-16T08:56:10.323 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:10 smithi031 ceph-mon[27844]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:10.323 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:10 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2846447331' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.322+0000 7f8551ffb640 1 -- 172.21.15.31:0/3405028902 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f85400564c0 con 0x7f8554103010 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:56:10.324 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:56:10.325 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:10.326 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:10.327 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:56:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:10.327 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:10.327 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:10.327 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f854406cc50 msgr2=0x7f854406f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f854406cc50 0x7f854406f110 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f85480314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 msgr2=0x7f8554195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 0x7f8554195bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f854000e970 tx=0x7f854000ee30 comp rx=0 tx=0).stop 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 shutdown_connections 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f854406cc50 0x7f854406f110 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8554103010 0x7f8554195bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 --2- 172.21.15.31:0/3405028902 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85541006e0 0x7f8554195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 >> 172.21.15.31:0/3405028902 conn(0x7f85540fa280 msgr2=0x7f85540fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 shutdown_connections 2024-09-16T08:56:10.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.326+0000 7f855aba2640 1 -- 172.21.15.31:0/3405028902 wait complete. 2024-09-16T08:56:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.452+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/1443058253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fe470 msgr2=0x7fdc840fe870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.452+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/1443058253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fe470 0x7fdc840fe870 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdc800097e0 tx=0x7fdc8002a3f0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.453 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.452+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/1443058253 shutdown_connections 2024-09-16T08:56:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.452+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/1443058253 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc840fedb0 0x7fdc840ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.452+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/1443058253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fe470 0x7fdc840fe870 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.452+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/1443058253 >> 172.21.15.31:0/1443058253 conn(0x7fdc840fa280 msgr2=0x7fdc840fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.453+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/1443058253 shutdown_connections 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.453+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/1443058253 wait complete. 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc8c0bc640 1 Processor -- start 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc8c0bc640 1 -- start start 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc8c0bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc840fe470 0x7fdc84197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc8c0bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fedb0 0x7fdc84197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc8c0bc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc84198370 con 0x7fdc840fe470 2024-09-16T08:56:10.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc8c0bc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc841984e0 con 0x7fdc840fedb0 2024-09-16T08:56:10.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc89657640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fedb0 0x7fdc84197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc89657640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fedb0 0x7fdc84197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:33428/0 (socket says 172.21.15.31:33428) 2024-09-16T08:56:10.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc89657640 1 -- 172.21.15.31:0/4016195877 learned_addr learned my addr 172.21.15.31:0/4016195877 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:10.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.454+0000 7fdc89e58640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc840fe470 0x7fdc84197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc89657640 1 -- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc840fe470 msgr2=0x7fdc84197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc89657640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc840fe470 0x7fdc84197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc89657640 1 -- 172.21.15.31:0/4016195877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc74009560 con 0x7fdc840fedb0 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc89657640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fedb0 0x7fdc84197da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e940 tx=0x7fdc7400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc7affd640 1 -- 172.21.15.31:0/4016195877 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc7400cd50 con 0x7fdc840fedb0 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc7affd640 1 -- 172.21.15.31:0/4016195877 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc740035f0 con 0x7fdc840fedb0 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc7affd640 1 -- 172.21.15.31:0/4016195877 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc740044e0 con 0x7fdc840fedb0 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc80009490 con 0x7fdc840fedb0 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.455+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc8419d310 con 0x7fdc840fedb0 2024-09-16T08:56:10.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.457+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc840fe870 con 0x7fdc840fedb0 2024-09-16T08:56:10.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.458+0000 7fdc7affd640 1 -- 172.21.15.31:0/4016195877 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fdc74003760 con 0x7fdc840fedb0 2024-09-16T08:56:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.458+0000 7fdc7affd640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc7006cc50 0x7fdc7006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.458+0000 7fdc7affd640 1 -- 172.21.15.31:0/4016195877 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdc74014070 con 0x7fdc840fedb0 2024-09-16T08:56:10.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.458+0000 7fdc89e58640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc7006cc50 0x7fdc7006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.459+0000 7fdc89e58640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc7006cc50 0x7fdc7006f110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc800308d0 tx=0x7fdc80005ab0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:10.467 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.467+0000 7fdc7affd640 1 -- 172.21.15.31:0/4016195877 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fdc74056ba0 con 0x7fdc840fedb0 2024-09-16T08:56:10.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:10 smithi097 ceph-mon[35478]: from='client.24407 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:10.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:10 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2846447331' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:56:10.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.657+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc8402c540 con 0x7fdc7006cc50 2024-09-16T08:56:10.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.658+0000 7fdc7affd640 1 -- 172.21.15.31:0/4016195877 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fdc8402c540 con 0x7fdc7006cc50 2024-09-16T08:56:10.659 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:56:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:56:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:56:10.659 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:56:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-09-16T08:56:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "0/23 daemons upgraded", 2024-09-16T08:56:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce image on host smithi097", 2024-09-16T08:56:10.660 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:56:10.660 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:56:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.660+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc7006cc50 msgr2=0x7fdc7006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.660+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc7006cc50 0x7fdc7006f110 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdc800308d0 tx=0x7fdc80005ab0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.660+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fedb0 msgr2=0x7fdc84197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.660+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fedb0 0x7fdc84197da0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc7400e940 tx=0x7fdc7400ee00 comp rx=0 tx=0).stop 2024-09-16T08:56:10.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.661+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 shutdown_connections 2024-09-16T08:56:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.661+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fdc7006cc50 0x7fdc7006f110 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.661+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc840fedb0 0x7fdc84197da0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.661+0000 7fdc8c0bc640 1 --2- 172.21.15.31:0/4016195877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc840fe470 0x7fdc84197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.661+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 >> 172.21.15.31:0/4016195877 conn(0x7fdc840fa280 msgr2=0x7fdc84106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.661+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 shutdown_connections 2024-09-16T08:56:10.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.661+0000 7fdc8c0bc640 1 -- 172.21.15.31:0/4016195877 wait complete. 2024-09-16T08:56:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.787+0000 7fc396f8a640 1 -- 172.21.15.31:0/1550207175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3900ffca0 msgr2=0x7fc390100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.787+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1550207175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3900ffca0 0x7fc390100120 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009880 tx=0x7fc37c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.787+0000 7fc396f8a640 1 -- 172.21.15.31:0/1550207175 shutdown_connections 2024-09-16T08:56:10.788 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.787+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1550207175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3900ffca0 0x7fc390100120 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.787+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1550207175 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 0x7fc3900feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.787+0000 7fc396f8a640 1 -- 172.21.15.31:0/1550207175 >> 172.21.15.31:0/1550207175 conn(0x7fc3900fa250 msgr2=0x7fc3900fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.787+0000 7fc396f8a640 1 -- 172.21.15.31:0/1550207175 shutdown_connections 2024-09-16T08:56:10.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.788+0000 7fc396f8a640 1 -- 172.21.15.31:0/1550207175 wait complete. 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.788+0000 7fc396f8a640 1 Processor -- start 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.788+0000 7fc396f8a640 1 -- start start 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc396f8a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 0x7fc390068fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc396f8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3900ffca0 0x7fc39019a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc396f8a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39019a940 con 0x7fc3900ffca0 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc396f8a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39019aa80 con 0x7fc3900feaa0 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc394d26640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 0x7fc390068fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc394d26640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 0x7fc390068fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:33444/0 (socket says 172.21.15.31:33444) 2024-09-16T08:56:10.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc394d26640 1 -- 172.21.15.31:0/1338459384 learned_addr learned my addr 172.21.15.31:0/1338459384 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc394d26640 1 -- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3900ffca0 msgr2=0x7fc39019a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:10.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc394d26640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3900ffca0 0x7fc39019a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.789+0000 7fc394d26640 1 -- 172.21.15.31:0/1338459384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c009560 con 0x7fc3900feaa0 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.790+0000 7fc394d26640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 0x7fc390068fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc38400e940 tx=0x7fc38400ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.790+0000 7fc38dffb640 1 -- 172.21.15.31:0/1338459384 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc38400cc90 con 0x7fc3900feaa0 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.790+0000 7fc38dffb640 1 -- 172.21.15.31:0/1338459384 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3840044f0 con 0x7fc3900feaa0 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.790+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39019ad10 con 0x7fc3900feaa0 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.790+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39019b2e0 con 0x7fc3900feaa0 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.790+0000 7fc38dffb640 1 -- 172.21.15.31:0/1338459384 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc384010430 con 0x7fc3900feaa0 2024-09-16T08:56:10.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.792+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc374005280 con 0x7fc3900feaa0 2024-09-16T08:56:10.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.792+0000 7fc38dffb640 1 -- 172.21.15.31:0/1338459384 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fc384010660 con 0x7fc3900feaa0 2024-09-16T08:56:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.793+0000 7fc38dffb640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc38006cc50 0x7fc38006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.793+0000 7fc38dffb640 1 -- 172.21.15.31:0/1338459384 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc384014070 con 0x7fc3900feaa0 2024-09-16T08:56:10.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.793+0000 7fc38ffff640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc38006cc50 0x7fc38006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:10.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.794+0000 7fc38ffff640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc38006cc50 0x7fc38006f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc37c02a9c0 tx=0x7fc37c000bc0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:10.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:10.798+0000 7fc38dffb640 1 -- 172.21.15.31:0/1338459384 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fc384056aa0 con 0x7fc3900feaa0 2024-09-16T08:56:11.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.193+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc3740050f0 con 0x7fc3900feaa0 2024-09-16T08:56:11.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.193+0000 7fc38dffb640 1 -- 172.21.15.31:0/1338459384 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc38405a0c0 con 0x7fc3900feaa0 2024-09-16T08:56:11.194 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:56:11.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc38006cc50 msgr2=0x7fc38006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:11.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc38006cc50 0x7fc38006f110 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc37c02a9c0 tx=0x7fc37c000bc0 comp rx=0 tx=0).stop 2024-09-16T08:56:11.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 msgr2=0x7fc390068fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:11.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 0x7fc390068fa0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc38400e940 tx=0x7fc38400ee00 comp rx=0 tx=0).stop 2024-09-16T08:56:11.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 shutdown_connections 2024-09-16T08:56:11.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fc38006cc50 0x7fc38006f110 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:11.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc3900ffca0 0x7fc39019a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:11.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 --2- 172.21.15.31:0/1338459384 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3900feaa0 0x7fc390068fa0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:11.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.195+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 >> 172.21.15.31:0/1338459384 conn(0x7fc3900fa250 msgr2=0x7fc390103320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:11.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.196+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 shutdown_connections 2024-09-16T08:56:11.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:11.196+0000 7fc396f8a640 1 -- 172.21.15.31:0/1338459384 wait complete. 2024-09-16T08:56:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:11 smithi031 ceph-mon[27844]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:11 smithi031 ceph-mon[27844]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:11 smithi031 ceph-mon[27844]: pgmap v246: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:56:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:11 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3405028902' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:56:11.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:11 smithi097 ceph-mon[35478]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:11.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:11 smithi097 ceph-mon[35478]: from='client.24413 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:11.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:11 smithi097 ceph-mon[35478]: pgmap v246: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:56:11.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:11 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3405028902' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:56:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:12 smithi031 ceph-mon[27844]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:12 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1338459384' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:56:12.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:12 smithi097 ceph-mon[35478]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:12.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:12 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1338459384' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:56:13.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:13 smithi031 ceph-mon[27844]: pgmap v247: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:56:13.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:13 smithi097 ceph-mon[35478]: pgmap v247: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:56:15.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:15 smithi097 ceph-mon[35478]: pgmap v248: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:56:15.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:15 smithi031 ceph-mon[27844]: pgmap v248: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-09-16T08:56:17.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:17 smithi097 ceph-mon[35478]: pgmap v249: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:17.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:17 smithi031 ceph-mon[27844]: pgmap v249: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:19.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:19 smithi097 ceph-mon[35478]: pgmap v250: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:19.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:19 smithi031 ceph-mon[27844]: pgmap v250: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:21.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:21 smithi097 ceph-mon[35478]: pgmap v251: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:21.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:21 smithi031 ceph-mon[27844]: pgmap v251: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:23.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:23 smithi031 ceph-mon[27844]: pgmap v252: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:23.425 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:23 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:23.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:23 smithi097 ceph-mon[35478]: pgmap v252: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:23.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:23 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:24.019 INFO:tasks.workunit.client.1.smithi097.stderr:Updating files: 88% (11185/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-16T08:56:24.363 INFO:tasks.workunit.client.1.smithi097.stderr:Note: switching to '8293d73f8690540e843a81caec373f9cc29cf705'. 2024-09-16T08:56:24.363 INFO:tasks.workunit.client.1.smithi097.stderr: 2024-09-16T08:56:24.363 INFO:tasks.workunit.client.1.smithi097.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T08:56:24.363 INFO:tasks.workunit.client.1.smithi097.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T08:56:24.363 INFO:tasks.workunit.client.1.smithi097.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T08:56:24.363 INFO:tasks.workunit.client.1.smithi097.stderr: 2024-09-16T08:56:24.363 INFO:tasks.workunit.client.1.smithi097.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr: 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr: git switch -c 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr: 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr:Or undo this operation with: 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr: 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr: git switch - 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr: 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr: 2024-09-16T08:56:24.364 INFO:tasks.workunit.client.1.smithi097.stderr:HEAD is now at 8293d73f869 Merge branch 'wip-aclamk-cbt-combined' of https://github.com/aclamk/ceph into wip-guits-main-2024-09-13-1248 2024-09-16T08:56:24.368 DEBUG:teuthology.orchestra.run.smithi097:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-16T08:56:24.431 INFO:tasks.workunit.client.1.smithi097.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T08:56:24.433 INFO:tasks.workunit.client.1.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-16T08:56:24.433 INFO:tasks.workunit.client.1.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T08:56:25.086 INFO:tasks.workunit.client.1.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T08:56:25.163 INFO:tasks.workunit.client.1.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T08:56:25.204 INFO:tasks.workunit.client.1.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-16T08:56:25.205 INFO:tasks.workunit.client.1.smithi097.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-16T08:56:25.206 INFO:tasks.workunit.client.1.smithi097.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T08:56:25.246 INFO:tasks.workunit.client.1.smithi097.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-16T08:56:25.250 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T08:56:25.250 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-16T08:56:25.306 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-09-16T08:56:25.307 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-16T08:56:25.307 DEBUG:teuthology.orchestra.run.smithi097:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8293d73f8690540e843a81caec373f9cc29cf705 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-09-16T08:56:25.369 INFO:tasks.workunit.client.1.smithi097.stderr:+ mkdir -p fsstress 2024-09-16T08:56:25.371 INFO:tasks.workunit.client.1.smithi097.stderr:+ pushd fsstress 2024-09-16T08:56:25.372 INFO:tasks.workunit.client.1.smithi097.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T08:56:25.372 INFO:tasks.workunit.client.1.smithi097.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-16T08:56:25.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:25 smithi097 ceph-mon[35478]: pgmap v253: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:25.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:25 smithi031 ceph-mon[27844]: pgmap v253: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:25.895 INFO:tasks.workunit.client.1.smithi097.stderr:+ tar xzf ltp-full.tgz 2024-09-16T08:56:26.227 INFO:tasks.workunit.client.0.smithi031.stderr:Updating files: 90% (11456/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-16T08:56:26.717 INFO:tasks.workunit.client.0.smithi031.stderr:Note: switching to '8293d73f8690540e843a81caec373f9cc29cf705'. 2024-09-16T08:56:26.717 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T08:56:26.717 INFO:tasks.workunit.client.0.smithi031.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T08:56:26.717 INFO:tasks.workunit.client.0.smithi031.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr: git switch -c 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr:Or undo this operation with: 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr: git switch - 2024-09-16T08:56:26.718 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T08:56:26.719 INFO:tasks.workunit.client.0.smithi031.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T08:56:26.719 INFO:tasks.workunit.client.0.smithi031.stderr: 2024-09-16T08:56:26.719 INFO:tasks.workunit.client.0.smithi031.stderr:HEAD is now at 8293d73f869 Merge branch 'wip-aclamk-cbt-combined' of https://github.com/aclamk/ceph into wip-guits-main-2024-09-13-1248 2024-09-16T08:56:26.724 DEBUG:teuthology.orchestra.run.smithi031:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T08:56:26.750 INFO:tasks.workunit.client.0.smithi031.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T08:56:26.753 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T08:56:26.753 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T08:56:27.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:27 smithi097 ceph-mon[35478]: pgmap v254: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:27.563 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T08:56:27.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:27 smithi031 ceph-mon[27844]: pgmap v254: 65 pgs: 65 active+clean; 589 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-16T08:56:27.639 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T08:56:27.683 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T08:56:27.685 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T08:56:27.685 INFO:tasks.workunit.client.0.smithi031.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T08:56:27.739 INFO:tasks.workunit.client.0.smithi031.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T08:56:27.743 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T08:56:27.743 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T08:56:27.800 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-16T08:56:27.802 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-16T08:56:27.802 DEBUG:teuthology.orchestra.run.smithi031:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8293d73f8690540e843a81caec373f9cc29cf705 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-16T08:56:27.863 INFO:tasks.workunit.client.0.smithi031.stderr:+ mkdir -p fsstress 2024-09-16T08:56:27.865 INFO:tasks.workunit.client.0.smithi031.stderr:+ pushd fsstress 2024-09-16T08:56:27.865 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T08:56:27.865 INFO:tasks.workunit.client.0.smithi031.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-16T08:56:28.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:28 smithi097 ceph-mon[35478]: pgmap v255: 65 pgs: 65 active+clean; 5.7 MiB data, 175 MiB used, 536 GiB / 536 GiB avail; 440 KiB/s wr, 0 op/s 2024-09-16T08:56:28.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:28 smithi031 ceph-mon[27844]: pgmap v255: 65 pgs: 65 active+clean; 5.7 MiB data, 175 MiB used, 536 GiB / 536 GiB avail; 440 KiB/s wr, 0 op/s 2024-09-16T08:56:28.764 INFO:tasks.workunit.client.0.smithi031.stderr:+ tar xzf ltp-full.tgz 2024-09-16T08:56:31.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:31 smithi097 ceph-mon[35478]: pgmap v256: 65 pgs: 65 active+clean; 5.7 MiB data, 180 MiB used, 536 GiB / 536 GiB avail; 440 KiB/s wr, 0 op/s 2024-09-16T08:56:31.517 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:31 smithi031 ceph-mon[27844]: pgmap v256: 65 pgs: 65 active+clean; 5.7 MiB data, 180 MiB used, 536 GiB / 536 GiB avail; 440 KiB/s wr, 0 op/s 2024-09-16T08:56:33.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:33 smithi097 ceph-mon[35478]: pgmap v257: 65 pgs: 65 active+clean; 5.7 MiB data, 180 MiB used, 536 GiB / 536 GiB avail; 440 KiB/s wr, 0 op/s 2024-09-16T08:56:33.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:33 smithi031 ceph-mon[27844]: pgmap v257: 65 pgs: 65 active+clean; 5.7 MiB data, 180 MiB used, 536 GiB / 536 GiB avail; 440 KiB/s wr, 0 op/s 2024-09-16T08:56:35.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:35 smithi097 ceph-mon[35478]: pgmap v258: 65 pgs: 65 active+clean; 50 MiB data, 320 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 273 op/s 2024-09-16T08:56:35.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:35 smithi031 ceph-mon[27844]: pgmap v258: 65 pgs: 65 active+clean; 50 MiB data, 320 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 273 op/s 2024-09-16T08:56:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:37 smithi097 ceph-mon[35478]: pgmap v259: 65 pgs: 65 active+clean; 57 MiB data, 348 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 323 op/s 2024-09-16T08:56:37.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:37 smithi031 ceph-mon[27844]: pgmap v259: 65 pgs: 65 active+clean; 57 MiB data, 348 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 323 op/s 2024-09-16T08:56:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:38 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:38.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:38 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:39.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:39 smithi097 ceph-mon[35478]: pgmap v260: 65 pgs: 65 active+clean; 78 MiB data, 411 MiB used, 536 GiB / 536 GiB avail; 6.7 MiB/s wr, 404 op/s 2024-09-16T08:56:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:39 smithi097 ceph-mon[35478]: Upgrade: Updating mgr.smithi097.mpashf 2024-09-16T08:56:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:56:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:56:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:39 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:56:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:39 smithi097 ceph-mon[35478]: Deploying daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:56:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:39 smithi031 ceph-mon[27844]: pgmap v260: 65 pgs: 65 active+clean; 78 MiB data, 411 MiB used, 536 GiB / 536 GiB avail; 6.7 MiB/s wr, 404 op/s 2024-09-16T08:56:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:39 smithi031 ceph-mon[27844]: Upgrade: Updating mgr.smithi097.mpashf 2024-09-16T08:56:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:56:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:56:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:39 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:56:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:39 smithi031 ceph-mon[27844]: Deploying daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:56:40.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:40 smithi097 ceph-mon[35478]: pgmap v261: 65 pgs: 65 active+clean; 82 MiB data, 446 MiB used, 536 GiB / 536 GiB avail; 6.6 MiB/s wr, 415 op/s 2024-09-16T08:56:40.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:40 smithi031 ceph-mon[27844]: pgmap v261: 65 pgs: 65 active+clean; 82 MiB data, 446 MiB used, 536 GiB / 536 GiB avail; 6.6 MiB/s wr, 415 op/s 2024-09-16T08:56:41.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 -- 172.21.15.31:0/3978067587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 msgr2=0x7f5ce8102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 --2- 172.21.15.31:0/3978067587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce8102a70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80098b0 tx=0x7f5cd802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 -- 172.21.15.31:0/3978067587 shutdown_connections 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 --2- 172.21.15.31:0/3978067587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ce8102fb0 0x7f5ce81053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 --2- 172.21.15.31:0/3978067587 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce8102a70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 -- 172.21.15.31:0/3978067587 >> 172.21.15.31:0/3978067587 conn(0x7f5ce80fa280 msgr2=0x7f5ce80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 -- 172.21.15.31:0/3978067587 shutdown_connections 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.298+0000 7f5cedec1640 1 -- 172.21.15.31:0/3978067587 wait complete. 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.299+0000 7f5cedec1640 1 Processor -- start 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.299+0000 7f5cedec1640 1 -- start start 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5cedec1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce81956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5cedec1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ce8102fb0 0x7f5ce8195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5cedec1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce8196200 con 0x7f5ce8100680 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5cedec1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce8196370 con 0x7f5ce8102fb0 2024-09-16T08:56:41.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5ce77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce81956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5ce77fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce81956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41518/0 (socket says 172.21.15.31:41518) 2024-09-16T08:56:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5ce77fe640 1 -- 172.21.15.31:0/1120852053 learned_addr learned my addr 172.21.15.31:0/1120852053 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5ce6ffd640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ce8102fb0 0x7f5ce8195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5ce77fe640 1 -- 172.21.15.31:0/1120852053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ce8102fb0 msgr2=0x7f5ce8195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5ce77fe640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ce8102fb0 0x7f5ce8195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.300+0000 7f5ce77fe640 1 -- 172.21.15.31:0/1120852053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd8009560 con 0x7f5ce8100680 2024-09-16T08:56:41.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.301+0000 7f5ce77fe640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce81956f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80054f0 tx=0x7f5cd8005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:41.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.301+0000 7f5ce4ff9640 1 -- 172.21.15.31:0/1120852053 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd8004aa0 con 0x7f5ce8100680 2024-09-16T08:56:41.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.301+0000 7f5ce4ff9640 1 -- 172.21.15.31:0/1120852053 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5cd8004c00 con 0x7f5ce8100680 2024-09-16T08:56:41.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.301+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce819adb0 con 0x7f5ce8100680 2024-09-16T08:56:41.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.301+0000 7f5ce4ff9640 1 -- 172.21.15.31:0/1120852053 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cd8003b10 con 0x7f5ce8100680 2024-09-16T08:56:41.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.301+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce819b240 con 0x7f5ce8100680 2024-09-16T08:56:41.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.302+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce8108810 con 0x7f5ce8100680 2024-09-16T08:56:41.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.303+0000 7f5ce4ff9640 1 -- 172.21.15.31:0/1120852053 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f5cd8003d80 con 0x7f5ce8100680 2024-09-16T08:56:41.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.303+0000 7f5ce4ff9640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5cd006cc50 0x7f5cd006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.303+0000 7f5ce4ff9640 1 -- 172.21.15.31:0/1120852053 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5cd80ad300 con 0x7f5ce8100680 2024-09-16T08:56:41.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.303+0000 7f5ce6ffd640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5cd006cc50 0x7f5cd006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.304+0000 7f5ce6ffd640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5cd006cc50 0x7f5cd006f110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8196c10 tx=0x7f5cdc00b390 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:41.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.308+0000 7f5ce4ff9640 1 -- 172.21.15.31:0/1120852053 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f5cd80777a0 con 0x7f5ce8100680 2024-09-16T08:56:41.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.523+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ce819b420 con 0x7f5cd006cc50 2024-09-16T08:56:41.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.524+0000 7f5ce4ff9640 1 -- 172.21.15.31:0/1120852053 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f5ce819b420 con 0x7f5cd006cc50 2024-09-16T08:56:41.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5cd006cc50 msgr2=0x7f5cd006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5cd006cc50 0x7f5cd006f110 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8196c10 tx=0x7f5cdc00b390 comp rx=0 tx=0).stop 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 msgr2=0x7f5ce81956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce81956f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80054f0 tx=0x7f5cd8005520 comp rx=0 tx=0).stop 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 shutdown_connections 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f5cd006cc50 0x7f5cd006f110 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ce8102fb0 0x7f5ce8195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 --2- 172.21.15.31:0/1120852053 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5ce8100680 0x7f5ce81956f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.526+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 >> 172.21.15.31:0/1120852053 conn(0x7f5ce80fa280 msgr2=0x7f5ce80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.527+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 shutdown_connections 2024-09-16T08:56:41.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.527+0000 7f5cedec1640 1 -- 172.21.15.31:0/1120852053 wait complete. 2024-09-16T08:56:41.537 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.633+0000 7fb9571f4640 1 -- 172.21.15.31:0/2176568165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb950100d90 msgr2=0x7fb950101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.633+0000 7fb9571f4640 1 --2- 172.21.15.31:0/2176568165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb950100d90 0x7fb950101210 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009880 tx=0x7fb93c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 -- 172.21.15.31:0/2176568165 shutdown_connections 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 --2- 172.21.15.31:0/2176568165 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb950100d90 0x7fb950101210 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 --2- 172.21.15.31:0/2176568165 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9500ffb90 0x7fb9500fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 -- 172.21.15.31:0/2176568165 >> 172.21.15.31:0/2176568165 conn(0x7fb9500fb320 msgr2=0x7fb9500fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 -- 172.21.15.31:0/2176568165 shutdown_connections 2024-09-16T08:56:41.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 -- 172.21.15.31:0/2176568165 wait complete. 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 Processor -- start 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.634+0000 7fb9571f4640 1 -- start start 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb9571f4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9500ffb90 0x7fb950195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb9571f4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb950100d90 0x7fb950195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb9571f4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950196170 con 0x7fb9500ffb90 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb9571f4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9501962e0 con 0x7fb950100d90 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb954f90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9500ffb90 0x7fb950195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb954f90640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9500ffb90 0x7fb950195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35746/0 (socket says 172.21.15.31:35746) 2024-09-16T08:56:41.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb954f90640 1 -- 172.21.15.31:0/178384168 learned_addr learned my addr 172.21.15.31:0/178384168 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb94ffff640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb950100d90 0x7fb950195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb954f90640 1 -- 172.21.15.31:0/178384168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb950100d90 msgr2=0x7fb950195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb954f90640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb950100d90 0x7fb950195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.635+0000 7fb954f90640 1 -- 172.21.15.31:0/178384168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c009560 con 0x7fb9500ffb90 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.636+0000 7fb954f90640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9500ffb90 0x7fb950195660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb94400e970 tx=0x7fb94400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.636+0000 7fb94dffb640 1 -- 172.21.15.31:0/178384168 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9440096e0 con 0x7fb9500ffb90 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.636+0000 7fb94dffb640 1 -- 172.21.15.31:0/178384168 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9440044f0 con 0x7fb9500ffb90 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.636+0000 7fb94dffb640 1 -- 172.21.15.31:0/178384168 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb944010430 con 0x7fb9500ffb90 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.636+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95019ad80 con 0x7fb9500ffb90 2024-09-16T08:56:41.638 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.636+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95019b270 con 0x7fb9500ffb90 2024-09-16T08:56:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.637+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95018f730 con 0x7fb9500ffb90 2024-09-16T08:56:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.637+0000 7fb94dffb640 1 -- 172.21.15.31:0/178384168 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fb94400cc90 con 0x7fb9500ffb90 2024-09-16T08:56:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.638+0000 7fb94dffb640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb94006cc50 0x7fb94006f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.638+0000 7fb94dffb640 1 -- 172.21.15.31:0/178384168 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb944014070 con 0x7fb9500ffb90 2024-09-16T08:56:41.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.638+0000 7fb94ffff640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb94006cc50 0x7fb94006f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.639+0000 7fb94ffff640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb94006cc50 0x7fb94006f110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009230 tx=0x7fb93c0058d0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:41.643 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.643+0000 7fb94dffb640 1 -- 172.21.15.31:0/178384168 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fb944056ad0 con 0x7fb9500ffb90 2024-09-16T08:56:41.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.891+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb95002c590 con 0x7fb94006cc50 2024-09-16T08:56:41.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.892+0000 7fb94dffb640 1 -- 172.21.15.31:0/178384168 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fb95002c590 con 0x7fb94006cc50 2024-09-16T08:56:41.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb94006cc50 msgr2=0x7fb94006f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb94006cc50 0x7fb94006f110 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb93c009230 tx=0x7fb93c0058d0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9500ffb90 msgr2=0x7fb950195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9500ffb90 0x7fb950195660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb94400e970 tx=0x7fb94400ee30 comp rx=0 tx=0).stop 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 shutdown_connections 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fb94006cc50 0x7fb94006f110 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb950100d90 0x7fb950195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 --2- 172.21.15.31:0/178384168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9500ffb90 0x7fb950195660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 >> 172.21.15.31:0/178384168 conn(0x7fb9500fb320 msgr2=0x7fb950104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.894+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 shutdown_connections 2024-09-16T08:56:41.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.895+0000 7fb9571f4640 1 -- 172.21.15.31:0/178384168 wait complete. 2024-09-16T08:56:41.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.993+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1331443400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8102fb0 msgr2=0x7f41b81053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1331443400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8102fb0 0x7f41b81053a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f41a80098b0 tx=0x7f41a802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1331443400 shutdown_connections 2024-09-16T08:56:41.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1331443400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8102fb0 0x7f41b81053a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1331443400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b8100680 0x7f41b8102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1331443400 >> 172.21.15.31:0/1331443400 conn(0x7f41b80fa280 msgr2=0x7f41b80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:41.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1331443400 shutdown_connections 2024-09-16T08:56:41.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1331443400 wait complete. 2024-09-16T08:56:41.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 Processor -- start 2024-09-16T08:56:41.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.994+0000 7f41bf7e9640 1 -- start start 2024-09-16T08:56:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bf7e9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8100680 0x7f41b8195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bf7e9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b8102fb0 0x7f41b8195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bf7e9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b8196230 con 0x7f41b8100680 2024-09-16T08:56:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bf7e9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b81963a0 con 0x7f41b8102fb0 2024-09-16T08:56:41.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bd585640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8100680 0x7f41b8195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bd585640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8100680 0x7f41b8195720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35768/0 (socket says 172.21.15.31:35768) 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bd585640 1 -- 172.21.15.31:0/1520028814 learned_addr learned my addr 172.21.15.31:0/1520028814 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bcd84640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b8102fb0 0x7f41b8195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bd585640 1 -- 172.21.15.31:0/1520028814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b8102fb0 msgr2=0x7f41b8195c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bd585640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b8102fb0 0x7f41b8195c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bd585640 1 -- 172.21.15.31:0/1520028814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a8009560 con 0x7f41b8100680 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.995+0000 7f41bd585640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8100680 0x7f41b8195720 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f41b400e9a0 tx=0x7f41b400eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.996+0000 7f41ae7fc640 1 -- 172.21.15.31:0/1520028814 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b400cd50 con 0x7f41b8100680 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.996+0000 7f41ae7fc640 1 -- 172.21.15.31:0/1520028814 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41b400ceb0 con 0x7f41b8100680 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.996+0000 7f41ae7fc640 1 -- 172.21.15.31:0/1520028814 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b4010660 con 0x7f41b8100680 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.996+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b819ae40 con 0x7f41b8100680 2024-09-16T08:56:41.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.996+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b819b300 con 0x7f41b8100680 2024-09-16T08:56:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.997+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b8100a60 con 0x7f41b8100680 2024-09-16T08:56:41.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.998+0000 7f41ae7fc640 1 -- 172.21.15.31:0/1520028814 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f41b40048a0 con 0x7f41b8100680 2024-09-16T08:56:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.998+0000 7f41ae7fc640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41a406cc60 0x7f41a406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.998+0000 7f41ae7fc640 1 -- 172.21.15.31:0/1520028814 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f41b4014070 con 0x7f41b8100680 2024-09-16T08:56:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.998+0000 7f41bcd84640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41a406cc60 0x7f41a406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:42.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:41.999+0000 7f41bcd84640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41a406cc60 0x7f41a406f120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009880 tx=0x7f41a80314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:42.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.002+0000 7f41ae7fc640 1 -- 172.21.15.31:0/1520028814 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f41b4056b00 con 0x7f41b8100680 2024-09-16T08:56:42.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.216+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f41b8103a50 con 0x7f41a406cc60 2024-09-16T08:56:42.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.226+0000 7f41ae7fc640 1 -- 172.21.15.31:0/1520028814 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f41b8103a50 con 0x7f41a406cc60 2024-09-16T08:56:42.227 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:56:42.227 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 90s ago 8m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:56:42.227 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (8m) 90s ago 8m 7528k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 66d87845da99 2024-09-16T08:56:42.227 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (7m) 100s ago 7m 7440k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 edd14d48bff9 2024-09-16T08:56:42.227 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (8m) 90s ago 8m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:56:42.227 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (7m) 100s ago 7m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 90s ago 7m 83.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (2m) 90s ago 2m 11.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (114s) 90s ago 113s 18.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (117s) 100s ago 117s 18.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (111s) 100s ago 110s 13.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:9283,8765,8443 running (9m) 90s ago 9m 486M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 1c78052ec7c9 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (7m) 100s ago 7m 441M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 b82e4c86f40c 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (9m) 90s ago 9m 47.7M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (7m) 100s ago 7m 35.3M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (8m) 90s ago 8m 15.3M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (7m) 100s ago 7m 16.1M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 90s ago 5m 47.3M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:56:42.228 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 90s ago 5m 54.9M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:56:42.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 90s ago 4m 46.6M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:56:42.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (4m) 100s ago 4m 51.5M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:56:42.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (3m) 100s ago 3m 44.3M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:56:42.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (3m) 100s ago 3m 44.0M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:56:42.229 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 90s ago 7m 60.0M - 2.43.0 a07b618ecd1d b0b74488871e 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41a406cc60 msgr2=0x7f41a406f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41a406cc60 0x7f41a406f120 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f41a8009880 tx=0x7f41a80314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8100680 msgr2=0x7f41b8195720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8100680 0x7f41b8195720 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f41b400e9a0 tx=0x7f41b400eda0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 shutdown_connections 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f41a406cc60 0x7f41a406f120 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41b8102fb0 0x7f41b8195c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 --2- 172.21.15.31:0/1520028814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f41b8100680 0x7f41b8195720 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.228+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 >> 172.21.15.31:0/1520028814 conn(0x7f41b80fa280 msgr2=0x7f41b80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:42.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.229+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 shutdown_connections 2024-09-16T08:56:42.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.229+0000 7f41bf7e9640 1 -- 172.21.15.31:0/1520028814 wait complete. 2024-09-16T08:56:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 -- 172.21.15.31:0/2299717205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c103000 msgr2=0x7f223c1053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 --2- 172.21.15.31:0/2299717205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c103000 0x7f223c1053f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f222c009880 tx=0x7f222c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 -- 172.21.15.31:0/2299717205 shutdown_connections 2024-09-16T08:56:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 --2- 172.21.15.31:0/2299717205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c103000 0x7f223c1053f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 --2- 172.21.15.31:0/2299717205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c1006d0 0x7f223c102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 -- 172.21.15.31:0/2299717205 >> 172.21.15.31:0/2299717205 conn(0x7f223c0fa230 msgr2=0x7f223c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 -- 172.21.15.31:0/2299717205 shutdown_connections 2024-09-16T08:56:42.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.318+0000 7f22446e0640 1 -- 172.21.15.31:0/2299717205 wait complete. 2024-09-16T08:56:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f22446e0640 1 Processor -- start 2024-09-16T08:56:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f22446e0640 1 -- start start 2024-09-16T08:56:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f22446e0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c1006d0 0x7f223c100030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f22446e0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c103000 0x7f223c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:42.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f22446e0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0febc0 con 0x7f223c1006d0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f22446e0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c0fed30 con 0x7f223c103000 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f224247c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c1006d0 0x7f223c100030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f224247c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c1006d0 0x7f223c100030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35790/0 (socket says 172.21.15.31:35790) 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f224247c640 1 -- 172.21.15.31:0/4167152954 learned_addr learned my addr 172.21.15.31:0/4167152954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.319+0000 7f2241c7b640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c103000 0x7f223c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f224247c640 1 -- 172.21.15.31:0/4167152954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c103000 msgr2=0x7f223c0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f224247c640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c103000 0x7f223c0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f224247c640 1 -- 172.21.15.31:0/4167152954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c009560 con 0x7f223c1006d0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f224247c640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c1006d0 0x7f223c100030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f223800e970 tx=0x7f223800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f22337fe640 1 -- 172.21.15.31:0/4167152954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223800cc90 con 0x7f223c1006d0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f22337fe640 1 -- 172.21.15.31:0/4167152954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f223800cdf0 con 0x7f223c1006d0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c0ff010 con 0x7f223c1006d0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f22337fe640 1 -- 172.21.15.31:0/4167152954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2238010430 con 0x7f223c1006d0 2024-09-16T08:56:42.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.320+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c19d650 con 0x7f223c1006d0 2024-09-16T08:56:42.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.321+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c1087a0 con 0x7f223c1006d0 2024-09-16T08:56:42.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.323+0000 7f22337fe640 1 -- 172.21.15.31:0/4167152954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f22380046c0 con 0x7f223c1006d0 2024-09-16T08:56:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.323+0000 7f22337fe640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f222806cc50 0x7f222806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.323+0000 7f22337fe640 1 -- 172.21.15.31:0/4167152954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2238014070 con 0x7f223c1006d0 2024-09-16T08:56:42.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.323+0000 7f2241c7b640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f222806cc50 0x7f222806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:42.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.324+0000 7f2241c7b640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f222806cc50 0x7f222806f110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f222c009230 tx=0x7f222c0314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:42.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.330+0000 7f22337fe640 1 -- 172.21.15.31:0/4167152954 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f2238056b60 con 0x7f223c1006d0 2024-09-16T08:56:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.692+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f223c19da40 con 0x7f223c1006d0 2024-09-16T08:56:42.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.692+0000 7f22337fe640 1 -- 172.21.15.31:0/4167152954 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+628 (secure 0 0 0) 0x7f223805a180 con 0x7f223c1006d0 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:56:42.694 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:56:42.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:56:42.695 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:56:42.695 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 14 2024-09-16T08:56:42.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:56:42.695 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:56:42.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f222806cc50 msgr2=0x7f222806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f222806cc50 0x7f222806f110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f222c009230 tx=0x7f222c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c1006d0 msgr2=0x7f223c100030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c1006d0 0x7f223c100030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f223800e970 tx=0x7f223800ed70 comp rx=0 tx=0).stop 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 shutdown_connections 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f222806cc50 0x7f222806f110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f223c103000 0x7f223c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 --2- 172.21.15.31:0/4167152954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223c1006d0 0x7f223c100030 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 >> 172.21.15.31:0/4167152954 conn(0x7f223c0fa230 msgr2=0x7f223c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 shutdown_connections 2024-09-16T08:56:42.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.696+0000 7f22446e0640 1 -- 172.21.15.31:0/4167152954 wait complete. 2024-09-16T08:56:42.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 -- 172.21.15.31:0/4234869992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102fb0 msgr2=0x7f805c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 --2- 172.21.15.31:0/4234869992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102fb0 0x7f805c1053a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f80580098b0 tx=0x7f805802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 -- 172.21.15.31:0/4234869992 shutdown_connections 2024-09-16T08:56:42.789 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 --2- 172.21.15.31:0/4234869992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c102fb0 0x7f805c1053a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 --2- 172.21.15.31:0/4234869992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f805c100680 0x7f805c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 -- 172.21.15.31:0/4234869992 >> 172.21.15.31:0/4234869992 conn(0x7f805c0fa280 msgr2=0x7f805c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 -- 172.21.15.31:0/4234869992 shutdown_connections 2024-09-16T08:56:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.788+0000 7f80645cc640 1 -- 172.21.15.31:0/4234869992 wait complete. 2024-09-16T08:56:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f80645cc640 1 Processor -- start 2024-09-16T08:56:42.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f80645cc640 1 -- start start 2024-09-16T08:56:42.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f80645cc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c100680 0x7f805c1956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:42.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f80645cc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f805c102fb0 0x7f805c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:42.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f80645cc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c196200 con 0x7f805c100680 2024-09-16T08:56:42.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f80645cc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c196370 con 0x7f805c102fb0 2024-09-16T08:56:42.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f8062368640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c100680 0x7f805c1956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:42.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f8062368640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c100680 0x7f805c1956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35794/0 (socket says 172.21.15.31:35794) 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f8062368640 1 -- 172.21.15.31:0/3681635680 learned_addr learned my addr 172.21.15.31:0/3681635680 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.789+0000 7f8061b67640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f805c102fb0 0x7f805c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f8062368640 1 -- 172.21.15.31:0/3681635680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f805c102fb0 msgr2=0x7f805c195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f8062368640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f805c102fb0 0x7f805c195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f8062368640 1 -- 172.21.15.31:0/3681635680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8058009560 con 0x7f805c100680 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f8062368640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c100680 0x7f805c1956f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f805400dbb0 tx=0x7f805400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f80537fe640 1 -- 172.21.15.31:0/3681635680 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f805400de70 con 0x7f805c100680 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f80537fe640 1 -- 172.21.15.31:0/3681635680 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8054003e30 con 0x7f805c100680 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f80537fe640 1 -- 172.21.15.31:0/3681635680 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8054010760 con 0x7f805c100680 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c19ae10 con 0x7f805c100680 2024-09-16T08:56:42.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.790+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c19b2d0 con 0x7f805c100680 2024-09-16T08:56:42.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.791+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c1087e0 con 0x7f805c100680 2024-09-16T08:56:42.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.792+0000 7f80537fe640 1 -- 172.21.15.31:0/3681635680 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f8054004020 con 0x7f805c100680 2024-09-16T08:56:42.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.793+0000 7f80537fe640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f804806cc50 0x7f804806f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:42.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.793+0000 7f80537fe640 1 -- 172.21.15.31:0/3681635680 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f805408c560 con 0x7f805c100680 2024-09-16T08:56:42.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.793+0000 7f8061b67640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f804806cc50 0x7f804806f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:42.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.793+0000 7f8061b67640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f804806cc50 0x7f804806f110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8058009880 tx=0x7f80580314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:42.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:42.799+0000 7f80537fe640 1 -- 172.21.15.31:0/3681635680 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f8054056950 con 0x7f805c100680 2024-09-16T08:56:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.038+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f805c19b890 con 0x7f805c100680 2024-09-16T08:56:43.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.038+0000 7f80537fe640 1 -- 172.21.15.31:0/3681635680 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f8054059f70 con 0x7f805c100680 2024-09-16T08:56:43.040 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:56:43.040 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:56:43.040 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:56:43.040 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:56:43.040 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:43.040 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:56:43.041 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:56:43.041 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:56:43.041 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:56:43.041 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:56:43.041 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:56:43.041 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:56:43.042 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:56:43.042 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:56:43.042 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:56:43.042 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:56:43.042 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:56:43.043 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:56:43.043 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:56:43.043 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:56:43.043 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:56:43.043 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:56:43.043 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:56:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:56:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:56:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:56:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:56:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:56:43.044 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:56:43.045 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:56:43.045 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:56:43.045 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:56:43.045 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:56:43.045 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:43.046 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:43.046 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:56:43.046 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:56:43.046 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:43.047 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:56:43.048 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:56:43.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f804806cc50 msgr2=0x7f804806f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f804806cc50 0x7f804806f110 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8058009880 tx=0x7f80580314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c100680 msgr2=0x7f805c1956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c100680 0x7f805c1956f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f805400dbb0 tx=0x7f805400b630 comp rx=0 tx=0).stop 2024-09-16T08:56:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 shutdown_connections 2024-09-16T08:56:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f804806cc50 0x7f804806f110 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f805c102fb0 0x7f805c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 --2- 172.21.15.31:0/3681635680 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f805c100680 0x7f805c1956f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.041+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 >> 172.21.15.31:0/3681635680 conn(0x7f805c0fa280 msgr2=0x7f805c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.042+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 shutdown_connections 2024-09-16T08:56:43.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.042+0000 7f80645cc640 1 -- 172.21.15.31:0/3681635680 wait complete. 2024-09-16T08:56:43.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.141+0000 7fd91aacd640 1 -- 172.21.15.31:0/635084274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 msgr2=0x7fd9140fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.141+0000 7fd91aacd640 1 --2- 172.21.15.31:0/635084274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 0x7fd9140fff90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd908009880 tx=0x7fd90802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.141+0000 7fd91aacd640 1 -- 172.21.15.31:0/635084274 shutdown_connections 2024-09-16T08:56:43.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.141+0000 7fd91aacd640 1 --2- 172.21.15.31:0/635084274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 0x7fd914101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.141+0000 7fd91aacd640 1 --2- 172.21.15.31:0/635084274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 0x7fd9140fff90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.141+0000 7fd91aacd640 1 -- 172.21.15.31:0/635084274 >> 172.21.15.31:0/635084274 conn(0x7fd9140fb360 msgr2=0x7fd9140fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:43.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.141+0000 7fd91aacd640 1 -- 172.21.15.31:0/635084274 shutdown_connections 2024-09-16T08:56:43.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.142+0000 7fd91aacd640 1 -- 172.21.15.31:0/635084274 wait complete. 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.142+0000 7fd91aacd640 1 Processor -- start 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.142+0000 7fd91aacd640 1 -- start start 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd91aacd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 0x7fd9141977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd91aacd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 0x7fd914197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd91aacd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914198300 con 0x7fd9140ffb90 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd91aacd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914198470 con 0x7fd914100d90 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd913fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 0x7fd914197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd913fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 0x7fd914197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51522/0 (socket says 172.21.15.31:51522) 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd913fff640 1 -- 172.21.15.31:0/3121415741 learned_addr learned my addr 172.21.15.31:0/3121415741 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:43.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd918869640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 0x7fd9141977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd913fff640 1 -- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 msgr2=0x7fd9141977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd913fff640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 0x7fd9141977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd913fff640 1 -- 172.21.15.31:0/3121415741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908009560 con 0x7fd914100d90 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.143+0000 7fd918869640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 0x7fd9141977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.144+0000 7fd913fff640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 0x7fd914197d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd90000d8d0 tx=0x7fd90000dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.144+0000 7fd911ffb640 1 -- 172.21.15.31:0/3121415741 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd900009c00 con 0x7fd914100d90 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.144+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91419cf10 con 0x7fd914100d90 2024-09-16T08:56:43.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.144+0000 7fd911ffb640 1 -- 172.21.15.31:0/3121415741 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd900009d60 con 0x7fd914100d90 2024-09-16T08:56:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.145+0000 7fd911ffb640 1 -- 172.21.15.31:0/3121415741 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9000049d0 con 0x7fd914100d90 2024-09-16T08:56:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.145+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91419d430 con 0x7fd914100d90 2024-09-16T08:56:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.146+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9140fff90 con 0x7fd914100d90 2024-09-16T08:56:43.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.147+0000 7fd911ffb640 1 -- 172.21.15.31:0/3121415741 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7fd900004c40 con 0x7fd914100d90 2024-09-16T08:56:43.148 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.147+0000 7fd911ffb640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd90406cc60 0x7fd90406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:43.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.147+0000 7fd911ffb640 1 -- 172.21.15.31:0/3121415741 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd90008bf60 con 0x7fd914100d90 2024-09-16T08:56:43.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.147+0000 7fd918869640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd90406cc60 0x7fd90406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:43.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.148+0000 7fd918869640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd90406cc60 0x7fd90406f120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd908006010 tx=0x7fd9080314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:43.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.152+0000 7fd911ffb640 1 -- 172.21.15.31:0/3121415741 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7fd9000bfe20 con 0x7fd914100d90 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: pgmap v262: 65 pgs: 65 active+clean; 82 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 6.6 MiB/s wr, 415 op/s 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/4167152954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3681635680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:43 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:56:43.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.407+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 --> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd91402c4f0 con 0x7fd90406cc60 2024-09-16T08:56:43.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.408+0000 7fd911ffb640 1 -- 172.21.15.31:0/3121415741 <== mgr.14211 v2:172.21.15.31:6800/2575391456 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7fd91402c4f0 con 0x7fd90406cc60 2024-09-16T08:56:43.409 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:56:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:56:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:56:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:56:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-09-16T08:56:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "", 2024-09-16T08:56:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T08:56:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:56:43.411 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:56:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.410+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd90406cc60 msgr2=0x7fd90406f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.410+0000 7fd91aacd640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd90406cc60 0x7fd90406f120 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd908006010 tx=0x7fd9080314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.412 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.410+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 msgr2=0x7fd914197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.410+0000 7fd91aacd640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 0x7fd914197d30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd90000d8d0 tx=0x7fd90000dd90 comp rx=0 tx=0).stop 2024-09-16T08:56:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.411+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 shutdown_connections 2024-09-16T08:56:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.411+0000 7fd91aacd640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7fd90406cc60 0x7fd90406f120 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.411+0000 7fd91aacd640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd914100d90 0x7fd914197d30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.411+0000 7fd91aacd640 1 --2- 172.21.15.31:0/3121415741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9140ffb90 0x7fd9141977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.411+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 >> 172.21.15.31:0/3121415741 conn(0x7fd9140fb360 msgr2=0x7fd914072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:43.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.411+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 shutdown_connections 2024-09-16T08:56:43.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.411+0000 7fd91aacd640 1 -- 172.21.15.31:0/3121415741 wait complete. 2024-09-16T08:56:43.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: pgmap v262: 65 pgs: 65 active+clean; 82 MiB data, 450 MiB used, 536 GiB / 536 GiB avail; 6.6 MiB/s wr, 415 op/s 2024-09-16T08:56:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/4167152954' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:56:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3681635680' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:56:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:43.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:43 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:56:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.511+0000 7f998241d640 1 -- 172.21.15.31:0/1213312942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c102fb0 msgr2=0x7f997c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.511+0000 7f998241d640 1 --2- 172.21.15.31:0/1213312942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c102fb0 0x7f997c1053a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f99700098b0 tx=0x7f997002a4d0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 -- 172.21.15.31:0/1213312942 shutdown_connections 2024-09-16T08:56:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 --2- 172.21.15.31:0/1213312942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c102fb0 0x7f997c1053a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 --2- 172.21.15.31:0/1213312942 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c100680 0x7f997c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.513 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 -- 172.21.15.31:0/1213312942 >> 172.21.15.31:0/1213312942 conn(0x7f997c0fa280 msgr2=0x7f997c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:43.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 -- 172.21.15.31:0/1213312942 shutdown_connections 2024-09-16T08:56:43.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 -- 172.21.15.31:0/1213312942 wait complete. 2024-09-16T08:56:43.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 Processor -- start 2024-09-16T08:56:43.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.512+0000 7f998241d640 1 -- start start 2024-09-16T08:56:43.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f998241d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c100680 0x7f997c1956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:43.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f998241d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102fb0 0x7f997c195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:43.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f998241d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c196200 con 0x7f997c100680 2024-09-16T08:56:43.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f998241d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c196370 con 0x7f997c102fb0 2024-09-16T08:56:43.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f997bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c100680 0x7f997c1956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f997b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102fb0 0x7f997c195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f997b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102fb0 0x7f997c195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:51540/0 (socket says 172.21.15.31:51540) 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f997b7fe640 1 -- 172.21.15.31:0/373826243 learned_addr learned my addr 172.21.15.31:0/373826243 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.513+0000 7f997bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c100680 0x7f997c1956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35826/0 (socket says 172.21.15.31:35826) 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f997bfff640 1 -- 172.21.15.31:0/373826243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102fb0 msgr2=0x7f997c195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f997bfff640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102fb0 0x7f997c195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f997bfff640 1 -- 172.21.15.31:0/373826243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970009560 con 0x7f997c100680 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f997b7fe640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102fb0 0x7f997c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f997bfff640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c100680 0x7f997c1956f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e9a0 tx=0x7f996c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f99797fa640 1 -- 172.21.15.31:0/373826243 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f996c009640 con 0x7f997c100680 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c19ada0 con 0x7f997c100680 2024-09-16T08:56:43.516 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f99797fa640 1 -- 172.21.15.31:0/373826243 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f996c00cc90 con 0x7f997c100680 2024-09-16T08:56:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f99797fa640 1 -- 172.21.15.31:0/373826243 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f996c018810 con 0x7f997c100680 2024-09-16T08:56:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.514+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c19b270 con 0x7f997c100680 2024-09-16T08:56:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.515+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c1087e0 con 0x7f997c100680 2024-09-16T08:56:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.516+0000 7f99797fa640 1 -- 172.21.15.31:0/373826243 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 19) v1 ==== 90850+0+0 (secure 0 0 0) 0x7f996c018970 con 0x7f997c100680 2024-09-16T08:56:43.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.517+0000 7f99797fa640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f996406cc60 0x7f996406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:56:43.518 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.517+0000 7f99797fa640 1 -- 172.21.15.31:0/373826243 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f996c014070 con 0x7f997c100680 2024-09-16T08:56:43.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.522+0000 7f997b7fe640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f996406cc60 0x7f996406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:56:43.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.523+0000 7f997b7fe640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f996406cc60 0x7f996406f120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f997c196ba0 tx=0x7f99700314e0 comp rx=0 tx=0).ready entity=mgr.14211 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:56:43.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.525+0000 7f99797fa640 1 -- 172.21.15.31:0/373826243 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183819 (secure 0 0 0) 0x7f996c056810 con 0x7f997c100680 2024-09-16T08:56:43.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.844+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f997c108a40 con 0x7f997c100680 2024-09-16T08:56:43.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.844+0000 7f99797fa640 1 -- 172.21.15.31:0/373826243 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f996c059e30 con 0x7f997c100680 2024-09-16T08:56:43.846 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:56:43.848 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f996406cc60 msgr2=0x7f996406f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f996406cc60 0x7f996406f120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f997c196ba0 tx=0x7f99700314e0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c100680 msgr2=0x7f997c1956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:56:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c100680 0x7f997c1956f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e9a0 tx=0x7f996c00ee60 comp rx=0 tx=0).stop 2024-09-16T08:56:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 shutdown_connections 2024-09-16T08:56:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:6800/2575391456,v1:172.21.15.31:6801/2575391456] conn(0x7f996406cc60 0x7f996406f120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.849 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f997c102fb0 0x7f997c195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 --2- 172.21.15.31:0/373826243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f997c100680 0x7f997c1956f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:56:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 >> 172.21.15.31:0/373826243 conn(0x7f997c0fa280 msgr2=0x7f997c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:56:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 shutdown_connections 2024-09-16T08:56:43.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:56:43.847+0000 7f998241d640 1 -- 172.21.15.31:0/373826243 wait complete. 2024-09-16T08:56:44.164 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:44 smithi097 ceph-mon[35478]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:44.164 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:44 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/373826243' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:56:44.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:44 smithi031 ceph-mon[27844]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:44.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:44 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/373826243' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:56:45.424 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:45 smithi097 ceph-mon[35478]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:45.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:45 smithi097 ceph-mon[35478]: pgmap v263: 65 pgs: 65 active+clean; 94 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 7.7 MiB/s wr, 423 op/s 2024-09-16T08:56:45.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:45 smithi031 ceph-mon[27844]: from='client.24457 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:56:45.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:45 smithi031 ceph-mon[27844]: pgmap v263: 65 pgs: 65 active+clean; 94 MiB data, 500 MiB used, 536 GiB / 536 GiB avail; 7.7 MiB/s wr, 423 op/s 2024-09-16T08:56:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:46 smithi031 ceph-mon[27844]: pgmap v264: 65 pgs: 65 active+clean; 96 MiB data, 508 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s wr, 195 op/s 2024-09-16T08:56:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:46 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:47.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:46 smithi097 ceph-mon[35478]: pgmap v264: 65 pgs: 65 active+clean; 96 MiB data, 508 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s wr, 195 op/s 2024-09-16T08:56:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:46 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:56:49.501 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:49 smithi097 ceph-mon[35478]: pgmap v265: 65 pgs: 65 active+clean; 121 MiB data, 580 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s wr, 371 op/s 2024-09-16T08:56:49.501 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:49 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf restarted 2024-09-16T08:56:49.501 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:49 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:56:49.501 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:49 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:56:49.501 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:49 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:56:49.501 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:49 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:56:49.501 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:49 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:56:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:49 smithi031 ceph-mon[27844]: pgmap v265: 65 pgs: 65 active+clean; 121 MiB data, 580 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s wr, 371 op/s 2024-09-16T08:56:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:49 smithi031 ceph-mon[27844]: Standby manager daemon smithi097.mpashf restarted 2024-09-16T08:56:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:49 smithi031 ceph-mon[27844]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:56:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:49 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:56:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:49 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:56:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:49 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:56:49.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:49 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:56:50.451 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:50 smithi097 ceph-mon[35478]: mgrmap e20: smithi031.dcmqpv(active, since 7m), standbys: smithi097.mpashf 2024-09-16T08:56:50.451 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:50 smithi097 ceph-mon[35478]: pgmap v266: 65 pgs: 65 active+clean; 127 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 393 op/s 2024-09-16T08:56:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:50 smithi031 ceph-mon[27844]: mgrmap e20: smithi031.dcmqpv(active, since 7m), standbys: smithi097.mpashf 2024-09-16T08:56:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:50 smithi031 ceph-mon[27844]: pgmap v266: 65 pgs: 65 active+clean; 127 MiB data, 626 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 393 op/s 2024-09-16T08:56:53.450 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:53 smithi097 ceph-mon[35478]: pgmap v267: 65 pgs: 65 active+clean; 128 MiB data, 639 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s wr, 383 op/s 2024-09-16T08:56:53.450 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:53 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:53.590 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:53 smithi031 ceph-mon[27844]: pgmap v267: 65 pgs: 65 active+clean; 128 MiB data, 639 MiB used, 536 GiB / 536 GiB avail; 4.0 MiB/s wr, 383 op/s 2024-09-16T08:56:53.590 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:53 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:56:55.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:55 smithi097 ceph-mon[35478]: pgmap v268: 65 pgs: 65 active+clean; 164 MiB data, 766 MiB used, 536 GiB / 536 GiB avail; 7.3 MiB/s wr, 790 op/s 2024-09-16T08:56:55.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:55 smithi031 ceph-mon[27844]: pgmap v268: 65 pgs: 65 active+clean; 164 MiB data, 766 MiB used, 536 GiB / 536 GiB avail; 7.3 MiB/s wr, 790 op/s 2024-09-16T08:56:57.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:57 smithi097 ceph-mon[35478]: pgmap v269: 65 pgs: 65 active+clean; 173 MiB data, 805 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s wr, 858 op/s 2024-09-16T08:56:57.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:57 smithi031 ceph-mon[27844]: pgmap v269: 65 pgs: 65 active+clean; 173 MiB data, 805 MiB used, 536 GiB / 536 GiB avail; 7.0 MiB/s wr, 858 op/s 2024-09-16T08:56:58.729 INFO:tasks.workunit.client.1.smithi097.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-16T08:56:58.746 INFO:tasks.workunit.client.1.smithi097.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T08:56:58.746 INFO:tasks.workunit.client.1.smithi097.stderr:+ make 2024-09-16T08:56:59.302 INFO:tasks.workunit.client.1.smithi097.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-16T08:56:59.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:56:59 smithi097 ceph-mon[35478]: pgmap v270: 65 pgs: 65 active+clean; 191 MiB data, 890 MiB used, 536 GiB / 536 GiB avail; 8.4 MiB/s wr, 921 op/s 2024-09-16T08:56:59.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:56:59 smithi031 ceph-mon[27844]: pgmap v270: 65 pgs: 65 active+clean; 191 MiB data, 890 MiB used, 536 GiB / 536 GiB avail; 8.4 MiB/s wr, 921 op/s 2024-09-16T08:57:00.656 INFO:tasks.workunit.client.1.smithi097.stderr:++ readlink -f fsstress 2024-09-16T08:57:00.656 INFO:tasks.workunit.client.1.smithi097.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-16T08:57:00.657 INFO:tasks.workunit.client.1.smithi097.stderr:+ popd 2024-09-16T08:57:00.657 INFO:tasks.workunit.client.1.smithi097.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-16T08:57:00.657 INFO:tasks.workunit.client.1.smithi097.stderr:+ popd 2024-09-16T08:57:00.657 INFO:tasks.workunit.client.1.smithi097.stdout:~/cephtest/mnt.1/client.1/tmp 2024-09-16T08:57:00.658 INFO:tasks.workunit.client.1.smithi097.stderr:++ mktemp -d -p . 2024-09-16T08:57:00.659 INFO:tasks.workunit.client.1.smithi097.stderr:+ T=./tmp.wACgg2kGuI 2024-09-16T08:57:00.659 INFO:tasks.workunit.client.1.smithi097.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.wACgg2kGuI -l 1 -n 1000 -p 10 -v 2024-09-16T08:57:00.662 INFO:tasks.workunit.client.1.smithi097.stdout:seed = 1726869358 2024-09-16T08:57:00.726 INFO:tasks.workunit.client.1.smithi097.stdout:1/0: dwrite - no filename 2024-09-16T08:57:00.726 INFO:tasks.workunit.client.1.smithi097.stdout:3/0: dwrite - no filename 2024-09-16T08:57:00.726 INFO:tasks.workunit.client.1.smithi097.stdout:3/1: truncate - no filename 2024-09-16T08:57:00.726 INFO:tasks.workunit.client.1.smithi097.stdout:3/2: dwrite - no filename 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:2/0: rmdir - no directory 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:2/1: dwrite - no filename 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:2/2: dread - no filename 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:2/3: fdatasync - no filename 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:5/0: unlink - no file 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/0: link - no file 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/1: dwrite - no filename 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/2: getdents . 0 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/3: unlink - no file 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/4: readlink - no filename 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/5: dread - no filename 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/6: rmdir - no directory 2024-09-16T08:57:00.727 INFO:tasks.workunit.client.1.smithi097.stdout:4/7: dwrite - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/0: dread - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/1: fsync - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/2: dread - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/3: truncate - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/4: truncate - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/5: rename - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/6: unlink - no file 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/7: write - no filename 2024-09-16T08:57:00.728 INFO:tasks.workunit.client.1.smithi097.stdout:6/8: chown . 4441337 1 2024-09-16T08:57:00.743 INFO:tasks.workunit.client.1.smithi097.stdout:1/1: creat f0 x:0 0 0 2024-09-16T08:57:00.745 INFO:tasks.workunit.client.1.smithi097.stdout:3/3: mkdir d0 0 2024-09-16T08:57:00.747 INFO:tasks.workunit.client.1.smithi097.stdout:2/4: creat f0 x:0 0 0 2024-09-16T08:57:00.749 INFO:tasks.workunit.client.1.smithi097.stdout:4/8: mkdir d0 0 2024-09-16T08:57:00.749 INFO:tasks.workunit.client.1.smithi097.stdout:4/9: stat d0 0 2024-09-16T08:57:00.749 INFO:tasks.workunit.client.1.smithi097.stdout:4/10: dwrite - no filename 2024-09-16T08:57:00.749 INFO:tasks.workunit.client.1.smithi097.stdout:4/11: stat d0 0 2024-09-16T08:57:00.749 INFO:tasks.workunit.client.1.smithi097.stdout:4/12: fsync - no filename 2024-09-16T08:57:00.749 INFO:tasks.workunit.client.1.smithi097.stdout:4/13: stat d0 0 2024-09-16T08:57:00.754 INFO:tasks.workunit.client.1.smithi097.stdout:6/9: symlink l0 0 2024-09-16T08:57:00.754 INFO:tasks.workunit.client.1.smithi097.stdout:6/10: dwrite - no filename 2024-09-16T08:57:00.755 INFO:tasks.workunit.client.1.smithi097.stdout:6/11: fdatasync - no filename 2024-09-16T08:57:00.755 INFO:tasks.workunit.client.1.smithi097.stdout:6/12: write - no filename 2024-09-16T08:57:00.755 INFO:tasks.workunit.client.1.smithi097.stdout:6/13: fsync - no filename 2024-09-16T08:57:00.755 INFO:tasks.workunit.client.1.smithi097.stdout:6/14: chown l0 2 1 2024-09-16T08:57:00.777 INFO:tasks.workunit.client.1.smithi097.stdout:3/4: rmdir d0 0 2024-09-16T08:57:00.777 INFO:tasks.workunit.client.1.smithi097.stdout:3/5: truncate - no filename 2024-09-16T08:57:00.777 INFO:tasks.workunit.client.1.smithi097.stdout:3/6: rename - no filename 2024-09-16T08:57:00.777 INFO:tasks.workunit.client.1.smithi097.stdout:3/7: dread - no filename 2024-09-16T08:57:00.777 INFO:tasks.workunit.client.1.smithi097.stdout:3/8: chown . 42257 1 2024-09-16T08:57:00.778 INFO:tasks.workunit.client.1.smithi097.stdout:3/9: dwrite - no filename 2024-09-16T08:57:00.780 INFO:tasks.workunit.client.1.smithi097.stdout:2/5: rename f0 to f1 0 2024-09-16T08:57:00.781 INFO:tasks.workunit.client.1.smithi097.stdout:2/6: creat f2 x:0 0 0 2024-09-16T08:57:00.782 INFO:tasks.workunit.client.1.smithi097.stdout:4/14: symlink d0/l1 0 2024-09-16T08:57:00.782 INFO:tasks.workunit.client.1.smithi097.stdout:4/15: dread - no filename 2024-09-16T08:57:00.783 INFO:tasks.workunit.client.1.smithi097.stdout:4/16: truncate - no filename 2024-09-16T08:57:00.783 INFO:tasks.workunit.client.1.smithi097.stdout:4/17: fdatasync - no filename 2024-09-16T08:57:00.783 INFO:tasks.workunit.client.1.smithi097.stdout:4/18: readlink d0/l1 0 2024-09-16T08:57:00.783 INFO:tasks.workunit.client.1.smithi097.stdout:4/19: rename d0 to d0/d2 22 2024-09-16T08:57:00.784 INFO:tasks.workunit.client.1.smithi097.stdout:8/0: write - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/1: dread - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/2: dread - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/3: dwrite - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/4: chown . 102213400 1 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/5: dwrite - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/6: chown . 37407811 1 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/7: fdatasync - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/8: fsync - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/9: link - no file 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/10: unlink - no file 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/11: dread - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/12: write - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/13: write - no filename 2024-09-16T08:57:00.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/14: truncate - no filename 2024-09-16T08:57:00.786 INFO:tasks.workunit.client.1.smithi097.stdout:8/15: write - no filename 2024-09-16T08:57:00.786 INFO:tasks.workunit.client.1.smithi097.stdout:8/16: dread - no filename 2024-09-16T08:57:00.787 INFO:tasks.workunit.client.1.smithi097.stdout:7/0: symlink l0 0 2024-09-16T08:57:00.789 INFO:tasks.workunit.client.1.smithi097.stdout:6/15: mkdir d1 0 2024-09-16T08:57:00.789 INFO:tasks.workunit.client.1.smithi097.stdout:6/16: stat l0 0 2024-09-16T08:57:00.789 INFO:tasks.workunit.client.1.smithi097.stdout:6/17: truncate - no filename 2024-09-16T08:57:00.792 INFO:tasks.workunit.client.1.smithi097.stdout:3/10: mknod c1 0 2024-09-16T08:57:00.792 INFO:tasks.workunit.client.1.smithi097.stdout:3/11: chown c1 54 1 2024-09-16T08:57:00.792 INFO:tasks.workunit.client.1.smithi097.stdout:3/12: chown c1 1525 1 2024-09-16T08:57:00.796 INFO:tasks.workunit.client.1.smithi097.stdout:2/7: dwrite f2 [0,4194304] 0 2024-09-16T08:57:00.796 INFO:tasks.workunit.client.1.smithi097.stdout:4/20: mkdir d0/d3 0 2024-09-16T08:57:00.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/0: truncate - no filename 2024-09-16T08:57:00.800 INFO:tasks.workunit.client.1.smithi097.stdout:8/17: mknod c0 0 2024-09-16T08:57:00.802 INFO:tasks.workunit.client.1.smithi097.stdout:7/1: creat f1 x:0 0 0 2024-09-16T08:57:00.802 INFO:tasks.workunit.client.1.smithi097.stdout:7/2: creat f2 x:0 0 0 2024-09-16T08:57:00.804 INFO:tasks.workunit.client.1.smithi097.stdout:6/18: unlink l0 0 2024-09-16T08:57:00.806 INFO:tasks.workunit.client.1.smithi097.stdout:3/13: rename c1 to c2 0 2024-09-16T08:57:00.811 INFO:tasks.workunit.client.1.smithi097.stdout:2/8: mkdir d3 0 2024-09-16T08:57:00.811 INFO:tasks.workunit.client.1.smithi097.stdout:2/9: write f1 [644819,84971] 0 2024-09-16T08:57:00.812 INFO:tasks.workunit.client.1.smithi097.stdout:4/21: creat d0/f4 x:0 0 0 2024-09-16T08:57:00.815 INFO:tasks.workunit.client.1.smithi097.stdout:9/1: creat f0 x:0 0 0 2024-09-16T08:57:00.818 INFO:tasks.workunit.client.1.smithi097.stdout:8/18: creat f1 x:0 0 0 2024-09-16T08:57:00.818 INFO:tasks.workunit.client.1.smithi097.stdout:8/19: write f1 [819369,113110] 0 2024-09-16T08:57:00.818 INFO:tasks.workunit.client.1.smithi097.stdout:8/20: chown c0 28092 1 2024-09-16T08:57:00.826 INFO:tasks.workunit.client.1.smithi097.stdout:7/3: link l0 l3 0 2024-09-16T08:57:00.826 INFO:tasks.workunit.client.1.smithi097.stdout:7/4: chown f2 3 1 2024-09-16T08:57:00.826 INFO:tasks.workunit.client.1.smithi097.stdout:3/14: symlink l3 0 2024-09-16T08:57:00.826 INFO:tasks.workunit.client.1.smithi097.stdout:3/15: dwrite - no filename 2024-09-16T08:57:00.826 INFO:tasks.workunit.client.1.smithi097.stdout:9/2: link f0 f1 0 2024-09-16T08:57:00.829 INFO:tasks.workunit.client.1.smithi097.stdout:7/5: mkdir d4 0 2024-09-16T08:57:00.831 INFO:tasks.workunit.client.1.smithi097.stdout:4/22: dwrite d0/f4 [0,4194304] 0 2024-09-16T08:57:00.832 INFO:tasks.workunit.client.1.smithi097.stdout:3/16: creat f4 x:0 0 0 2024-09-16T08:57:00.832 INFO:tasks.workunit.client.1.smithi097.stdout:3/17: write f4 [395135,95538] 0 2024-09-16T08:57:00.832 INFO:tasks.workunit.client.1.smithi097.stdout:3/18: chown l3 3 1 2024-09-16T08:57:00.835 INFO:tasks.workunit.client.1.smithi097.stdout:7/6: creat d4/f5 x:0 0 0 2024-09-16T08:57:00.835 INFO:tasks.workunit.client.1.smithi097.stdout:7/7: readlink l3 0 2024-09-16T08:57:00.835 INFO:tasks.workunit.client.1.smithi097.stdout:7/8: dread - d4/f5 zero size 2024-09-16T08:57:00.840 INFO:tasks.workunit.client.1.smithi097.stdout:3/19: rename l3 to l5 0 2024-09-16T08:57:00.840 INFO:tasks.workunit.client.1.smithi097.stdout:3/20: write f4 [1183021,96081] 0 2024-09-16T08:57:00.841 INFO:tasks.workunit.client.1.smithi097.stdout:7/9: mknod d4/c6 0 2024-09-16T08:57:00.842 INFO:tasks.workunit.client.1.smithi097.stdout:4/23: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:00.842 INFO:tasks.workunit.client.1.smithi097.stdout:4/24: chown d0 32263 1 2024-09-16T08:57:00.843 INFO:tasks.workunit.client.1.smithi097.stdout:9/3: dwrite f1 [0,4194304] 0 2024-09-16T08:57:00.844 INFO:tasks.workunit.client.1.smithi097.stdout:7/10: mkdir d4/d7 0 2024-09-16T08:57:00.846 INFO:tasks.workunit.client.1.smithi097.stdout:4/25: mknod d0/c5 0 2024-09-16T08:57:00.850 INFO:tasks.workunit.client.1.smithi097.stdout:4/26: symlink d0/l6 0 2024-09-16T08:57:00.853 INFO:tasks.workunit.client.1.smithi097.stdout:4/27: mkdir d0/d7 0 2024-09-16T08:57:00.866 INFO:tasks.workunit.client.1.smithi097.stdout:4/28: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:00.866 INFO:tasks.workunit.client.1.smithi097.stdout:4/29: chown d0/d7 3313 1 2024-09-16T08:57:00.866 INFO:tasks.workunit.client.1.smithi097.stdout:4/30: chown d0/d3 163 1 2024-09-16T08:57:00.871 INFO:tasks.workunit.client.1.smithi097.stdout:4/31: write d0/f4 [3157004,52407] 0 2024-09-16T08:57:00.877 INFO:tasks.workunit.client.1.smithi097.stdout:7/11: dwrite f1 [0,4194304] 0 2024-09-16T08:57:00.879 INFO:tasks.workunit.client.1.smithi097.stdout:3/21: dwrite f4 [0,4194304] 0 2024-09-16T08:57:00.879 INFO:tasks.workunit.client.1.smithi097.stdout:9/4: dwrite f0 [0,4194304] 0 2024-09-16T08:57:00.881 INFO:tasks.workunit.client.1.smithi097.stdout:7/12: rename l3 to d4/d7/l8 0 2024-09-16T08:57:00.885 INFO:tasks.workunit.client.1.smithi097.stdout:9/5: mkdir d2 0 2024-09-16T08:57:00.888 INFO:tasks.workunit.client.1.smithi097.stdout:7/13: mkdir d4/d9 0 2024-09-16T08:57:00.891 INFO:tasks.workunit.client.1.smithi097.stdout:7/14: link d4/c6 d4/d7/ca 0 2024-09-16T08:57:00.891 INFO:tasks.workunit.client.1.smithi097.stdout:7/15: readlink d4/d7/l8 0 2024-09-16T08:57:01.504 INFO:tasks.workunit.client.1.smithi097.stdout:1/2: getdents . 0 2024-09-16T08:57:01.504 INFO:tasks.workunit.client.1.smithi097.stdout:1/3: dread - f0 zero size 2024-09-16T08:57:01.505 INFO:tasks.workunit.client.1.smithi097.stdout:1/4: getdents . 0 2024-09-16T08:57:01.505 INFO:tasks.workunit.client.1.smithi097.stdout:1/5: write f0 [940057,33674] 0 2024-09-16T08:57:01.506 INFO:tasks.workunit.client.1.smithi097.stdout:1/6: fsync f0 0 2024-09-16T08:57:01.506 INFO:tasks.workunit.client.1.smithi097.stdout:1/7: stat f0 0 2024-09-16T08:57:01.506 INFO:tasks.workunit.client.1.smithi097.stdout:1/8: getdents . 0 2024-09-16T08:57:01.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:01 smithi097 ceph-mon[35478]: pgmap v271: 65 pgs: 65 active+clean; 195 MiB data, 912 MiB used, 536 GiB / 536 GiB avail; 6.6 MiB/s wr, 700 op/s 2024-09-16T08:57:01.523 INFO:tasks.workunit.client.1.smithi097.stdout:1/9: dwrite f0 [0,4194304] 0 2024-09-16T08:57:01.523 INFO:tasks.workunit.client.1.smithi097.stdout:1/10: chown f0 1644 1 2024-09-16T08:57:01.523 INFO:tasks.workunit.client.1.smithi097.stdout:1/11: chown f0 1918 1 2024-09-16T08:57:01.526 INFO:tasks.workunit.client.1.smithi097.stdout:1/12: mknod c1 0 2024-09-16T08:57:01.526 INFO:tasks.workunit.client.1.smithi097.stdout:1/13: fsync f0 0 2024-09-16T08:57:01.526 INFO:tasks.workunit.client.1.smithi097.stdout:1/14: rmdir - no directory 2024-09-16T08:57:01.532 INFO:tasks.workunit.client.1.smithi097.stdout:1/15: unlink f0 0 2024-09-16T08:57:01.532 INFO:tasks.workunit.client.1.smithi097.stdout:1/16: rmdir - no directory 2024-09-16T08:57:01.532 INFO:tasks.workunit.client.1.smithi097.stdout:1/17: dread - no filename 2024-09-16T08:57:01.532 INFO:tasks.workunit.client.1.smithi097.stdout:1/18: write - no filename 2024-09-16T08:57:01.536 INFO:tasks.workunit.client.1.smithi097.stdout:1/19: rename c1 to c2 0 2024-09-16T08:57:01.536 INFO:tasks.workunit.client.1.smithi097.stdout:1/20: write - no filename 2024-09-16T08:57:01.536 INFO:tasks.workunit.client.1.smithi097.stdout:1/21: dwrite - no filename 2024-09-16T08:57:01.539 INFO:tasks.workunit.client.1.smithi097.stdout:1/22: creat f3 x:0 0 0 2024-09-16T08:57:01.539 INFO:tasks.workunit.client.1.smithi097.stdout:1/23: truncate f3 190311 0 2024-09-16T08:57:01.543 INFO:tasks.workunit.client.1.smithi097.stdout:1/24: rename f3 to f4 0 2024-09-16T08:57:01.546 INFO:tasks.workunit.client.1.smithi097.stdout:1/25: mkdir d5 0 2024-09-16T08:57:01.546 INFO:tasks.workunit.client.1.smithi097.stdout:1/26: write f4 [566280,10464] 0 2024-09-16T08:57:01.562 INFO:tasks.workunit.client.1.smithi097.stdout:8/21: getdents . 0 2024-09-16T08:57:01.564 INFO:tasks.workunit.client.1.smithi097.stdout:8/22: mknod c2 0 2024-09-16T08:57:01.566 INFO:tasks.workunit.client.1.smithi097.stdout:8/23: creat f3 x:0 0 0 2024-09-16T08:57:01.567 INFO:tasks.workunit.client.1.smithi097.stdout:8/24: read f1 [235821,124921] 0 2024-09-16T08:57:01.570 INFO:tasks.workunit.client.1.smithi097.stdout:8/25: rename f1 to f4 0 2024-09-16T08:57:01.578 INFO:tasks.workunit.client.1.smithi097.stdout:3/22: getdents . 0 2024-09-16T08:57:01.586 INFO:tasks.workunit.client.1.smithi097.stdout:8/26: dwrite f4 [0,4194304] 0 2024-09-16T08:57:01.588 INFO:tasks.workunit.client.1.smithi097.stdout:3/23: dread f4 [0,4194304] 0 2024-09-16T08:57:01.591 INFO:tasks.workunit.client.1.smithi097.stdout:3/24: rename f4 to f6 0 2024-09-16T08:57:01.592 INFO:tasks.workunit.client.1.smithi097.stdout:8/27: dread f4 [0,4194304] 0 2024-09-16T08:57:01.594 INFO:tasks.workunit.client.1.smithi097.stdout:8/28: symlink l5 0 2024-09-16T08:57:01.596 INFO:tasks.workunit.client.1.smithi097.stdout:8/29: symlink l6 0 2024-09-16T08:57:01.596 INFO:tasks.workunit.client.1.smithi097.stdout:8/30: creat f7 x:0 0 0 2024-09-16T08:57:01.610 INFO:tasks.workunit.client.1.smithi097.stdout:4/32: truncate d0/f4 970534 0 2024-09-16T08:57:01.614 INFO:tasks.workunit.client.1.smithi097.stdout:8/31: dwrite f3 [0,4194304] 0 2024-09-16T08:57:01.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:01 smithi031 ceph-mon[27844]: pgmap v271: 65 pgs: 65 active+clean; 195 MiB data, 912 MiB used, 536 GiB / 536 GiB avail; 6.6 MiB/s wr, 700 op/s 2024-09-16T08:57:01.626 INFO:tasks.workunit.client.1.smithi097.stdout:9/6: truncate f1 2074108 0 2024-09-16T08:57:01.626 INFO:tasks.workunit.client.1.smithi097.stdout:9/7: chown f0 1606867758 1 2024-09-16T08:57:01.629 INFO:tasks.workunit.client.1.smithi097.stdout:8/32: dwrite f4 [0,4194304] 0 2024-09-16T08:57:01.629 INFO:tasks.workunit.client.1.smithi097.stdout:8/33: truncate f7 361714 0 2024-09-16T08:57:01.640 INFO:tasks.workunit.client.1.smithi097.stdout:8/34: read f3 [3919095,124244] 0 2024-09-16T08:57:01.641 INFO:tasks.workunit.client.1.smithi097.stdout:8/35: mkdir d8 0 2024-09-16T08:57:02.331 INFO:tasks.workunit.client.1.smithi097.stdout:1/27: rename c2 to d5/c6 0 2024-09-16T08:57:02.374 INFO:tasks.workunit.client.1.smithi097.stdout:3/25: fsync f6 0 2024-09-16T08:57:02.395 INFO:tasks.workunit.client.1.smithi097.stdout:4/33: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:02.395 INFO:tasks.workunit.client.1.smithi097.stdout:4/34: rename d0 to d0/d7/d8 22 2024-09-16T08:57:02.414 INFO:tasks.workunit.client.1.smithi097.stdout:4/35: dwrite d0/f4 [0,4194304] 0 2024-09-16T08:57:02.415 INFO:tasks.workunit.client.1.smithi097.stdout:8/36: truncate f4 824843 0 2024-09-16T08:57:02.415 INFO:tasks.workunit.client.1.smithi097.stdout:8/37: write f3 [2930394,81261] 0 2024-09-16T08:57:02.420 INFO:tasks.workunit.client.1.smithi097.stdout:9/8: dread f1 [0,4194304] 0 2024-09-16T08:57:02.426 INFO:tasks.workunit.client.1.smithi097.stdout:4/36: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:02.487 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:02 smithi097 ceph-mon[35478]: pgmap v272: 65 pgs: 65 active+clean; 198 MiB data, 923 MiB used, 536 GiB / 536 GiB avail; 6.3 MiB/s wr, 637 op/s 2024-09-16T08:57:02.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:02 smithi031 ceph-mon[27844]: pgmap v272: 65 pgs: 65 active+clean; 198 MiB data, 923 MiB used, 536 GiB / 536 GiB avail; 6.3 MiB/s wr, 637 op/s 2024-09-16T08:57:03.086 INFO:tasks.workunit.client.1.smithi097.stdout:3/26: mkdir d7 0 2024-09-16T08:57:03.086 INFO:tasks.workunit.client.1.smithi097.stdout:3/27: chown l5 50 1 2024-09-16T08:57:03.086 INFO:tasks.workunit.client.1.smithi097.stdout:3/28: stat f6 0 2024-09-16T08:57:03.092 INFO:tasks.workunit.client.1.smithi097.stdout:8/38: unlink f3 0 2024-09-16T08:57:03.092 INFO:tasks.workunit.client.1.smithi097.stdout:9/9: mknod d2/c3 0 2024-09-16T08:57:03.092 INFO:tasks.workunit.client.1.smithi097.stdout:9/10: truncate f1 2805877 0 2024-09-16T08:57:03.093 INFO:tasks.workunit.client.1.smithi097.stdout:4/37: mknod d0/d7/c9 0 2024-09-16T08:57:03.095 INFO:tasks.workunit.client.1.smithi097.stdout:3/29: symlink d7/l8 0 2024-09-16T08:57:03.097 INFO:tasks.workunit.client.1.smithi097.stdout:8/39: symlink d8/l9 0 2024-09-16T08:57:03.102 INFO:tasks.workunit.client.1.smithi097.stdout:1/28: link d5/c6 d5/c7 0 2024-09-16T08:57:03.102 INFO:tasks.workunit.client.1.smithi097.stdout:1/29: fdatasync f4 0 2024-09-16T08:57:03.103 INFO:tasks.workunit.client.1.smithi097.stdout:8/40: creat d8/fa x:0 0 0 2024-09-16T08:57:03.105 INFO:tasks.workunit.client.1.smithi097.stdout:1/30: dread f4 [0,4194304] 0 2024-09-16T08:57:03.106 INFO:tasks.workunit.client.0.smithi031.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-16T08:57:03.106 INFO:tasks.workunit.client.1.smithi097.stdout:4/38: getdents d0 0 2024-09-16T08:57:03.106 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T08:57:03.107 INFO:tasks.workunit.client.0.smithi031.stderr:+ make 2024-09-16T08:57:03.107 INFO:tasks.workunit.client.1.smithi097.stdout:1/31: dread f4 [0,4194304] 0 2024-09-16T08:57:03.110 INFO:tasks.workunit.client.1.smithi097.stdout:1/32: mknod d5/c8 0 2024-09-16T08:57:03.111 INFO:tasks.workunit.client.1.smithi097.stdout:4/39: write d0/f4 [3748838,23015] 0 2024-09-16T08:57:03.112 INFO:tasks.workunit.client.1.smithi097.stdout:1/33: creat d5/f9 x:0 0 0 2024-09-16T08:57:03.113 INFO:tasks.workunit.client.1.smithi097.stdout:9/11: dwrite f0 [0,4194304] 0 2024-09-16T08:57:03.115 INFO:tasks.workunit.client.1.smithi097.stdout:4/40: creat d0/fa x:0 0 0 2024-09-16T08:57:03.116 INFO:tasks.workunit.client.1.smithi097.stdout:1/34: symlink d5/la 0 2024-09-16T08:57:03.116 INFO:tasks.workunit.client.1.smithi097.stdout:1/35: chown d5/c6 299 1 2024-09-16T08:57:03.121 INFO:tasks.workunit.client.1.smithi097.stdout:4/41: link d0/f4 d0/d7/fb 0 2024-09-16T08:57:03.123 INFO:tasks.workunit.client.1.smithi097.stdout:1/36: mkdir d5/db 0 2024-09-16T08:57:03.125 INFO:tasks.workunit.client.1.smithi097.stdout:1/37: mkdir d5/db/dc 0 2024-09-16T08:57:03.125 INFO:tasks.workunit.client.1.smithi097.stdout:1/38: stat d5/db/dc 0 2024-09-16T08:57:03.125 INFO:tasks.workunit.client.1.smithi097.stdout:1/39: chown d5/c8 53 1 2024-09-16T08:57:03.125 INFO:tasks.workunit.client.1.smithi097.stdout:1/40: dread - d5/f9 zero size 2024-09-16T08:57:03.127 INFO:tasks.workunit.client.1.smithi097.stdout:1/41: dread f4 [0,4194304] 0 2024-09-16T08:57:03.129 INFO:tasks.workunit.client.1.smithi097.stdout:1/42: dread f4 [0,4194304] 0 2024-09-16T08:57:03.132 INFO:tasks.workunit.client.1.smithi097.stdout:1/43: mknod d5/cd 0 2024-09-16T08:57:03.134 INFO:tasks.workunit.client.1.smithi097.stdout:1/44: creat d5/db/dc/fe x:0 0 0 2024-09-16T08:57:03.136 INFO:tasks.workunit.client.1.smithi097.stdout:1/45: unlink d5/db/dc/fe 0 2024-09-16T08:57:03.138 INFO:tasks.workunit.client.1.smithi097.stdout:1/46: creat d5/db/ff x:0 0 0 2024-09-16T08:57:03.139 INFO:tasks.workunit.client.1.smithi097.stdout:1/47: stat d5/cd 0 2024-09-16T08:57:03.140 INFO:tasks.workunit.client.1.smithi097.stdout:1/48: dread f4 [0,4194304] 0 2024-09-16T08:57:03.142 INFO:tasks.workunit.client.1.smithi097.stdout:1/49: link f4 d5/db/dc/f10 0 2024-09-16T08:57:03.142 INFO:tasks.workunit.client.1.smithi097.stdout:4/42: dwrite d0/fa [0,4194304] 0 2024-09-16T08:57:03.143 INFO:tasks.workunit.client.1.smithi097.stdout:1/50: mkdir d5/db/d11 0 2024-09-16T08:57:03.143 INFO:tasks.workunit.client.1.smithi097.stdout:1/51: creat d5/f12 x:0 0 0 2024-09-16T08:57:03.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/43: link d0/l1 d0/d7/lc 0 2024-09-16T08:57:03.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/44: write d0/f4 [4074244,16606] 0 2024-09-16T08:57:03.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/45: getdents d0/d3 0 2024-09-16T08:57:03.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/46: creat d0/fd x:0 0 0 2024-09-16T08:57:03.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/47: creat d0/fe x:0 0 0 2024-09-16T08:57:03.154 INFO:tasks.workunit.client.1.smithi097.stdout:4/48: write d0/fa [1979792,13065] 0 2024-09-16T08:57:03.154 INFO:tasks.workunit.client.1.smithi097.stdout:4/49: read d0/f4 [2329541,44656] 0 2024-09-16T08:57:03.155 INFO:tasks.workunit.client.1.smithi097.stdout:4/50: mkdir d0/d7/df 0 2024-09-16T08:57:03.161 INFO:tasks.workunit.client.1.smithi097.stdout:4/51: dread d0/fa [0,4194304] 0 2024-09-16T08:57:03.162 INFO:tasks.workunit.client.1.smithi097.stdout:4/52: creat d0/d7/df/f10 x:0 0 0 2024-09-16T08:57:03.176 INFO:tasks.workunit.client.1.smithi097.stdout:4/53: dwrite d0/fa [0,4194304] 0 2024-09-16T08:57:03.177 INFO:tasks.workunit.client.1.smithi097.stdout:4/54: creat d0/d3/f11 x:0 0 0 2024-09-16T08:57:03.179 INFO:tasks.workunit.client.1.smithi097.stdout:4/55: fdatasync d0/f4 0 2024-09-16T08:57:03.180 INFO:tasks.workunit.client.1.smithi097.stdout:4/56: write d0/d3/f11 [968030,130739] 0 2024-09-16T08:57:03.180 INFO:tasks.workunit.client.1.smithi097.stdout:4/57: chown d0/d7/c9 0 1 2024-09-16T08:57:03.180 INFO:tasks.workunit.client.1.smithi097.stdout:4/58: chown d0/fa 500965 1 2024-09-16T08:57:03.180 INFO:tasks.workunit.client.1.smithi097.stdout:4/59: creat d0/d3/f12 x:0 0 0 2024-09-16T08:57:03.390 INFO:tasks.workunit.client.1.smithi097.stdout:9/12: getdents d2 0 2024-09-16T08:57:03.393 INFO:tasks.workunit.client.1.smithi097.stdout:8/41: rmdir d8 39 2024-09-16T08:57:03.396 INFO:tasks.workunit.client.1.smithi097.stdout:8/42: dread f4 [0,4194304] 0 2024-09-16T08:57:03.397 INFO:tasks.workunit.client.1.smithi097.stdout:8/43: mkdir d8/db 0 2024-09-16T08:57:03.399 INFO:tasks.workunit.client.1.smithi097.stdout:9/13: dread f0 [0,4194304] 0 2024-09-16T08:57:03.399 INFO:tasks.workunit.client.1.smithi097.stdout:7/16: sync 2024-09-16T08:57:03.399 INFO:tasks.workunit.client.1.smithi097.stdout:6/19: sync 2024-09-16T08:57:03.399 INFO:tasks.workunit.client.1.smithi097.stdout:2/10: sync 2024-09-16T08:57:03.399 INFO:tasks.workunit.client.1.smithi097.stdout:6/20: rename d1 to d1/d2 22 2024-09-16T08:57:03.400 INFO:tasks.workunit.client.1.smithi097.stdout:5/1: sync 2024-09-16T08:57:03.400 INFO:tasks.workunit.client.1.smithi097.stdout:0/0: sync 2024-09-16T08:57:03.400 INFO:tasks.workunit.client.1.smithi097.stdout:0/1: dread - no filename 2024-09-16T08:57:03.400 INFO:tasks.workunit.client.1.smithi097.stdout:0/2: rmdir - no directory 2024-09-16T08:57:03.400 INFO:tasks.workunit.client.1.smithi097.stdout:9/14: creat d2/f4 x:0 0 0 2024-09-16T08:57:03.402 INFO:tasks.workunit.client.1.smithi097.stdout:6/21: mkdir d1/d3 0 2024-09-16T08:57:03.403 INFO:tasks.workunit.client.1.smithi097.stdout:5/2: creat f0 x:0 0 0 2024-09-16T08:57:03.403 INFO:tasks.workunit.client.1.smithi097.stdout:5/3: write f0 [773912,62922] 0 2024-09-16T08:57:03.404 INFO:tasks.workunit.client.1.smithi097.stdout:5/4: stat f0 0 2024-09-16T08:57:03.404 INFO:tasks.workunit.client.1.smithi097.stdout:5/5: write f0 [1519631,47959] 0 2024-09-16T08:57:03.404 INFO:tasks.workunit.client.1.smithi097.stdout:0/3: creat f0 x:0 0 0 2024-09-16T08:57:03.404 INFO:tasks.workunit.client.1.smithi097.stdout:0/4: dread - f0 zero size 2024-09-16T08:57:03.404 INFO:tasks.workunit.client.1.smithi097.stdout:0/5: write f0 [754941,104949] 0 2024-09-16T08:57:03.405 INFO:tasks.workunit.client.1.smithi097.stdout:9/15: mkdir d2/d5 0 2024-09-16T08:57:03.406 INFO:tasks.workunit.client.1.smithi097.stdout:6/22: creat d1/d3/f4 x:0 0 0 2024-09-16T08:57:03.406 INFO:tasks.workunit.client.1.smithi097.stdout:6/23: creat d1/d3/f5 x:0 0 0 2024-09-16T08:57:03.407 INFO:tasks.workunit.client.1.smithi097.stdout:5/6: mkdir d1 0 2024-09-16T08:57:03.409 INFO:tasks.workunit.client.1.smithi097.stdout:7/17: read f1 [3881274,30554] 0 2024-09-16T08:57:03.409 INFO:tasks.workunit.client.1.smithi097.stdout:7/18: creat d4/fb x:0 0 0 2024-09-16T08:57:03.410 INFO:tasks.workunit.client.1.smithi097.stdout:9/16: rename f1 to d2/f6 0 2024-09-16T08:57:03.410 INFO:tasks.workunit.client.1.smithi097.stdout:9/17: fdatasync f0 0 2024-09-16T08:57:03.411 INFO:tasks.workunit.client.1.smithi097.stdout:6/24: mknod d1/d3/c6 0 2024-09-16T08:57:03.411 INFO:tasks.workunit.client.1.smithi097.stdout:6/25: write d1/d3/f5 [330858,34854] 0 2024-09-16T08:57:03.411 INFO:tasks.workunit.client.1.smithi097.stdout:6/26: chown d1 0 1 2024-09-16T08:57:03.412 INFO:tasks.workunit.client.1.smithi097.stdout:5/7: rmdir d1 0 2024-09-16T08:57:03.412 INFO:tasks.workunit.client.1.smithi097.stdout:5/8: chown f0 1329 1 2024-09-16T08:57:03.414 INFO:tasks.workunit.client.1.smithi097.stdout:8/44: dwrite f4 [0,4194304] 0 2024-09-16T08:57:03.417 INFO:tasks.workunit.client.1.smithi097.stdout:1/52: fsync d5/db/dc/f10 0 2024-09-16T08:57:03.418 INFO:tasks.workunit.client.1.smithi097.stdout:1/53: truncate d5/db/dc/f10 394263 0 2024-09-16T08:57:03.418 INFO:tasks.workunit.client.1.smithi097.stdout:1/54: stat d5/db/dc 0 2024-09-16T08:57:03.424 INFO:tasks.workunit.client.1.smithi097.stdout:4/60: getdents d0/d7 0 2024-09-16T08:57:03.431 INFO:tasks.workunit.client.1.smithi097.stdout:8/45: getdents d8 0 2024-09-16T08:57:03.432 INFO:tasks.workunit.client.1.smithi097.stdout:8/46: creat d8/fc x:0 0 0 2024-09-16T08:57:03.434 INFO:tasks.workunit.client.1.smithi097.stdout:3/30: sync 2024-09-16T08:57:03.438 INFO:tasks.workunit.client.1.smithi097.stdout:8/47: rename d8/db to d8/dd 0 2024-09-16T08:57:03.438 INFO:tasks.workunit.client.1.smithi097.stdout:8/48: write d8/fc [1023155,42901] 0 2024-09-16T08:57:03.440 INFO:tasks.workunit.client.1.smithi097.stdout:3/31: write f6 [885507,84806] 0 2024-09-16T08:57:03.440 INFO:tasks.workunit.client.1.smithi097.stdout:3/32: write f6 [3276898,108999] 0 2024-09-16T08:57:03.455 INFO:tasks.workunit.client.1.smithi097.stdout:4/61: dwrite d0/d7/df/f10 [0,4194304] 0 2024-09-16T08:57:03.455 INFO:tasks.workunit.client.1.smithi097.stdout:4/62: creat d0/d7/df/f13 x:0 0 0 2024-09-16T08:57:03.457 INFO:tasks.workunit.client.1.smithi097.stdout:4/63: symlink d0/l14 0 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:6/27: dwrite d1/d3/f4 [0,4194304] 0 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:6/28: creat d1/d3/f7 x:0 0 0 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:6/29: truncate d1/d3/f7 477320 0 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:7/19: dwrite d4/f5 [0,4194304] 0 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:7/20: truncate d4/fb 724634 0 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:7/21: dread - f2 zero size 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:9/18: dwrite d2/f6 [4194304,4194304] 0 2024-09-16T08:57:03.463 INFO:tasks.workunit.client.1.smithi097.stdout:0/6: dwrite f0 [0,4194304] 0 2024-09-16T08:57:03.465 INFO:tasks.workunit.client.1.smithi097.stdout:5/9: dread f0 [0,4194304] 0 2024-09-16T08:57:03.467 INFO:tasks.workunit.client.0.smithi031.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-16T08:57:03.467 INFO:tasks.workunit.client.1.smithi097.stdout:5/10: write f0 [482379,22649] 0 2024-09-16T08:57:03.467 INFO:tasks.workunit.client.1.smithi097.stdout:5/11: read f0 [458937,45289] 0 2024-09-16T08:57:03.467 INFO:tasks.workunit.client.1.smithi097.stdout:8/49: getdents d8 0 2024-09-16T08:57:03.469 INFO:tasks.workunit.client.1.smithi097.stdout:6/30: unlink d1/d3/f4 0 2024-09-16T08:57:03.469 INFO:tasks.workunit.client.1.smithi097.stdout:6/31: fdatasync d1/d3/f7 0 2024-09-16T08:57:03.469 INFO:tasks.workunit.client.1.smithi097.stdout:3/33: dread f6 [0,4194304] 0 2024-09-16T08:57:03.470 INFO:tasks.workunit.client.1.smithi097.stdout:4/64: write d0/d7/df/f10 [4701339,43933] 0 2024-09-16T08:57:03.471 INFO:tasks.workunit.client.1.smithi097.stdout:7/22: mkdir d4/d9/dc 0 2024-09-16T08:57:03.476 INFO:tasks.workunit.client.1.smithi097.stdout:0/7: dread f0 [0,4194304] 0 2024-09-16T08:57:03.476 INFO:tasks.workunit.client.1.smithi097.stdout:0/8: stat f0 0 2024-09-16T08:57:03.477 INFO:tasks.workunit.client.1.smithi097.stdout:9/19: mknod d2/c7 0 2024-09-16T08:57:03.478 INFO:tasks.workunit.client.1.smithi097.stdout:6/32: symlink d1/d3/l8 0 2024-09-16T08:57:03.480 INFO:tasks.workunit.client.1.smithi097.stdout:3/34: dread f6 [0,4194304] 0 2024-09-16T08:57:03.482 INFO:tasks.workunit.client.1.smithi097.stdout:5/12: getdents . 0 2024-09-16T08:57:03.484 INFO:tasks.workunit.client.1.smithi097.stdout:8/50: dwrite f7 [0,4194304] 0 2024-09-16T08:57:03.486 INFO:tasks.workunit.client.1.smithi097.stdout:6/33: mknod d1/c9 0 2024-09-16T08:57:03.486 INFO:tasks.workunit.client.1.smithi097.stdout:6/34: write d1/d3/f7 [51049,50360] 0 2024-09-16T08:57:03.487 INFO:tasks.workunit.client.1.smithi097.stdout:0/9: dread f0 [0,4194304] 0 2024-09-16T08:57:03.489 INFO:tasks.workunit.client.1.smithi097.stdout:5/13: mknod c2 0 2024-09-16T08:57:03.491 INFO:tasks.workunit.client.1.smithi097.stdout:5/14: dread f0 [0,4194304] 0 2024-09-16T08:57:03.491 INFO:tasks.workunit.client.1.smithi097.stdout:9/20: rmdir d2 39 2024-09-16T08:57:03.494 INFO:tasks.workunit.client.1.smithi097.stdout:7/23: dwrite d4/fb [0,4194304] 0 2024-09-16T08:57:03.494 INFO:tasks.workunit.client.1.smithi097.stdout:7/24: fdatasync d4/f5 0 2024-09-16T08:57:03.495 INFO:tasks.workunit.client.1.smithi097.stdout:8/51: unlink f4 0 2024-09-16T08:57:03.497 INFO:tasks.workunit.client.1.smithi097.stdout:6/35: mkdir d1/da 0 2024-09-16T08:57:03.500 INFO:tasks.workunit.client.1.smithi097.stdout:4/65: rename d0/d3 to d0/d7/df/d15 0 2024-09-16T08:57:03.500 INFO:tasks.workunit.client.1.smithi097.stdout:4/66: truncate d0/fe 1017635 0 2024-09-16T08:57:03.500 INFO:tasks.workunit.client.1.smithi097.stdout:4/67: chown d0/d7 422531912 1 2024-09-16T08:57:03.500 INFO:tasks.workunit.client.1.smithi097.stdout:5/15: mknod c3 0 2024-09-16T08:57:03.500 INFO:tasks.workunit.client.1.smithi097.stdout:9/21: symlink d2/l8 0 2024-09-16T08:57:03.502 INFO:tasks.workunit.client.1.smithi097.stdout:5/16: creat f4 x:0 0 0 2024-09-16T08:57:03.503 INFO:tasks.workunit.client.1.smithi097.stdout:8/52: dread f7 [0,4194304] 0 2024-09-16T08:57:03.505 INFO:tasks.workunit.client.1.smithi097.stdout:8/53: creat d8/dd/fe x:0 0 0 2024-09-16T08:57:03.505 INFO:tasks.workunit.client.1.smithi097.stdout:8/54: chown d8/fa 23002 1 2024-09-16T08:57:03.505 INFO:tasks.workunit.client.1.smithi097.stdout:5/17: dread f0 [0,4194304] 0 2024-09-16T08:57:03.505 INFO:tasks.workunit.client.1.smithi097.stdout:5/18: write f4 [627670,30782] 0 2024-09-16T08:57:03.505 INFO:tasks.workunit.client.1.smithi097.stdout:6/36: dread d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:03.506 INFO:tasks.workunit.client.1.smithi097.stdout:6/37: chown d1/da 0 1 2024-09-16T08:57:03.506 INFO:tasks.workunit.client.1.smithi097.stdout:8/55: mknod d8/dd/cf 0 2024-09-16T08:57:03.507 INFO:tasks.workunit.client.1.smithi097.stdout:5/19: symlink l5 0 2024-09-16T08:57:03.507 INFO:tasks.workunit.client.1.smithi097.stdout:5/20: creat f6 x:0 0 0 2024-09-16T08:57:03.507 INFO:tasks.workunit.client.1.smithi097.stdout:5/21: stat f0 0 2024-09-16T08:57:03.507 INFO:tasks.workunit.client.1.smithi097.stdout:5/22: write f6 [184627,28655] 0 2024-09-16T08:57:03.508 INFO:tasks.workunit.client.1.smithi097.stdout:8/56: creat d8/f10 x:0 0 0 2024-09-16T08:57:03.510 INFO:tasks.workunit.client.1.smithi097.stdout:5/23: mknod c7 0 2024-09-16T08:57:03.510 INFO:tasks.workunit.client.1.smithi097.stdout:5/24: readlink l5 0 2024-09-16T08:57:03.510 INFO:tasks.workunit.client.1.smithi097.stdout:0/10: dwrite f0 [4194304,4194304] 0 2024-09-16T08:57:03.510 INFO:tasks.workunit.client.1.smithi097.stdout:5/25: write f4 [125834,63748] 0 2024-09-16T08:57:03.510 INFO:tasks.workunit.client.1.smithi097.stdout:5/26: truncate f6 419400 0 2024-09-16T08:57:03.511 INFO:tasks.workunit.client.1.smithi097.stdout:5/27: read f4 [434315,44061] 0 2024-09-16T08:57:03.511 INFO:tasks.workunit.client.1.smithi097.stdout:5/28: creat f8 x:0 0 0 2024-09-16T08:57:03.511 INFO:tasks.workunit.client.1.smithi097.stdout:8/57: rename d8/dd/cf to d8/c11 0 2024-09-16T08:57:03.511 INFO:tasks.workunit.client.1.smithi097.stdout:5/29: write f0 [806487,77575] 0 2024-09-16T08:57:03.512 INFO:tasks.workunit.client.1.smithi097.stdout:5/30: chown c7 5959 1 2024-09-16T08:57:03.515 INFO:tasks.workunit.client.1.smithi097.stdout:5/31: link c3 c9 0 2024-09-16T08:57:03.515 INFO:tasks.workunit.client.1.smithi097.stdout:5/32: creat fa x:0 0 0 2024-09-16T08:57:03.517 INFO:tasks.workunit.client.1.smithi097.stdout:5/33: mkdir db 0 2024-09-16T08:57:03.518 INFO:tasks.workunit.client.1.smithi097.stdout:5/34: stat c3 0 2024-09-16T08:57:03.519 INFO:tasks.workunit.client.1.smithi097.stdout:8/58: dread f7 [0,4194304] 0 2024-09-16T08:57:03.520 INFO:tasks.workunit.client.1.smithi097.stdout:5/35: symlink db/lc 0 2024-09-16T08:57:03.521 INFO:tasks.workunit.client.1.smithi097.stdout:5/36: write fa [641403,50382] 0 2024-09-16T08:57:03.521 INFO:tasks.workunit.client.1.smithi097.stdout:5/37: write f6 [1263473,64865] 0 2024-09-16T08:57:03.522 INFO:tasks.workunit.client.1.smithi097.stdout:8/59: mknod d8/c12 0 2024-09-16T08:57:03.522 INFO:tasks.workunit.client.1.smithi097.stdout:8/60: chown l6 417 1 2024-09-16T08:57:03.533 INFO:tasks.workunit.client.1.smithi097.stdout:9/22: truncate d2/f6 1947525 0 2024-09-16T08:57:03.536 INFO:tasks.workunit.client.1.smithi097.stdout:6/38: dwrite d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:03.540 INFO:tasks.workunit.client.1.smithi097.stdout:6/39: rmdir d1/d3 39 2024-09-16T08:57:03.541 INFO:tasks.workunit.client.1.smithi097.stdout:6/40: mkdir d1/d3/db 0 2024-09-16T08:57:03.546 INFO:tasks.workunit.client.1.smithi097.stdout:0/11: dwrite f0 [4194304,4194304] 0 2024-09-16T08:57:03.546 INFO:tasks.workunit.client.1.smithi097.stdout:0/12: fsync f0 0 2024-09-16T08:57:03.546 INFO:tasks.workunit.client.1.smithi097.stdout:8/61: dwrite d8/fa [0,4194304] 0 2024-09-16T08:57:03.547 INFO:tasks.workunit.client.1.smithi097.stdout:8/62: truncate d8/dd/fe 995214 0 2024-09-16T08:57:03.547 INFO:tasks.workunit.client.1.smithi097.stdout:8/63: creat d8/f13 x:0 0 0 2024-09-16T08:57:03.547 INFO:tasks.workunit.client.1.smithi097.stdout:4/68: dwrite d0/f4 [0,4194304] 0 2024-09-16T08:57:03.550 INFO:tasks.workunit.client.1.smithi097.stdout:8/64: symlink d8/l14 0 2024-09-16T08:57:03.551 INFO:tasks.workunit.client.1.smithi097.stdout:5/38: dread f0 [0,4194304] 0 2024-09-16T08:57:03.551 INFO:tasks.workunit.client.1.smithi097.stdout:0/13: write f0 [2666825,28064] 0 2024-09-16T08:57:03.552 INFO:tasks.workunit.client.1.smithi097.stdout:0/14: write f0 [1460302,68250] 0 2024-09-16T08:57:03.552 INFO:tasks.workunit.client.1.smithi097.stdout:5/39: mknod db/cd 0 2024-09-16T08:57:03.553 INFO:tasks.workunit.client.1.smithi097.stdout:7/25: dwrite d4/fb [4194304,4194304] 0 2024-09-16T08:57:03.553 INFO:tasks.workunit.client.1.smithi097.stdout:0/15: rename f0 to f1 0 2024-09-16T08:57:03.554 INFO:tasks.workunit.client.1.smithi097.stdout:5/40: creat db/fe x:0 0 0 2024-09-16T08:57:03.554 INFO:tasks.workunit.client.1.smithi097.stdout:5/41: stat c2 0 2024-09-16T08:57:03.556 INFO:tasks.workunit.client.1.smithi097.stdout:7/26: symlink d4/d9/dc/ld 0 2024-09-16T08:57:03.556 INFO:tasks.workunit.client.1.smithi097.stdout:5/42: read f6 [787155,109788] 0 2024-09-16T08:57:03.556 INFO:tasks.workunit.client.1.smithi097.stdout:5/43: read f6 [563621,43295] 0 2024-09-16T08:57:03.556 INFO:tasks.workunit.client.1.smithi097.stdout:5/44: stat f0 0 2024-09-16T08:57:03.556 INFO:tasks.workunit.client.1.smithi097.stdout:5/45: chown c9 1150 1 2024-09-16T08:57:03.558 INFO:tasks.workunit.client.1.smithi097.stdout:0/16: mknod c2 0 2024-09-16T08:57:03.558 INFO:tasks.workunit.client.1.smithi097.stdout:0/17: chown c2 416 1 2024-09-16T08:57:03.567 INFO:tasks.workunit.client.1.smithi097.stdout:9/23: dread d2/f6 [0,4194304] 0 2024-09-16T08:57:03.568 INFO:tasks.workunit.client.1.smithi097.stdout:6/41: dwrite d1/d3/f5 [0,4194304] 0 2024-09-16T08:57:03.581 INFO:tasks.workunit.client.1.smithi097.stdout:6/42: dread d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:03.581 INFO:tasks.workunit.client.1.smithi097.stdout:8/65: dwrite f7 [0,4194304] 0 2024-09-16T08:57:03.584 INFO:tasks.workunit.client.1.smithi097.stdout:8/66: mkdir d8/dd/d15 0 2024-09-16T08:57:03.584 INFO:tasks.workunit.client.1.smithi097.stdout:8/67: chown d8/c12 1752351 1 2024-09-16T08:57:03.584 INFO:tasks.workunit.client.1.smithi097.stdout:8/68: getdents d8/dd/d15 0 2024-09-16T08:57:03.597 INFO:tasks.workunit.client.1.smithi097.stdout:8/69: dread f7 [0,4194304] 0 2024-09-16T08:57:03.597 INFO:tasks.workunit.client.1.smithi097.stdout:5/46: dwrite f0 [0,4194304] 0 2024-09-16T08:57:03.597 INFO:tasks.workunit.client.1.smithi097.stdout:8/70: write d8/fc [1830324,39619] 0 2024-09-16T08:57:03.598 INFO:tasks.workunit.client.1.smithi097.stdout:5/47: mknod db/cf 0 2024-09-16T08:57:03.610 INFO:tasks.workunit.client.1.smithi097.stdout:7/27: dwrite f1 [0,4194304] 0 2024-09-16T08:57:03.611 INFO:tasks.workunit.client.1.smithi097.stdout:7/28: rmdir d4/d9 39 2024-09-16T08:57:03.611 INFO:tasks.workunit.client.1.smithi097.stdout:7/29: creat d4/fe x:0 0 0 2024-09-16T08:57:03.612 INFO:tasks.workunit.client.1.smithi097.stdout:7/30: creat d4/d7/ff x:0 0 0 2024-09-16T08:57:03.613 INFO:tasks.workunit.client.1.smithi097.stdout:7/31: symlink d4/d9/dc/l10 0 2024-09-16T08:57:03.622 INFO:tasks.workunit.client.1.smithi097.stdout:8/71: dwrite f7 [0,4194304] 0 2024-09-16T08:57:03.622 INFO:tasks.workunit.client.1.smithi097.stdout:9/24: dwrite d2/f6 [0,4194304] 0 2024-09-16T08:57:03.622 INFO:tasks.workunit.client.1.smithi097.stdout:0/18: dwrite f1 [0,4194304] 0 2024-09-16T08:57:03.622 INFO:tasks.workunit.client.1.smithi097.stdout:0/19: chown f1 1927 1 2024-09-16T08:57:03.622 INFO:tasks.workunit.client.1.smithi097.stdout:6/43: dwrite d1/d3/f5 [0,4194304] 0 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr fail", "who": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: Activating manager daemon smithi097.mpashf 2024-09-16T08:57:03.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: osdmap e48: 6 total, 6 up, 6 in 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "mgr fail", "who": "smithi031.dcmqpv"}]': finished 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: mgrmap e21: smithi097.mpashf(active, starting, since 0.00802445s) 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:57:03.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:57:03.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:57:03.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:57:03.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:57:03.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:57:03.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:57:03.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:57:03.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:03 smithi031 ceph-mon[27844]: Manager daemon smithi097.mpashf is now available 2024-09-16T08:57:03.626 INFO:tasks.workunit.client.1.smithi097.stdout:7/32: dread d4/fb [4194304,4194304] 0 2024-09-16T08:57:03.631 INFO:tasks.workunit.client.1.smithi097.stdout:6/44: dread d1/d3/f5 [0,4194304] 0 2024-09-16T08:57:03.635 INFO:tasks.workunit.client.1.smithi097.stdout:6/45: dread d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:03.670 INFO:tasks.workunit.client.1.smithi097.stdout:8/72: unlink d8/l9 0 2024-09-16T08:57:03.670 INFO:tasks.workunit.client.1.smithi097.stdout:8/73: write d8/f10 [278163,120065] 0 2024-09-16T08:57:03.671 INFO:tasks.workunit.client.1.smithi097.stdout:9/25: symlink d2/l9 0 2024-09-16T08:57:03.673 INFO:tasks.workunit.client.1.smithi097.stdout:7/33: rmdir d4/d9 39 2024-09-16T08:57:03.673 INFO:tasks.workunit.client.1.smithi097.stdout:7/34: creat d4/d7/f11 x:0 0 0 2024-09-16T08:57:03.675 INFO:tasks.workunit.client.1.smithi097.stdout:6/46: rmdir d1/da 0 2024-09-16T08:57:03.678 INFO:tasks.workunit.client.1.smithi097.stdout:9/26: creat d2/d5/fa x:0 0 0 2024-09-16T08:57:03.678 INFO:tasks.workunit.client.1.smithi097.stdout:9/27: fsync d2/f4 0 2024-09-16T08:57:03.680 INFO:tasks.workunit.client.1.smithi097.stdout:7/35: mkdir d4/d12 0 2024-09-16T08:57:03.682 INFO:tasks.workunit.client.1.smithi097.stdout:6/47: rename d1/d3/db to d1/dc 0 2024-09-16T08:57:03.683 INFO:tasks.workunit.client.1.smithi097.stdout:7/36: mknod d4/d9/c13 0 2024-09-16T08:57:03.683 INFO:tasks.workunit.client.1.smithi097.stdout:7/37: truncate d4/d7/f11 375705 0 2024-09-16T08:57:03.688 INFO:tasks.workunit.client.1.smithi097.stdout:6/48: link d1/d3/c6 d1/dc/cd 0 2024-09-16T08:57:03.689 INFO:tasks.workunit.client.1.smithi097.stdout:6/49: mknod d1/ce 0 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:6/50: readlink d1/d3/l8 0 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:6/51: chown d1/dc 355 1 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:6/52: chown d1/d3/c6 6983478 1 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:5/48: rmdir db 39 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:9/28: write d2/f6 [315239,2733] 0 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:9/29: creat d2/d5/fb x:0 0 0 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:9/30: dread - d2/f4 zero size 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:9/31: dread - d2/f4 zero size 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:9/32: read - d2/f4 zero size 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:6/53: mkdir d1/d3/df 0 2024-09-16T08:57:03.690 INFO:tasks.workunit.client.1.smithi097.stdout:6/54: fsync d1/d3/f5 0 2024-09-16T08:57:03.691 INFO:tasks.workunit.client.1.smithi097.stdout:6/55: chown d1/dc 380104 1 2024-09-16T08:57:03.691 INFO:tasks.workunit.client.1.smithi097.stdout:5/49: creat db/f10 x:0 0 0 2024-09-16T08:57:03.691 INFO:tasks.workunit.client.1.smithi097.stdout:5/50: chown c9 3476 1 2024-09-16T08:57:03.692 INFO:tasks.workunit.client.1.smithi097.stdout:6/56: read d1/d3/f7 [742557,51871] 0 2024-09-16T08:57:03.692 INFO:tasks.workunit.client.1.smithi097.stdout:5/51: symlink db/l11 0 2024-09-16T08:57:03.694 INFO:tasks.workunit.client.1.smithi097.stdout:8/74: dwrite d8/fa [0,4194304] 0 2024-09-16T08:57:03.695 INFO:tasks.workunit.client.1.smithi097.stdout:5/52: mknod db/c12 0 2024-09-16T08:57:03.703 INFO:tasks.workunit.client.1.smithi097.stdout:8/75: dread d8/fa [0,4194304] 0 2024-09-16T08:57:03.705 INFO:tasks.workunit.client.1.smithi097.stdout:8/76: mkdir d8/dd/d15/d16 0 2024-09-16T08:57:03.706 INFO:tasks.workunit.client.1.smithi097.stdout:8/77: chown d8/fa 12137 1 2024-09-16T08:57:03.706 INFO:tasks.workunit.client.1.smithi097.stdout:8/78: mknod d8/dd/c17 0 2024-09-16T08:57:03.708 INFO:tasks.workunit.client.1.smithi097.stdout:8/79: truncate d8/fc 1718445 0 2024-09-16T08:57:03.708 INFO:tasks.workunit.client.1.smithi097.stdout:2/11: sync 2024-09-16T08:57:03.711 INFO:tasks.workunit.client.1.smithi097.stdout:5/53: dread fa [0,4194304] 0 2024-09-16T08:57:03.717 INFO:tasks.workunit.client.1.smithi097.stdout:7/38: dwrite f2 [0,4194304] 0 2024-09-16T08:57:03.718 INFO:tasks.workunit.client.1.smithi097.stdout:9/33: dwrite f0 [0,4194304] 0 2024-09-16T08:57:03.718 INFO:tasks.workunit.client.1.smithi097.stdout:6/57: dwrite d1/d3/f5 [0,4194304] 0 2024-09-16T08:57:03.719 INFO:tasks.workunit.client.1.smithi097.stdout:2/12: write f2 [3457965,2438] 0 2024-09-16T08:57:03.720 INFO:tasks.workunit.client.1.smithi097.stdout:7/39: rename f1 to d4/d7/f14 0 2024-09-16T08:57:03.722 INFO:tasks.workunit.client.1.smithi097.stdout:6/58: dread d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:03.723 INFO:tasks.workunit.client.1.smithi097.stdout:9/34: creat d2/fc x:0 0 0 2024-09-16T08:57:03.723 INFO:tasks.workunit.client.1.smithi097.stdout:9/35: write d2/f4 [875964,23678] 0 2024-09-16T08:57:03.725 INFO:tasks.workunit.client.1.smithi097.stdout:2/13: mkdir d3/d4 0 2024-09-16T08:57:03.727 INFO:tasks.workunit.client.1.smithi097.stdout:2/14: symlink d3/d4/l5 0 2024-09-16T08:57:03.729 INFO:tasks.workunit.client.1.smithi097.stdout:2/15: creat d3/d4/f6 x:0 0 0 2024-09-16T08:57:03.729 INFO:tasks.workunit.client.1.smithi097.stdout:2/16: write f2 [2183702,126345] 0 2024-09-16T08:57:03.729 INFO:tasks.workunit.client.1.smithi097.stdout:2/17: rename d3 to d3/d4/d7 22 2024-09-16T08:57:03.730 INFO:tasks.workunit.client.1.smithi097.stdout:2/18: unlink f1 0 2024-09-16T08:57:03.730 INFO:tasks.workunit.client.1.smithi097.stdout:2/19: stat d3/d4/l5 0 2024-09-16T08:57:03.731 INFO:tasks.workunit.client.1.smithi097.stdout:7/40: dread d4/d7/f14 [0,4194304] 0 2024-09-16T08:57:03.731 INFO:tasks.workunit.client.1.smithi097.stdout:9/36: getdents d2/d5 0 2024-09-16T08:57:03.732 INFO:tasks.workunit.client.1.smithi097.stdout:7/41: dread d4/d7/f11 [0,4194304] 0 2024-09-16T08:57:03.732 INFO:tasks.workunit.client.1.smithi097.stdout:5/54: dwrite f6 [0,4194304] 0 2024-09-16T08:57:03.733 INFO:tasks.workunit.client.1.smithi097.stdout:6/59: write d1/d3/f5 [1428654,58837] 0 2024-09-16T08:57:03.734 INFO:tasks.workunit.client.1.smithi097.stdout:7/42: link d4/fb d4/d9/f15 0 2024-09-16T08:57:03.739 INFO:tasks.workunit.client.1.smithi097.stdout:5/55: rmdir db 39 2024-09-16T08:57:03.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:57:03.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:57:03.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:03.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:57:03.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:57:03.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:03.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr fail", "who": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: Activating manager daemon smithi097.mpashf 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: osdmap e48: 6 total, 6 up, 6 in 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.14211 172.21.15.31:0/1472165832' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "mgr fail", "who": "smithi031.dcmqpv"}]': finished 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: mgrmap e21: smithi097.mpashf(active, starting, since 0.00802445s) 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:57:03.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:57:03.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:03 smithi097 ceph-mon[35478]: Manager daemon smithi097.mpashf is now available 2024-09-16T08:57:03.777 INFO:tasks.workunit.client.1.smithi097.stdout:2/20: dwrite f2 [0,4194304] 0 2024-09-16T08:57:03.777 INFO:tasks.workunit.client.1.smithi097.stdout:2/21: chown d3/d4 1580596 1 2024-09-16T08:57:03.777 INFO:tasks.workunit.client.1.smithi097.stdout:6/60: dwrite d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:03.777 INFO:tasks.workunit.client.1.smithi097.stdout:6/61: rename d1/dc to d1/dc/d10 22 2024-09-16T08:57:03.778 INFO:tasks.workunit.client.1.smithi097.stdout:9/37: dwrite f0 [0,4194304] 0 2024-09-16T08:57:03.778 INFO:tasks.workunit.client.1.smithi097.stdout:5/56: dwrite fa [0,4194304] 0 2024-09-16T08:57:03.778 INFO:tasks.workunit.client.1.smithi097.stdout:5/57: truncate f6 4555613 0 2024-09-16T08:57:03.778 INFO:tasks.workunit.client.1.smithi097.stdout:7/43: dwrite d4/d7/f14 [4194304,4194304] 0 2024-09-16T08:57:03.788 INFO:tasks.workunit.client.1.smithi097.stdout:7/44: dread f2 [0,4194304] 0 2024-09-16T08:57:03.788 INFO:tasks.workunit.client.1.smithi097.stdout:7/45: fdatasync d4/fe 0 2024-09-16T08:57:03.805 INFO:tasks.workunit.client.1.smithi097.stdout:2/22: mkdir d3/d4/d8 0 2024-09-16T08:57:03.805 INFO:tasks.workunit.client.1.smithi097.stdout:2/23: chown d3/d4/l5 1002026 1 2024-09-16T08:57:03.805 INFO:tasks.workunit.client.1.smithi097.stdout:2/24: fsync d3/d4/f6 0 2024-09-16T08:57:03.805 INFO:tasks.workunit.client.1.smithi097.stdout:2/25: write d3/d4/f6 [414017,124269] 0 2024-09-16T08:57:03.805 INFO:tasks.workunit.client.1.smithi097.stdout:2/26: write d3/d4/f6 [937913,49152] 0 2024-09-16T08:57:03.810 INFO:tasks.workunit.client.1.smithi097.stdout:6/62: creat d1/d3/f11 x:0 0 0 2024-09-16T08:57:03.810 INFO:tasks.workunit.client.1.smithi097.stdout:6/63: read - d1/d3/f11 zero size 2024-09-16T08:57:03.819 INFO:tasks.workunit.client.1.smithi097.stdout:9/38: symlink d2/ld 0 2024-09-16T08:57:03.819 INFO:tasks.workunit.client.1.smithi097.stdout:9/39: creat d2/fe x:0 0 0 2024-09-16T08:57:03.820 INFO:tasks.workunit.client.1.smithi097.stdout:9/40: write d2/d5/fa [40637,24078] 0 2024-09-16T08:57:03.820 INFO:tasks.workunit.client.1.smithi097.stdout:9/41: write f0 [5160389,88098] 0 2024-09-16T08:57:03.820 INFO:tasks.workunit.client.1.smithi097.stdout:9/42: write d2/fe [623555,54886] 0 2024-09-16T08:57:03.820 INFO:tasks.workunit.client.1.smithi097.stdout:9/43: chown f0 1 1 2024-09-16T08:57:03.827 INFO:tasks.workunit.client.1.smithi097.stdout:5/58: creat db/f13 x:0 0 0 2024-09-16T08:57:03.832 INFO:tasks.workunit.client.1.smithi097.stdout:9/44: dwrite d2/f4 [0,4194304] 0 2024-09-16T08:57:03.832 INFO:tasks.workunit.client.1.smithi097.stdout:9/45: chown d2/f4 811697735 1 2024-09-16T08:57:03.835 INFO:tasks.workunit.client.1.smithi097.stdout:9/46: fsync f0 0 2024-09-16T08:57:03.835 INFO:tasks.workunit.client.1.smithi097.stdout:9/47: fsync d2/f6 0 2024-09-16T08:57:03.838 INFO:tasks.workunit.client.1.smithi097.stdout:7/46: unlink d4/d7/l8 0 2024-09-16T08:57:03.838 INFO:tasks.workunit.client.1.smithi097.stdout:7/47: stat d4/fe 0 2024-09-16T08:57:03.849 INFO:tasks.workunit.client.1.smithi097.stdout:2/27: mkdir d3/d4/d9 0 2024-09-16T08:57:03.851 INFO:tasks.workunit.client.1.smithi097.stdout:6/64: creat d1/f12 x:0 0 0 2024-09-16T08:57:03.855 INFO:tasks.workunit.client.1.smithi097.stdout:7/48: dwrite d4/d7/ff [0,4194304] 0 2024-09-16T08:57:03.855 INFO:tasks.workunit.client.1.smithi097.stdout:9/48: rename d2/l8 to d2/d5/lf 0 2024-09-16T08:57:03.856 INFO:tasks.workunit.client.1.smithi097.stdout:6/65: symlink d1/l13 0 2024-09-16T08:57:03.860 INFO:tasks.workunit.client.1.smithi097.stdout:7/49: creat d4/d9/dc/f16 x:0 0 0 2024-09-16T08:57:03.860 INFO:tasks.workunit.client.1.smithi097.stdout:7/50: creat d4/f17 x:0 0 0 2024-09-16T08:57:03.862 INFO:tasks.workunit.client.1.smithi097.stdout:9/49: symlink d2/l10 0 2024-09-16T08:57:03.862 INFO:tasks.workunit.client.1.smithi097.stdout:9/50: write d2/fc [512925,58059] 0 2024-09-16T08:57:03.862 INFO:tasks.workunit.client.1.smithi097.stdout:9/51: stat d2/c3 0 2024-09-16T08:57:03.867 INFO:tasks.workunit.client.1.smithi097.stdout:7/51: write d4/d7/f14 [4718367,30848] 0 2024-09-16T08:57:03.870 INFO:tasks.workunit.client.1.smithi097.stdout:7/52: write d4/d7/f14 [1659500,86883] 0 2024-09-16T08:57:03.871 INFO:tasks.workunit.client.1.smithi097.stdout:7/53: mknod d4/c18 0 2024-09-16T08:57:03.873 INFO:tasks.workunit.client.1.smithi097.stdout:7/54: truncate d4/d9/f15 2605678 0 2024-09-16T08:57:03.873 INFO:tasks.workunit.client.1.smithi097.stdout:7/55: stat d4/d9/dc/ld 0 2024-09-16T08:57:03.873 INFO:tasks.workunit.client.1.smithi097.stdout:6/66: fsync d1/d3/f11 0 2024-09-16T08:57:03.874 INFO:tasks.workunit.client.1.smithi097.stdout:6/67: symlink d1/d3/l14 0 2024-09-16T08:57:03.881 INFO:tasks.workunit.client.1.smithi097.stdout:5/59: getdents db 0 2024-09-16T08:57:03.882 INFO:tasks.workunit.client.1.smithi097.stdout:1/55: sync 2024-09-16T08:57:03.884 INFO:tasks.workunit.client.1.smithi097.stdout:4/69: sync 2024-09-16T08:57:03.885 INFO:tasks.workunit.client.1.smithi097.stdout:3/35: sync 2024-09-16T08:57:03.885 INFO:tasks.workunit.client.1.smithi097.stdout:4/70: fsync d0/d7/df/d15/f12 0 2024-09-16T08:57:03.885 INFO:tasks.workunit.client.1.smithi097.stdout:8/80: sync 2024-09-16T08:57:03.885 INFO:tasks.workunit.client.1.smithi097.stdout:0/20: sync 2024-09-16T08:57:03.886 INFO:tasks.workunit.client.1.smithi097.stdout:3/36: getdents d7 0 2024-09-16T08:57:03.886 INFO:tasks.workunit.client.1.smithi097.stdout:3/37: write f6 [1389995,85497] 0 2024-09-16T08:57:03.887 INFO:tasks.workunit.client.1.smithi097.stdout:4/71: mkdir d0/d7/df/d15/d16 0 2024-09-16T08:57:03.888 INFO:tasks.workunit.client.1.smithi097.stdout:0/21: truncate f1 310222 0 2024-09-16T08:57:03.888 INFO:tasks.workunit.client.1.smithi097.stdout:0/22: chown f1 1941011865 1 2024-09-16T08:57:03.889 INFO:tasks.workunit.client.1.smithi097.stdout:8/81: mknod d8/dd/d15/c18 0 2024-09-16T08:57:03.891 INFO:tasks.workunit.client.1.smithi097.stdout:5/60: dread fa [0,4194304] 0 2024-09-16T08:57:03.891 INFO:tasks.workunit.client.1.smithi097.stdout:5/61: dread - db/fe zero size 2024-09-16T08:57:03.891 INFO:tasks.workunit.client.1.smithi097.stdout:3/38: unlink c2 0 2024-09-16T08:57:03.891 INFO:tasks.workunit.client.1.smithi097.stdout:0/23: symlink l3 0 2024-09-16T08:57:03.892 INFO:tasks.workunit.client.1.smithi097.stdout:8/82: creat d8/dd/f19 x:0 0 0 2024-09-16T08:57:03.897 INFO:tasks.workunit.client.1.smithi097.stdout:5/62: creat db/f14 x:0 0 0 2024-09-16T08:57:03.897 INFO:tasks.workunit.client.1.smithi097.stdout:3/39: creat d7/f9 x:0 0 0 2024-09-16T08:57:03.897 INFO:tasks.workunit.client.1.smithi097.stdout:3/40: write f6 [3273302,14714] 0 2024-09-16T08:57:03.897 INFO:tasks.workunit.client.1.smithi097.stdout:4/72: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:03.897 INFO:tasks.workunit.client.1.smithi097.stdout:4/73: chown d0/d7/df 9732 1 2024-09-16T08:57:03.897 INFO:tasks.workunit.client.1.smithi097.stdout:3/41: link d7/f9 d7/fa 0 2024-09-16T08:57:03.898 INFO:tasks.workunit.client.1.smithi097.stdout:4/74: symlink d0/d7/df/d15/l17 0 2024-09-16T08:57:03.899 INFO:tasks.workunit.client.1.smithi097.stdout:3/42: rename d7/l8 to d7/lb 0 2024-09-16T08:57:03.899 INFO:tasks.workunit.client.1.smithi097.stdout:3/43: dread - d7/f9 zero size 2024-09-16T08:57:03.900 INFO:tasks.workunit.client.1.smithi097.stdout:9/52: getdents d2/d5 0 2024-09-16T08:57:03.901 INFO:tasks.workunit.client.1.smithi097.stdout:9/53: chown d2/f6 0 1 2024-09-16T08:57:03.901 INFO:tasks.workunit.client.1.smithi097.stdout:9/54: dread - d2/d5/fb zero size 2024-09-16T08:57:03.901 INFO:tasks.workunit.client.1.smithi097.stdout:9/55: readlink d2/ld 0 2024-09-16T08:57:03.901 INFO:tasks.workunit.client.1.smithi097.stdout:9/56: fsync d2/fe 0 2024-09-16T08:57:03.903 INFO:tasks.workunit.client.1.smithi097.stdout:7/56: fsync d4/d9/dc/f16 0 2024-09-16T08:57:03.905 INFO:tasks.workunit.client.1.smithi097.stdout:7/57: mkdir d4/d7/d19 0 2024-09-16T08:57:03.907 INFO:tasks.workunit.client.1.smithi097.stdout:7/58: mknod d4/d7/c1a 0 2024-09-16T08:57:03.907 INFO:tasks.workunit.client.1.smithi097.stdout:7/59: fdatasync d4/d7/f11 0 2024-09-16T08:57:03.907 INFO:tasks.workunit.client.1.smithi097.stdout:7/60: write f2 [4987799,112371] 0 2024-09-16T08:57:03.909 INFO:tasks.workunit.client.1.smithi097.stdout:7/61: mkdir d4/d1b 0 2024-09-16T08:57:03.909 INFO:tasks.workunit.client.1.smithi097.stdout:7/62: read d4/d7/f14 [4234123,65756] 0 2024-09-16T08:57:03.914 INFO:tasks.workunit.client.1.smithi097.stdout:5/63: dwrite f8 [0,4194304] 0 2024-09-16T08:57:03.914 INFO:tasks.workunit.client.1.smithi097.stdout:5/64: write db/fe [1033425,19185] 0 2024-09-16T08:57:03.917 INFO:tasks.workunit.client.1.smithi097.stdout:0/24: fsync f1 0 2024-09-16T08:57:03.917 INFO:tasks.workunit.client.1.smithi097.stdout:0/25: write f1 [1294369,89362] 0 2024-09-16T08:57:03.917 INFO:tasks.workunit.client.1.smithi097.stdout:0/26: write f1 [1717005,120043] 0 2024-09-16T08:57:03.918 INFO:tasks.workunit.client.1.smithi097.stdout:0/27: chown f1 2124641 1 2024-09-16T08:57:03.918 INFO:tasks.workunit.client.1.smithi097.stdout:0/28: creat f4 x:0 0 0 2024-09-16T08:57:03.918 INFO:tasks.workunit.client.1.smithi097.stdout:0/29: creat f5 x:0 0 0 2024-09-16T08:57:03.920 INFO:tasks.workunit.client.1.smithi097.stdout:3/44: fsync d7/fa 0 2024-09-16T08:57:03.922 INFO:tasks.workunit.client.1.smithi097.stdout:5/65: write fa [2669309,46831] 0 2024-09-16T08:57:03.924 INFO:tasks.workunit.client.1.smithi097.stdout:3/45: mknod d7/cc 0 2024-09-16T08:57:03.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/46: truncate d7/f9 931349 0 2024-09-16T08:57:03.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/47: rename d7 to d7/dd 22 2024-09-16T08:57:03.926 INFO:tasks.workunit.client.1.smithi097.stdout:5/66: unlink db/cf 0 2024-09-16T08:57:03.927 INFO:tasks.workunit.client.1.smithi097.stdout:0/30: fsync f1 0 2024-09-16T08:57:03.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/75: dwrite d0/d7/df/d15/f12 [0,4194304] 0 2024-09-16T08:57:03.927 INFO:tasks.workunit.client.1.smithi097.stdout:7/63: write d4/d9/f15 [2455199,52346] 0 2024-09-16T08:57:03.928 INFO:tasks.workunit.client.1.smithi097.stdout:9/57: dwrite d2/f4 [0,4194304] 0 2024-09-16T08:57:03.928 INFO:tasks.workunit.client.1.smithi097.stdout:9/58: write d2/fe [1485047,79149] 0 2024-09-16T08:57:03.933 INFO:tasks.workunit.client.1.smithi097.stdout:0/31: rename f4 to f6 0 2024-09-16T08:57:03.936 INFO:tasks.workunit.client.1.smithi097.stdout:4/76: unlink d0/fa 0 2024-09-16T08:57:03.936 INFO:tasks.workunit.client.1.smithi097.stdout:4/77: getdents d0/d7/df/d15/d16 0 2024-09-16T08:57:03.936 INFO:tasks.workunit.client.1.smithi097.stdout:4/78: write d0/fe [1376084,108584] 0 2024-09-16T08:57:03.937 INFO:tasks.workunit.client.1.smithi097.stdout:7/64: mkdir d4/d9/d1c 0 2024-09-16T08:57:03.937 INFO:tasks.workunit.client.1.smithi097.stdout:7/65: dread - d4/f17 zero size 2024-09-16T08:57:03.937 INFO:tasks.workunit.client.1.smithi097.stdout:7/66: dread - d4/fe zero size 2024-09-16T08:57:03.939 INFO:tasks.workunit.client.1.smithi097.stdout:0/32: dread f1 [0,4194304] 0 2024-09-16T08:57:03.939 INFO:tasks.workunit.client.1.smithi097.stdout:0/33: write f5 [518053,114036] 0 2024-09-16T08:57:03.939 INFO:tasks.workunit.client.1.smithi097.stdout:0/34: dread - f6 zero size 2024-09-16T08:57:03.939 INFO:tasks.workunit.client.1.smithi097.stdout:4/79: unlink d0/d7/df/f10 0 2024-09-16T08:57:03.939 INFO:tasks.workunit.client.1.smithi097.stdout:4/80: write d0/d7/df/f13 [395823,11970] 0 2024-09-16T08:57:03.939 INFO:tasks.workunit.client.1.smithi097.stdout:4/81: getdents d0/d7 0 2024-09-16T08:57:03.940 INFO:tasks.workunit.client.1.smithi097.stdout:9/59: dread d2/f4 [0,4194304] 0 2024-09-16T08:57:03.940 INFO:tasks.workunit.client.1.smithi097.stdout:3/48: getdents d7 0 2024-09-16T08:57:03.941 INFO:tasks.workunit.client.1.smithi097.stdout:9/60: dread d2/d5/fa [0,4194304] 0 2024-09-16T08:57:03.941 INFO:tasks.workunit.client.1.smithi097.stdout:9/61: dread - d2/d5/fb zero size 2024-09-16T08:57:03.941 INFO:tasks.workunit.client.1.smithi097.stdout:7/67: getdents d4 0 2024-09-16T08:57:03.943 INFO:tasks.workunit.client.1.smithi097.stdout:3/49: unlink f6 0 2024-09-16T08:57:03.944 INFO:tasks.workunit.client.1.smithi097.stdout:0/35: dread f5 [0,4194304] 0 2024-09-16T08:57:03.944 INFO:tasks.workunit.client.1.smithi097.stdout:0/36: creat f7 x:0 0 0 2024-09-16T08:57:03.945 INFO:tasks.workunit.client.1.smithi097.stdout:7/68: mknod d4/d1b/c1d 0 2024-09-16T08:57:03.945 INFO:tasks.workunit.client.1.smithi097.stdout:7/69: chown d4/d7/ff 155727 1 2024-09-16T08:57:03.945 INFO:tasks.workunit.client.1.smithi097.stdout:4/82: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:03.946 INFO:tasks.workunit.client.1.smithi097.stdout:3/50: unlink d7/f9 0 2024-09-16T08:57:03.953 INFO:tasks.workunit.client.1.smithi097.stdout:0/37: link f6 f8 0 2024-09-16T08:57:03.953 INFO:tasks.workunit.client.1.smithi097.stdout:7/70: creat d4/d1b/f1e x:0 0 0 2024-09-16T08:57:03.953 INFO:tasks.workunit.client.1.smithi097.stdout:7/71: fdatasync d4/d1b/f1e 0 2024-09-16T08:57:03.953 INFO:tasks.workunit.client.1.smithi097.stdout:7/72: chown l0 0 1 2024-09-16T08:57:03.953 INFO:tasks.workunit.client.1.smithi097.stdout:7/73: dread - d4/fe zero size 2024-09-16T08:57:03.955 INFO:tasks.workunit.client.1.smithi097.stdout:4/83: creat d0/d7/f18 x:0 0 0 2024-09-16T08:57:03.958 INFO:tasks.workunit.client.1.smithi097.stdout:0/38: mknod c9 0 2024-09-16T08:57:03.958 INFO:tasks.workunit.client.1.smithi097.stdout:0/39: truncate f8 717700 0 2024-09-16T08:57:03.958 INFO:tasks.workunit.client.1.smithi097.stdout:0/40: chown f6 1 1 2024-09-16T08:57:03.959 INFO:tasks.workunit.client.1.smithi097.stdout:0/41: write f6 [269882,39796] 0 2024-09-16T08:57:03.959 INFO:tasks.workunit.client.1.smithi097.stdout:7/74: mkdir d4/d12/d1f 0 2024-09-16T08:57:03.959 INFO:tasks.workunit.client.1.smithi097.stdout:7/75: readlink d4/d9/dc/l10 0 2024-09-16T08:57:03.960 INFO:tasks.workunit.client.1.smithi097.stdout:0/42: mkdir da 0 2024-09-16T08:57:03.962 INFO:tasks.workunit.client.1.smithi097.stdout:7/76: symlink d4/d9/l20 0 2024-09-16T08:57:03.962 INFO:tasks.workunit.client.1.smithi097.stdout:7/77: chown d4/d1b/f1e 4686975 1 2024-09-16T08:57:03.962 INFO:tasks.workunit.client.1.smithi097.stdout:7/78: write d4/fb [2832118,610] 0 2024-09-16T08:57:03.962 INFO:tasks.workunit.client.1.smithi097.stdout:7/79: write d4/fe [38612,89195] 0 2024-09-16T08:57:03.963 INFO:tasks.workunit.client.1.smithi097.stdout:0/43: unlink c9 0 2024-09-16T08:57:03.963 INFO:tasks.workunit.client.1.smithi097.stdout:0/44: write f6 [375409,73429] 0 2024-09-16T08:57:03.964 INFO:tasks.workunit.client.1.smithi097.stdout:0/45: creat da/fb x:0 0 0 2024-09-16T08:57:03.965 INFO:tasks.workunit.client.1.smithi097.stdout:9/62: dwrite d2/fe [0,4194304] 0 2024-09-16T08:57:03.967 INFO:tasks.workunit.client.1.smithi097.stdout:0/46: mkdir da/dc 0 2024-09-16T08:57:03.971 INFO:tasks.workunit.client.1.smithi097.stdout:4/84: dwrite d0/d7/f18 [0,4194304] 0 2024-09-16T08:57:03.975 INFO:tasks.workunit.client.1.smithi097.stdout:7/80: dread d4/f5 [0,4194304] 0 2024-09-16T08:57:03.975 INFO:tasks.workunit.client.1.smithi097.stdout:7/81: chown d4/d7 3 1 2024-09-16T08:57:03.975 INFO:tasks.workunit.client.1.smithi097.stdout:7/82: chown d4/d7 804223 1 2024-09-16T08:57:03.975 INFO:tasks.workunit.client.1.smithi097.stdout:7/83: readlink l0 0 2024-09-16T08:57:03.977 INFO:tasks.workunit.client.1.smithi097.stdout:4/85: write d0/f4 [2871352,99977] 0 2024-09-16T08:57:03.977 INFO:tasks.workunit.client.1.smithi097.stdout:4/86: readlink d0/d7/df/d15/l17 0 2024-09-16T08:57:03.977 INFO:tasks.workunit.client.1.smithi097.stdout:4/87: chown d0/d7/df/d15/f11 31855187 1 2024-09-16T08:57:03.977 INFO:tasks.workunit.client.1.smithi097.stdout:4/88: fsync d0/d7/df/f13 0 2024-09-16T08:57:03.983 INFO:tasks.workunit.client.1.smithi097.stdout:7/84: dread f2 [0,4194304] 0 2024-09-16T08:57:03.983 INFO:tasks.workunit.client.1.smithi097.stdout:4/89: rmdir d0 39 2024-09-16T08:57:03.984 INFO:tasks.workunit.client.1.smithi097.stdout:7/85: fdatasync d4/d7/f14 0 2024-09-16T08:57:03.984 INFO:tasks.workunit.client.1.smithi097.stdout:7/86: chown d4/d9/dc 5202 1 2024-09-16T08:57:03.984 INFO:tasks.workunit.client.1.smithi097.stdout:7/87: chown d4/d7/ca 512 1 2024-09-16T08:57:03.984 INFO:tasks.workunit.client.1.smithi097.stdout:7/88: write d4/d7/f14 [5484127,33761] 0 2024-09-16T08:57:03.985 INFO:tasks.workunit.client.1.smithi097.stdout:4/90: creat d0/d7/df/d15/d16/f19 x:0 0 0 2024-09-16T08:57:03.986 INFO:tasks.workunit.client.1.smithi097.stdout:4/91: unlink d0/d7/c9 0 2024-09-16T08:57:03.988 INFO:tasks.workunit.client.1.smithi097.stdout:4/92: rename d0/l14 to d0/l1a 0 2024-09-16T08:57:03.990 INFO:tasks.workunit.client.1.smithi097.stdout:0/47: dwrite f7 [0,4194304] 0 2024-09-16T08:57:03.992 INFO:tasks.workunit.client.1.smithi097.stdout:0/48: dread f5 [0,4194304] 0 2024-09-16T08:57:03.992 INFO:tasks.workunit.client.1.smithi097.stdout:0/49: fsync f1 0 2024-09-16T08:57:03.994 INFO:tasks.workunit.client.1.smithi097.stdout:4/93: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:03.994 INFO:tasks.workunit.client.1.smithi097.stdout:4/94: chown d0/l1a 275807654 1 2024-09-16T08:57:03.994 INFO:tasks.workunit.client.1.smithi097.stdout:0/50: dread f5 [0,4194304] 0 2024-09-16T08:57:03.994 INFO:tasks.workunit.client.1.smithi097.stdout:4/95: creat d0/d7/df/d15/f1b x:0 0 0 2024-09-16T08:57:03.994 INFO:tasks.workunit.client.1.smithi097.stdout:4/96: stat d0/d7/df/d15/f11 0 2024-09-16T08:57:03.995 INFO:tasks.workunit.client.1.smithi097.stdout:4/97: rename d0/c5 to d0/c1c 0 2024-09-16T08:57:03.996 INFO:tasks.workunit.client.1.smithi097.stdout:4/98: fdatasync d0/d7/df/d15/f11 0 2024-09-16T08:57:03.996 INFO:tasks.workunit.client.1.smithi097.stdout:4/99: creat d0/d7/df/d15/d16/f1d x:0 0 0 2024-09-16T08:57:03.997 INFO:tasks.workunit.client.1.smithi097.stdout:4/100: creat d0/d7/df/f1e x:0 0 0 2024-09-16T08:57:03.997 INFO:tasks.workunit.client.1.smithi097.stdout:4/101: fdatasync d0/d7/df/d15/d16/f19 0 2024-09-16T08:57:04.007 INFO:tasks.workunit.client.1.smithi097.stdout:0/51: dwrite f6 [0,4194304] 0 2024-09-16T08:57:04.008 INFO:tasks.workunit.client.1.smithi097.stdout:0/52: mknod da/dc/cd 0 2024-09-16T08:57:04.008 INFO:tasks.workunit.client.1.smithi097.stdout:0/53: truncate da/fb 516461 0 2024-09-16T08:57:04.009 INFO:tasks.workunit.client.1.smithi097.stdout:0/54: write da/fb [1356394,124911] 0 2024-09-16T08:57:04.009 INFO:tasks.workunit.client.1.smithi097.stdout:0/55: mknod da/dc/ce 0 2024-09-16T08:57:04.010 INFO:tasks.workunit.client.1.smithi097.stdout:0/56: mkdir da/df 0 2024-09-16T08:57:04.010 INFO:tasks.workunit.client.1.smithi097.stdout:0/57: chown da/fb 316090 1 2024-09-16T08:57:04.010 INFO:tasks.workunit.client.1.smithi097.stdout:0/58: readlink l3 0 2024-09-16T08:57:04.018 INFO:tasks.workunit.client.1.smithi097.stdout:0/59: dread f6 [0,4194304] 0 2024-09-16T08:57:04.021 INFO:tasks.workunit.client.1.smithi097.stdout:0/60: dread f8 [0,4194304] 0 2024-09-16T08:57:04.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/61: creat da/dc/f10 x:0 0 0 2024-09-16T08:57:04.023 INFO:tasks.workunit.client.1.smithi097.stdout:0/62: symlink da/df/l11 0 2024-09-16T08:57:04.028 INFO:tasks.workunit.client.1.smithi097.stdout:0/63: getdents da/df 0 2024-09-16T08:57:04.031 INFO:tasks.workunit.client.1.smithi097.stdout:0/64: fsync da/fb 0 2024-09-16T08:57:04.031 INFO:tasks.workunit.client.1.smithi097.stdout:0/65: chown f1 186490091 1 2024-09-16T08:57:04.031 INFO:tasks.workunit.client.1.smithi097.stdout:0/66: truncate f5 1085715 0 2024-09-16T08:57:04.031 INFO:tasks.workunit.client.1.smithi097.stdout:0/67: readlink l3 0 2024-09-16T08:57:04.033 INFO:tasks.workunit.client.1.smithi097.stdout:0/68: rename da/dc/ce to da/c12 0 2024-09-16T08:57:04.050 INFO:tasks.workunit.client.1.smithi097.stdout:0/69: dwrite f1 [0,4194304] 0 2024-09-16T08:57:04.066 INFO:tasks.workunit.client.1.smithi097.stdout:0/70: dwrite f5 [0,4194304] 0 2024-09-16T08:57:04.092 INFO:tasks.workunit.client.1.smithi097.stdout:0/71: dwrite f5 [4194304,4194304] 0 2024-09-16T08:57:04.092 INFO:tasks.workunit.client.1.smithi097.stdout:0/72: write da/dc/f10 [111673,128029] 0 2024-09-16T08:57:04.093 INFO:tasks.workunit.client.1.smithi097.stdout:0/73: rename da/dc to da/df/d13 0 2024-09-16T08:57:04.114 INFO:tasks.workunit.client.1.smithi097.stdout:2/28: sync 2024-09-16T08:57:04.114 INFO:tasks.workunit.client.1.smithi097.stdout:2/29: truncate d3/d4/f6 963597 0 2024-09-16T08:57:04.115 INFO:tasks.workunit.client.1.smithi097.stdout:2/30: symlink d3/d4/la 0 2024-09-16T08:57:04.118 INFO:tasks.workunit.client.1.smithi097.stdout:2/31: write d3/d4/f6 [1353073,86666] 0 2024-09-16T08:57:04.133 INFO:tasks.workunit.client.1.smithi097.stdout:2/32: dwrite f2 [0,4194304] 0 2024-09-16T08:57:04.133 INFO:tasks.workunit.client.1.smithi097.stdout:2/33: chown f2 15732 1 2024-09-16T08:57:04.142 INFO:tasks.workunit.client.1.smithi097.stdout:2/34: dread f2 [0,4194304] 0 2024-09-16T08:57:04.144 INFO:tasks.workunit.client.1.smithi097.stdout:2/35: fsync d3/d4/f6 0 2024-09-16T08:57:04.145 INFO:tasks.workunit.client.1.smithi097.stdout:2/36: read d3/d4/f6 [810087,2969] 0 2024-09-16T08:57:04.149 INFO:tasks.workunit.client.1.smithi097.stdout:2/37: dread f2 [0,4194304] 0 2024-09-16T08:57:04.149 INFO:tasks.workunit.client.1.smithi097.stdout:2/38: readlink d3/d4/la 0 2024-09-16T08:57:04.150 INFO:tasks.workunit.client.1.smithi097.stdout:2/39: getdents d3 0 2024-09-16T08:57:04.152 INFO:tasks.workunit.client.1.smithi097.stdout:2/40: creat d3/d4/d9/fb x:0 0 0 2024-09-16T08:57:04.152 INFO:tasks.workunit.client.1.smithi097.stdout:2/41: fdatasync d3/d4/f6 0 2024-09-16T08:57:04.152 INFO:tasks.workunit.client.1.smithi097.stdout:2/42: dread - d3/d4/d9/fb zero size 2024-09-16T08:57:04.152 INFO:tasks.workunit.client.1.smithi097.stdout:2/43: write d3/d4/d9/fb [907477,47037] 0 2024-09-16T08:57:04.152 INFO:tasks.workunit.client.1.smithi097.stdout:2/44: chown d3/d4 15463 1 2024-09-16T08:57:04.167 INFO:tasks.workunit.client.1.smithi097.stdout:2/45: dwrite f2 [0,4194304] 0 2024-09-16T08:57:04.167 INFO:tasks.workunit.client.1.smithi097.stdout:2/46: creat d3/d4/fc x:0 0 0 2024-09-16T08:57:04.171 INFO:tasks.workunit.client.1.smithi097.stdout:2/47: dread d3/d4/f6 [0,4194304] 0 2024-09-16T08:57:04.171 INFO:tasks.workunit.client.1.smithi097.stdout:2/48: stat d3/d4/fc 0 2024-09-16T08:57:04.177 INFO:tasks.workunit.client.1.smithi097.stdout:2/49: write f2 [905313,4605] 0 2024-09-16T08:57:04.182 INFO:tasks.workunit.client.1.smithi097.stdout:2/50: dread f2 [0,4194304] 0 2024-09-16T08:57:04.185 INFO:tasks.workunit.client.1.smithi097.stdout:2/51: dread d3/d4/f6 [0,4194304] 0 2024-09-16T08:57:04.186 INFO:tasks.workunit.client.1.smithi097.stdout:2/52: mkdir d3/d4/dd 0 2024-09-16T08:57:04.186 INFO:tasks.workunit.client.1.smithi097.stdout:2/53: symlink d3/d4/le 0 2024-09-16T08:57:04.187 INFO:tasks.workunit.client.1.smithi097.stdout:2/54: truncate d3/d4/fc 1015942 0 2024-09-16T08:57:04.187 INFO:tasks.workunit.client.1.smithi097.stdout:2/55: creat d3/d4/d9/ff x:0 0 0 2024-09-16T08:57:04.187 INFO:tasks.workunit.client.1.smithi097.stdout:2/56: fdatasync d3/d4/fc 0 2024-09-16T08:57:04.193 INFO:tasks.workunit.client.1.smithi097.stdout:6/68: sync 2024-09-16T08:57:04.194 INFO:tasks.workunit.client.1.smithi097.stdout:6/69: fsync d1/f12 0 2024-09-16T08:57:04.200 INFO:tasks.workunit.client.1.smithi097.stdout:2/57: dwrite d3/d4/d9/ff [0,4194304] 0 2024-09-16T08:57:04.200 INFO:tasks.workunit.client.1.smithi097.stdout:2/58: fdatasync d3/d4/d9/fb 0 2024-09-16T08:57:04.202 INFO:tasks.workunit.client.1.smithi097.stdout:2/59: unlink d3/d4/fc 0 2024-09-16T08:57:04.203 INFO:tasks.workunit.client.1.smithi097.stdout:2/60: mknod d3/d4/dd/c10 0 2024-09-16T08:57:04.211 INFO:tasks.workunit.client.1.smithi097.stdout:6/70: dwrite d1/d3/f5 [0,4194304] 0 2024-09-16T08:57:04.212 INFO:tasks.workunit.client.1.smithi097.stdout:2/61: getdents d3/d4 0 2024-09-16T08:57:04.213 INFO:tasks.workunit.client.1.smithi097.stdout:2/62: readlink d3/d4/le 0 2024-09-16T08:57:04.236 INFO:tasks.workunit.client.1.smithi097.stdout:2/63: dwrite d3/d4/d9/ff [4194304,4194304] 0 2024-09-16T08:57:04.236 INFO:tasks.workunit.client.1.smithi097.stdout:6/71: dwrite d1/f12 [0,4194304] 0 2024-09-16T08:57:04.238 INFO:tasks.workunit.client.1.smithi097.stdout:6/72: mknod d1/c15 0 2024-09-16T08:57:04.238 INFO:tasks.workunit.client.1.smithi097.stdout:6/73: stat d1 0 2024-09-16T08:57:04.238 INFO:tasks.workunit.client.1.smithi097.stdout:6/74: fdatasync d1/f12 0 2024-09-16T08:57:04.243 INFO:tasks.workunit.client.1.smithi097.stdout:2/64: truncate d3/d4/d9/ff 7824862 0 2024-09-16T08:57:04.243 INFO:tasks.workunit.client.1.smithi097.stdout:2/65: getdents d3/d4 0 2024-09-16T08:57:04.243 INFO:tasks.workunit.client.1.smithi097.stdout:2/66: write f2 [598327,48772] 0 2024-09-16T08:57:04.245 INFO:tasks.workunit.client.1.smithi097.stdout:2/67: dread d3/d4/d9/fb [0,4194304] 0 2024-09-16T08:57:04.247 INFO:tasks.workunit.client.1.smithi097.stdout:2/68: read d3/d4/d9/fb [376931,63504] 0 2024-09-16T08:57:04.247 INFO:tasks.workunit.client.1.smithi097.stdout:2/69: fsync d3/d4/d9/fb 0 2024-09-16T08:57:04.247 INFO:tasks.workunit.client.1.smithi097.stdout:2/70: rename d3/d4/d9 to d3/d4/d9/d11 22 2024-09-16T08:57:04.247 INFO:tasks.workunit.client.1.smithi097.stdout:6/75: dread d1/f12 [0,4194304] 0 2024-09-16T08:57:04.248 INFO:tasks.workunit.client.1.smithi097.stdout:6/76: mkdir d1/d3/d16 0 2024-09-16T08:57:04.250 INFO:tasks.workunit.client.1.smithi097.stdout:6/77: symlink d1/dc/l17 0 2024-09-16T08:57:04.250 INFO:tasks.workunit.client.1.smithi097.stdout:6/78: readlink d1/l13 0 2024-09-16T08:57:04.250 INFO:tasks.workunit.client.1.smithi097.stdout:8/83: sync 2024-09-16T08:57:04.250 INFO:tasks.workunit.client.1.smithi097.stdout:1/56: sync 2024-09-16T08:57:04.250 INFO:tasks.workunit.client.1.smithi097.stdout:1/57: stat f4 0 2024-09-16T08:57:04.250 INFO:tasks.workunit.client.1.smithi097.stdout:8/84: unlink d8/l14 0 2024-09-16T08:57:04.251 INFO:tasks.workunit.client.1.smithi097.stdout:1/58: mknod d5/db/d11/c13 0 2024-09-16T08:57:04.252 INFO:tasks.workunit.client.1.smithi097.stdout:8/85: rename l6 to d8/dd/l1a 0 2024-09-16T08:57:04.253 INFO:tasks.workunit.client.1.smithi097.stdout:8/86: truncate f7 1009384 0 2024-09-16T08:57:04.256 INFO:tasks.workunit.client.1.smithi097.stdout:8/87: mknod d8/dd/d15/d16/c1b 0 2024-09-16T08:57:04.257 INFO:tasks.workunit.client.1.smithi097.stdout:8/88: write d8/fc [420818,72151] 0 2024-09-16T08:57:04.257 INFO:tasks.workunit.client.1.smithi097.stdout:8/89: creat d8/dd/f1c x:0 0 0 2024-09-16T08:57:04.258 INFO:tasks.workunit.client.1.smithi097.stdout:8/90: rename l5 to d8/dd/d15/l1d 0 2024-09-16T08:57:04.258 INFO:tasks.workunit.client.1.smithi097.stdout:2/71: write d3/d4/d9/ff [7129727,73564] 0 2024-09-16T08:57:04.258 INFO:tasks.workunit.client.1.smithi097.stdout:8/91: creat d8/f1e x:0 0 0 2024-09-16T08:57:04.258 INFO:tasks.workunit.client.1.smithi097.stdout:2/72: creat d3/d4/d9/f12 x:0 0 0 2024-09-16T08:57:04.258 INFO:tasks.workunit.client.1.smithi097.stdout:8/92: write d8/f10 [344386,120104] 0 2024-09-16T08:57:04.262 INFO:tasks.workunit.client.1.smithi097.stdout:8/93: creat d8/dd/f1f x:0 0 0 2024-09-16T08:57:04.267 INFO:tasks.workunit.client.1.smithi097.stdout:8/94: getdents d8/dd/d15/d16 0 2024-09-16T08:57:04.273 INFO:tasks.workunit.client.1.smithi097.stdout:8/95: dread f7 [0,4194304] 0 2024-09-16T08:57:04.273 INFO:tasks.workunit.client.1.smithi097.stdout:8/96: symlink d8/dd/d15/l20 0 2024-09-16T08:57:04.275 INFO:tasks.workunit.client.1.smithi097.stdout:2/73: dwrite d3/d4/d9/ff [0,4194304] 0 2024-09-16T08:57:04.280 INFO:tasks.workunit.client.1.smithi097.stdout:2/74: dread f2 [0,4194304] 0 2024-09-16T08:57:04.302 INFO:tasks.workunit.client.1.smithi097.stdout:8/97: dwrite d8/dd/f1f [0,4194304] 0 2024-09-16T08:57:04.302 INFO:tasks.workunit.client.1.smithi097.stdout:8/98: write f7 [1750367,129776] 0 2024-09-16T08:57:04.304 INFO:tasks.workunit.client.1.smithi097.stdout:2/75: dwrite d3/d4/d9/fb [0,4194304] 0 2024-09-16T08:57:04.305 INFO:tasks.workunit.client.1.smithi097.stdout:8/99: creat d8/dd/d15/f21 x:0 0 0 2024-09-16T08:57:04.307 INFO:tasks.workunit.client.1.smithi097.stdout:2/76: mknod d3/c13 0 2024-09-16T08:57:04.310 INFO:tasks.workunit.client.1.smithi097.stdout:8/100: rename d8/dd/f1c to d8/dd/d15/f22 0 2024-09-16T08:57:04.310 INFO:tasks.workunit.client.1.smithi097.stdout:8/101: creat d8/f23 x:0 0 0 2024-09-16T08:57:04.310 INFO:tasks.workunit.client.1.smithi097.stdout:8/102: readlink d8/dd/d15/l1d 0 2024-09-16T08:57:04.310 INFO:tasks.workunit.client.1.smithi097.stdout:8/103: write d8/fc [2235290,105956] 0 2024-09-16T08:57:04.310 INFO:tasks.workunit.client.1.smithi097.stdout:8/104: creat d8/f24 x:0 0 0 2024-09-16T08:57:04.310 INFO:tasks.workunit.client.1.smithi097.stdout:8/105: stat d8/dd/f19 0 2024-09-16T08:57:04.311 INFO:tasks.workunit.client.1.smithi097.stdout:8/106: chown d8/f24 0 1 2024-09-16T08:57:04.312 INFO:tasks.workunit.client.1.smithi097.stdout:8/107: symlink d8/dd/d15/d16/l25 0 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:9/63: sync 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:7/89: sync 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:0/74: sync 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:9/64: chown d2/d5 7937 1 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:7/90: chown d4/c18 1460099 1 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:7/91: chown d4/d9/c13 84811 1 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:0/75: truncate da/df/d13/f10 796607 0 2024-09-16T08:57:04.316 INFO:tasks.workunit.client.1.smithi097.stdout:4/102: sync 2024-09-16T08:57:04.317 INFO:tasks.workunit.client.1.smithi097.stdout:5/67: sync 2024-09-16T08:57:04.317 INFO:tasks.workunit.client.1.smithi097.stdout:3/51: sync 2024-09-16T08:57:04.317 INFO:tasks.workunit.client.1.smithi097.stdout:5/68: chown db 49413610 1 2024-09-16T08:57:04.319 INFO:tasks.workunit.client.1.smithi097.stdout:4/103: mkdir d0/d7/df/d15/d1f 0 2024-09-16T08:57:04.319 INFO:tasks.workunit.client.1.smithi097.stdout:9/65: creat d2/d5/f11 x:0 0 0 2024-09-16T08:57:04.321 INFO:tasks.workunit.client.1.smithi097.stdout:5/69: unlink f4 0 2024-09-16T08:57:04.321 INFO:tasks.workunit.client.1.smithi097.stdout:3/52: unlink d7/lb 0 2024-09-16T08:57:04.322 INFO:tasks.workunit.client.1.smithi097.stdout:4/104: mknod d0/d7/c20 0 2024-09-16T08:57:04.323 INFO:tasks.workunit.client.1.smithi097.stdout:9/66: symlink d2/d5/l12 0 2024-09-16T08:57:04.323 INFO:tasks.workunit.client.1.smithi097.stdout:9/67: fdatasync d2/fc 0 2024-09-16T08:57:04.325 INFO:tasks.workunit.client.1.smithi097.stdout:3/53: write d7/fa [308202,72385] 0 2024-09-16T08:57:04.328 INFO:tasks.workunit.client.1.smithi097.stdout:3/54: mkdir d7/de 0 2024-09-16T08:57:04.330 INFO:tasks.workunit.client.1.smithi097.stdout:3/55: creat d7/de/ff x:0 0 0 2024-09-16T08:57:04.332 INFO:tasks.workunit.client.1.smithi097.stdout:3/56: symlink d7/de/l10 0 2024-09-16T08:57:04.336 INFO:tasks.workunit.client.1.smithi097.stdout:2/77: dwrite d3/d4/d9/ff [4194304,4194304] 0 2024-09-16T08:57:04.336 INFO:tasks.workunit.client.1.smithi097.stdout:0/76: write f7 [2106920,85742] 0 2024-09-16T08:57:04.336 INFO:tasks.workunit.client.1.smithi097.stdout:0/77: truncate da/df/d13/f10 1492008 0 2024-09-16T08:57:04.339 INFO:tasks.workunit.client.1.smithi097.stdout:2/78: creat d3/d4/dd/f14 x:0 0 0 2024-09-16T08:57:04.342 INFO:tasks.workunit.client.1.smithi097.stdout:2/79: unlink d3/d4/d9/fb 0 2024-09-16T08:57:04.342 INFO:tasks.workunit.client.1.smithi097.stdout:2/80: stat d3/d4/dd/c10 0 2024-09-16T08:57:04.343 INFO:tasks.workunit.client.1.smithi097.stdout:2/81: stat d3/d4/le 0 2024-09-16T08:57:04.343 INFO:tasks.workunit.client.1.smithi097.stdout:2/82: write f2 [4403478,107133] 0 2024-09-16T08:57:04.343 INFO:tasks.workunit.client.1.smithi097.stdout:0/78: rmdir da 39 2024-09-16T08:57:04.343 INFO:tasks.workunit.client.1.smithi097.stdout:0/79: rename da/df/d13 to da/df/d13/d14 22 2024-09-16T08:57:04.344 INFO:tasks.workunit.client.1.smithi097.stdout:2/83: symlink d3/d4/d9/l15 0 2024-09-16T08:57:04.344 INFO:tasks.workunit.client.1.smithi097.stdout:2/84: write d3/d4/d9/f12 [684294,53578] 0 2024-09-16T08:57:04.345 INFO:tasks.workunit.client.1.smithi097.stdout:0/80: rename da/fb to da/f15 0 2024-09-16T08:57:04.348 INFO:tasks.workunit.client.1.smithi097.stdout:2/85: rename f2 to d3/d4/d8/f16 0 2024-09-16T08:57:04.348 INFO:tasks.workunit.client.1.smithi097.stdout:2/86: write d3/d4/f6 [2195454,65301] 0 2024-09-16T08:57:04.348 INFO:tasks.workunit.client.1.smithi097.stdout:2/87: fdatasync d3/d4/dd/f14 0 2024-09-16T08:57:04.348 INFO:tasks.workunit.client.1.smithi097.stdout:8/108: dwrite f7 [0,4194304] 0 2024-09-16T08:57:04.350 INFO:tasks.workunit.client.1.smithi097.stdout:0/81: rename da/df/d13/f10 to da/df/f16 0 2024-09-16T08:57:04.355 INFO:tasks.workunit.client.1.smithi097.stdout:9/68: dwrite d2/fc [0,4194304] 0 2024-09-16T08:57:04.356 INFO:tasks.workunit.client.1.smithi097.stdout:7/92: dwrite d4/d7/f11 [0,4194304] 0 2024-09-16T08:57:04.356 INFO:tasks.workunit.client.1.smithi097.stdout:7/93: chown d4/d12 30442676 1 2024-09-16T08:57:04.358 INFO:tasks.workunit.client.1.smithi097.stdout:4/105: dwrite d0/d7/df/d15/d16/f1d [0,4194304] 0 2024-09-16T08:57:04.360 INFO:tasks.workunit.client.1.smithi097.stdout:2/88: write d3/d4/d9/ff [6521368,52202] 0 2024-09-16T08:57:04.364 INFO:tasks.workunit.client.1.smithi097.stdout:4/106: rename d0/d7/c20 to d0/d7/df/d15/d1f/c21 0 2024-09-16T08:57:04.364 INFO:tasks.workunit.client.1.smithi097.stdout:4/107: truncate d0/d7/df/d15/f11 1544721 0 2024-09-16T08:57:04.366 INFO:tasks.workunit.client.1.smithi097.stdout:2/89: symlink d3/d4/d9/l17 0 2024-09-16T08:57:04.367 INFO:tasks.workunit.client.1.smithi097.stdout:9/69: dread d2/fe [0,4194304] 0 2024-09-16T08:57:04.367 INFO:tasks.workunit.client.1.smithi097.stdout:9/70: creat d2/f13 x:0 0 0 2024-09-16T08:57:04.367 INFO:tasks.workunit.client.1.smithi097.stdout:4/108: link d0/d7/lc d0/d7/df/l22 0 2024-09-16T08:57:04.370 INFO:tasks.workunit.client.1.smithi097.stdout:4/109: mkdir d0/d23 0 2024-09-16T08:57:04.370 INFO:tasks.workunit.client.1.smithi097.stdout:4/110: stat d0/fd 0 2024-09-16T08:57:04.371 INFO:tasks.workunit.client.1.smithi097.stdout:4/111: creat d0/d7/f24 x:0 0 0 2024-09-16T08:57:04.371 INFO:tasks.workunit.client.1.smithi097.stdout:4/112: fsync d0/fd 0 2024-09-16T08:57:04.372 INFO:tasks.workunit.client.1.smithi097.stdout:4/113: mknod d0/d7/df/d15/d16/c25 0 2024-09-16T08:57:04.372 INFO:tasks.workunit.client.1.smithi097.stdout:4/114: dread - d0/fd zero size 2024-09-16T08:57:04.372 INFO:tasks.workunit.client.1.smithi097.stdout:4/115: write d0/d7/df/d15/f11 [2514485,94834] 0 2024-09-16T08:57:04.373 INFO:tasks.workunit.client.1.smithi097.stdout:4/116: fdatasync d0/fe 0 2024-09-16T08:57:04.373 INFO:tasks.workunit.client.1.smithi097.stdout:9/71: write d2/fe [1736652,6838] 0 2024-09-16T08:57:04.373 INFO:tasks.workunit.client.1.smithi097.stdout:9/72: write d2/f13 [620562,104075] 0 2024-09-16T08:57:04.375 INFO:tasks.workunit.client.1.smithi097.stdout:9/73: rename d2/l9 to d2/l14 0 2024-09-16T08:57:04.375 INFO:tasks.workunit.client.1.smithi097.stdout:9/74: chown d2 0 1 2024-09-16T08:57:04.377 INFO:tasks.workunit.client.1.smithi097.stdout:9/75: symlink d2/l15 0 2024-09-16T08:57:04.377 INFO:tasks.workunit.client.1.smithi097.stdout:9/76: creat d2/d5/f16 x:0 0 0 2024-09-16T08:57:04.378 INFO:tasks.workunit.client.1.smithi097.stdout:3/57: dwrite d7/fa [0,4194304] 0 2024-09-16T08:57:04.378 INFO:tasks.workunit.client.1.smithi097.stdout:3/58: readlink d7/de/l10 0 2024-09-16T08:57:04.379 INFO:tasks.workunit.client.1.smithi097.stdout:3/59: dread - d7/de/ff zero size 2024-09-16T08:57:04.379 INFO:tasks.workunit.client.1.smithi097.stdout:9/77: mknod d2/d5/c17 0 2024-09-16T08:57:04.379 INFO:tasks.workunit.client.1.smithi097.stdout:9/78: creat d2/d5/f18 x:0 0 0 2024-09-16T08:57:04.379 INFO:tasks.workunit.client.1.smithi097.stdout:9/79: chown d2/d5/lf 415954820 1 2024-09-16T08:57:04.380 INFO:tasks.workunit.client.1.smithi097.stdout:2/90: fdatasync d3/d4/d9/f12 0 2024-09-16T08:57:04.380 INFO:tasks.workunit.client.1.smithi097.stdout:0/82: dwrite f1 [0,4194304] 0 2024-09-16T08:57:04.380 INFO:tasks.workunit.client.1.smithi097.stdout:8/109: dwrite d8/dd/f19 [0,4194304] 0 2024-09-16T08:57:04.380 INFO:tasks.workunit.client.1.smithi097.stdout:8/110: chown d8/f23 0 1 2024-09-16T08:57:04.380 INFO:tasks.workunit.client.1.smithi097.stdout:8/111: write d8/f1e [461218,50504] 0 2024-09-16T08:57:04.384 INFO:tasks.workunit.client.1.smithi097.stdout:0/83: mknod da/df/d13/c17 0 2024-09-16T08:57:04.388 INFO:tasks.workunit.client.1.smithi097.stdout:2/91: link d3/d4/d8/f16 d3/d4/dd/f18 0 2024-09-16T08:57:04.388 INFO:tasks.workunit.client.1.smithi097.stdout:2/92: write d3/d4/dd/f14 [695570,121167] 0 2024-09-16T08:57:04.388 INFO:tasks.workunit.client.1.smithi097.stdout:2/93: rename d3 to d3/d4/d8/d19 22 2024-09-16T08:57:04.388 INFO:tasks.workunit.client.1.smithi097.stdout:2/94: chown d3/d4/d8/f16 13929309 1 2024-09-16T08:57:04.389 INFO:tasks.workunit.client.1.smithi097.stdout:2/95: stat d3/d4/d9/ff 0 2024-09-16T08:57:04.389 INFO:tasks.workunit.client.1.smithi097.stdout:2/96: chown d3/d4/la 1674 1 2024-09-16T08:57:04.390 INFO:tasks.workunit.client.1.smithi097.stdout:9/80: write d2/fc [4925445,64149] 0 2024-09-16T08:57:04.391 INFO:tasks.workunit.client.1.smithi097.stdout:3/60: dread d7/fa [0,4194304] 0 2024-09-16T08:57:04.393 INFO:tasks.workunit.client.1.smithi097.stdout:7/94: truncate d4/d7/f11 1316432 0 2024-09-16T08:57:04.395 INFO:tasks.workunit.client.1.smithi097.stdout:9/81: mknod d2/d5/c19 0 2024-09-16T08:57:04.401 INFO:tasks.workunit.client.1.smithi097.stdout:7/95: link d4/f17 d4/d12/d1f/f21 0 2024-09-16T08:57:04.401 INFO:tasks.workunit.client.1.smithi097.stdout:4/117: rmdir d0/d7/df/d15/d16 39 2024-09-16T08:57:04.404 INFO:tasks.workunit.client.1.smithi097.stdout:4/118: mkdir d0/d7/df/d15/d16/d26 0 2024-09-16T08:57:04.404 INFO:tasks.workunit.client.1.smithi097.stdout:4/119: fsync d0/d7/fb 0 2024-09-16T08:57:04.409 INFO:tasks.workunit.client.1.smithi097.stdout:4/120: rename d0/d7/df/d15/f12 to d0/d7/df/f27 0 2024-09-16T08:57:04.409 INFO:tasks.workunit.client.1.smithi097.stdout:4/121: truncate d0/d7/df/d15/f1b 94769 0 2024-09-16T08:57:04.410 INFO:tasks.workunit.client.1.smithi097.stdout:8/112: dwrite d8/fa [0,4194304] 0 2024-09-16T08:57:04.412 INFO:tasks.workunit.client.1.smithi097.stdout:8/113: write d8/fc [1417298,83381] 0 2024-09-16T08:57:04.412 INFO:tasks.workunit.client.1.smithi097.stdout:8/114: chown d8/fc 0 1 2024-09-16T08:57:04.412 INFO:tasks.workunit.client.1.smithi097.stdout:8/115: chown d8/dd/d15/d16 5085832 1 2024-09-16T08:57:04.412 INFO:tasks.workunit.client.1.smithi097.stdout:4/122: mknod d0/d23/c28 0 2024-09-16T08:57:04.414 INFO:tasks.workunit.client.1.smithi097.stdout:1/59: sync 2024-09-16T08:57:04.414 INFO:tasks.workunit.client.1.smithi097.stdout:6/79: sync 2024-09-16T08:57:04.414 INFO:tasks.workunit.client.1.smithi097.stdout:1/60: write d5/db/dc/f10 [1023286,20368] 0 2024-09-16T08:57:04.421 INFO:tasks.workunit.client.1.smithi097.stdout:8/116: rmdir d8/dd/d15/d16 39 2024-09-16T08:57:04.421 INFO:tasks.workunit.client.1.smithi097.stdout:8/117: truncate d8/dd/fe 1289391 0 2024-09-16T08:57:04.422 INFO:tasks.workunit.client.1.smithi097.stdout:6/80: truncate d1/d3/f5 3187209 0 2024-09-16T08:57:04.423 INFO:tasks.workunit.client.1.smithi097.stdout:6/81: mknod d1/d3/df/c18 0 2024-09-16T08:57:04.425 INFO:tasks.workunit.client.1.smithi097.stdout:6/82: truncate d1/f12 4028452 0 2024-09-16T08:57:04.426 INFO:tasks.workunit.client.1.smithi097.stdout:3/61: dwrite d7/fa [0,4194304] 0 2024-09-16T08:57:04.429 INFO:tasks.workunit.client.1.smithi097.stdout:0/84: dwrite f7 [0,4194304] 0 2024-09-16T08:57:04.431 INFO:tasks.workunit.client.1.smithi097.stdout:7/96: dwrite d4/d7/ff [4194304,4194304] 0 2024-09-16T08:57:04.438 INFO:tasks.workunit.client.1.smithi097.stdout:2/97: dwrite d3/d4/f6 [0,4194304] 0 2024-09-16T08:57:04.440 INFO:tasks.workunit.client.1.smithi097.stdout:2/98: unlink d3/d4/dd/f14 0 2024-09-16T08:57:04.440 INFO:tasks.workunit.client.1.smithi097.stdout:2/99: chown d3/d4/le 3383 1 2024-09-16T08:57:04.440 INFO:tasks.workunit.client.1.smithi097.stdout:2/100: chown d3/d4/d9/l15 2563591 1 2024-09-16T08:57:04.442 INFO:tasks.workunit.client.1.smithi097.stdout:3/62: dread d7/fa [0,4194304] 0 2024-09-16T08:57:04.442 INFO:tasks.workunit.client.1.smithi097.stdout:4/123: dwrite d0/d7/df/f13 [0,4194304] 0 2024-09-16T08:57:04.442 INFO:tasks.workunit.client.1.smithi097.stdout:8/118: dwrite d8/dd/d15/f21 [0,4194304] 0 2024-09-16T08:57:04.444 INFO:tasks.workunit.client.1.smithi097.stdout:4/124: mkdir d0/d7/df/d15/d16/d29 0 2024-09-16T08:57:04.445 INFO:tasks.workunit.client.1.smithi097.stdout:3/63: symlink d7/l11 0 2024-09-16T08:57:04.446 INFO:tasks.workunit.client.1.smithi097.stdout:4/125: creat d0/f2a x:0 0 0 2024-09-16T08:57:04.447 INFO:tasks.workunit.client.1.smithi097.stdout:7/97: getdents d4/d12/d1f 0 2024-09-16T08:57:04.447 INFO:tasks.workunit.client.1.smithi097.stdout:7/98: read - d4/f17 zero size 2024-09-16T08:57:04.449 INFO:tasks.workunit.client.1.smithi097.stdout:4/126: creat d0/d7/df/d15/d16/d29/f2b x:0 0 0 2024-09-16T08:57:04.450 INFO:tasks.workunit.client.1.smithi097.stdout:7/99: rmdir d4/d9 39 2024-09-16T08:57:04.454 INFO:tasks.workunit.client.1.smithi097.stdout:7/100: mkdir d4/d12/d1f/d22 0 2024-09-16T08:57:04.454 INFO:tasks.workunit.client.1.smithi097.stdout:7/101: write d4/d1b/f1e [924534,46432] 0 2024-09-16T08:57:04.458 INFO:tasks.workunit.client.1.smithi097.stdout:7/102: creat d4/f23 x:0 0 0 2024-09-16T08:57:04.461 INFO:tasks.workunit.client.1.smithi097.stdout:7/103: rename d4/d9/f15 to d4/d9/f24 0 2024-09-16T08:57:04.462 INFO:tasks.workunit.client.1.smithi097.stdout:0/85: dwrite f7 [0,4194304] 0 2024-09-16T08:57:04.462 INFO:tasks.workunit.client.1.smithi097.stdout:0/86: truncate da/df/f16 1898060 0 2024-09-16T08:57:04.463 INFO:tasks.workunit.client.1.smithi097.stdout:7/104: mkdir d4/d9/d25 0 2024-09-16T08:57:04.463 INFO:tasks.workunit.client.1.smithi097.stdout:7/105: readlink d4/d9/dc/l10 0 2024-09-16T08:57:04.464 INFO:tasks.workunit.client.1.smithi097.stdout:0/87: mknod da/df/d13/c18 0 2024-09-16T08:57:04.464 INFO:tasks.workunit.client.1.smithi097.stdout:0/88: chown da/c12 46463086 1 2024-09-16T08:57:04.465 INFO:tasks.workunit.client.1.smithi097.stdout:0/89: creat da/df/d13/f19 x:0 0 0 2024-09-16T08:57:04.466 INFO:tasks.workunit.client.1.smithi097.stdout:0/90: mkdir da/d1a 0 2024-09-16T08:57:04.467 INFO:tasks.workunit.client.1.smithi097.stdout:5/70: sync 2024-09-16T08:57:04.468 INFO:tasks.workunit.client.1.smithi097.stdout:0/91: creat da/f1b x:0 0 0 2024-09-16T08:57:04.470 INFO:tasks.workunit.client.1.smithi097.stdout:0/92: unlink da/df/d13/f19 0 2024-09-16T08:57:04.471 INFO:tasks.workunit.client.1.smithi097.stdout:4/127: read d0/d7/df/f27 [2013715,42804] 0 2024-09-16T08:57:04.472 INFO:tasks.workunit.client.1.smithi097.stdout:6/83: unlink d1/d3/f5 0 2024-09-16T08:57:04.472 INFO:tasks.workunit.client.1.smithi097.stdout:6/84: fdatasync d1/d3/f11 0 2024-09-16T08:57:04.473 INFO:tasks.workunit.client.1.smithi097.stdout:4/128: truncate d0/fe 1241493 0 2024-09-16T08:57:04.473 INFO:tasks.workunit.client.1.smithi097.stdout:4/129: chown d0/f4 54 1 2024-09-16T08:57:04.473 INFO:tasks.workunit.client.1.smithi097.stdout:4/130: stat d0/d7/df/d15/l17 0 2024-09-16T08:57:04.474 INFO:tasks.workunit.client.1.smithi097.stdout:7/106: dread d4/fb [0,4194304] 0 2024-09-16T08:57:04.474 INFO:tasks.workunit.client.1.smithi097.stdout:7/107: chown d4/d9/dc 0 1 2024-09-16T08:57:04.474 INFO:tasks.workunit.client.1.smithi097.stdout:7/108: chown f2 3163542 1 2024-09-16T08:57:04.475 INFO:tasks.workunit.client.1.smithi097.stdout:6/85: rename d1/d3/c6 to d1/d3/d16/c19 0 2024-09-16T08:57:04.476 INFO:tasks.workunit.client.1.smithi097.stdout:6/86: truncate d1/d3/f11 582725 0 2024-09-16T08:57:04.476 INFO:tasks.workunit.client.1.smithi097.stdout:6/87: chown d1/dc/cd 0 1 2024-09-16T08:57:04.476 INFO:tasks.workunit.client.1.smithi097.stdout:6/88: fdatasync d1/f12 0 2024-09-16T08:57:04.477 INFO:tasks.workunit.client.1.smithi097.stdout:7/109: rename l0 to d4/d1b/l26 0 2024-09-16T08:57:04.478 INFO:tasks.workunit.client.1.smithi097.stdout:6/89: creat d1/d3/d16/f1a x:0 0 0 2024-09-16T08:57:04.478 INFO:tasks.workunit.client.1.smithi097.stdout:6/90: readlink d1/d3/l8 0 2024-09-16T08:57:04.478 INFO:tasks.workunit.client.1.smithi097.stdout:6/91: stat d1/dc/l17 0 2024-09-16T08:57:04.478 INFO:tasks.workunit.client.1.smithi097.stdout:6/92: chown d1/d3/d16 0 1 2024-09-16T08:57:04.479 INFO:tasks.workunit.client.1.smithi097.stdout:7/110: stat d4/c6 0 2024-09-16T08:57:04.483 INFO:tasks.workunit.client.1.smithi097.stdout:7/111: rename d4/d9/d25 to d4/d27 0 2024-09-16T08:57:04.483 INFO:tasks.workunit.client.1.smithi097.stdout:7/112: dread - d4/d12/d1f/f21 zero size 2024-09-16T08:57:04.483 INFO:tasks.workunit.client.1.smithi097.stdout:7/113: stat d4/d7/ff 0 2024-09-16T08:57:04.483 INFO:tasks.workunit.client.1.smithi097.stdout:5/71: dread fa [0,4194304] 0 2024-09-16T08:57:04.483 INFO:tasks.workunit.client.1.smithi097.stdout:2/101: dwrite d3/d4/f6 [0,4194304] 0 2024-09-16T08:57:04.485 INFO:tasks.workunit.client.1.smithi097.stdout:8/119: dwrite d8/f1e [0,4194304] 0 2024-09-16T08:57:04.485 INFO:tasks.workunit.client.1.smithi097.stdout:1/61: dwrite d5/db/dc/f10 [0,4194304] 0 2024-09-16T08:57:04.486 INFO:tasks.workunit.client.1.smithi097.stdout:1/62: write d5/db/ff [1029117,63981] 0 2024-09-16T08:57:04.486 INFO:tasks.workunit.client.1.smithi097.stdout:3/64: dwrite d7/fa [0,4194304] 0 2024-09-16T08:57:04.486 INFO:tasks.workunit.client.1.smithi097.stdout:7/114: unlink d4/fe 0 2024-09-16T08:57:04.487 INFO:tasks.workunit.client.1.smithi097.stdout:2/102: symlink d3/d4/d9/l1a 0 2024-09-16T08:57:04.491 INFO:tasks.workunit.client.1.smithi097.stdout:4/131: dwrite d0/d7/f18 [0,4194304] 0 2024-09-16T08:57:04.491 INFO:tasks.workunit.client.1.smithi097.stdout:1/63: mkdir d5/d14 0 2024-09-16T08:57:04.491 INFO:tasks.workunit.client.1.smithi097.stdout:1/64: truncate d5/f9 382104 0 2024-09-16T08:57:04.491 INFO:tasks.workunit.client.1.smithi097.stdout:4/132: creat d0/d7/df/f2c x:0 0 0 2024-09-16T08:57:04.498 INFO:tasks.workunit.client.1.smithi097.stdout:7/115: creat d4/d9/d1c/f28 x:0 0 0 2024-09-16T08:57:04.498 INFO:tasks.workunit.client.1.smithi097.stdout:7/116: fsync d4/f23 0 2024-09-16T08:57:04.500 INFO:tasks.workunit.client.1.smithi097.stdout:2/103: mkdir d3/d1b 0 2024-09-16T08:57:04.500 INFO:tasks.workunit.client.1.smithi097.stdout:2/104: truncate d3/d4/d9/f12 1500369 0 2024-09-16T08:57:04.505 INFO:tasks.workunit.client.1.smithi097.stdout:4/133: mknod d0/d23/c2d 0 2024-09-16T08:57:04.507 INFO:tasks.workunit.client.1.smithi097.stdout:3/65: symlink d7/l12 0 2024-09-16T08:57:04.508 INFO:tasks.workunit.client.1.smithi097.stdout:8/120: dwrite d8/fa [0,4194304] 0 2024-09-16T08:57:04.508 INFO:tasks.workunit.client.1.smithi097.stdout:8/121: fsync d8/dd/f1f 0 2024-09-16T08:57:04.509 INFO:tasks.workunit.client.1.smithi097.stdout:1/65: getdents d5/db 0 2024-09-16T08:57:04.509 INFO:tasks.workunit.client.1.smithi097.stdout:1/66: chown f4 57189475 1 2024-09-16T08:57:04.511 INFO:tasks.workunit.client.1.smithi097.stdout:4/134: mknod d0/d7/df/c2e 0 2024-09-16T08:57:04.511 INFO:tasks.workunit.client.1.smithi097.stdout:4/135: stat d0/d7/df/d15/d16/f1d 0 2024-09-16T08:57:04.511 INFO:tasks.workunit.client.1.smithi097.stdout:4/136: stat d0/d7/df/d15/f1b 0 2024-09-16T08:57:04.511 INFO:tasks.workunit.client.1.smithi097.stdout:4/137: chown d0/l6 2035030 1 2024-09-16T08:57:04.511 INFO:tasks.workunit.client.1.smithi097.stdout:4/138: chown d0/d7/df/c2e 196892041 1 2024-09-16T08:57:04.513 INFO:tasks.workunit.client.1.smithi097.stdout:8/122: creat d8/dd/f26 x:0 0 0 2024-09-16T08:57:04.513 INFO:tasks.workunit.client.1.smithi097.stdout:8/123: fdatasync d8/dd/d15/f21 0 2024-09-16T08:57:04.514 INFO:tasks.workunit.client.1.smithi097.stdout:1/67: creat d5/db/d11/f15 x:0 0 0 2024-09-16T08:57:04.515 INFO:tasks.workunit.client.1.smithi097.stdout:3/66: rmdir d7/de 39 2024-09-16T08:57:04.517 INFO:tasks.workunit.client.1.smithi097.stdout:8/124: mknod d8/dd/c27 0 2024-09-16T08:57:04.517 INFO:tasks.workunit.client.1.smithi097.stdout:8/125: creat d8/dd/d15/f28 x:0 0 0 2024-09-16T08:57:04.517 INFO:tasks.workunit.client.1.smithi097.stdout:8/126: write d8/dd/fe [529122,89587] 0 2024-09-16T08:57:04.522 INFO:tasks.workunit.client.1.smithi097.stdout:4/139: write d0/d7/df/d15/d16/f1d [3483046,65746] 0 2024-09-16T08:57:04.522 INFO:tasks.workunit.client.1.smithi097.stdout:4/140: write d0/f2a [31494,113080] 0 2024-09-16T08:57:04.522 INFO:tasks.workunit.client.1.smithi097.stdout:4/141: creat d0/f2f x:0 0 0 2024-09-16T08:57:04.522 INFO:tasks.workunit.client.1.smithi097.stdout:4/142: write d0/f2a [545326,35879] 0 2024-09-16T08:57:04.522 INFO:tasks.workunit.client.1.smithi097.stdout:4/143: chown d0/d23/c28 10970 1 2024-09-16T08:57:04.522 INFO:tasks.workunit.client.1.smithi097.stdout:4/144: stat d0/f2f 0 2024-09-16T08:57:04.522 INFO:tasks.workunit.client.1.smithi097.stdout:3/67: symlink d7/l13 0 2024-09-16T08:57:04.523 INFO:tasks.workunit.client.1.smithi097.stdout:2/105: write d3/d4/f6 [439774,67471] 0 2024-09-16T08:57:04.523 INFO:tasks.workunit.client.1.smithi097.stdout:2/106: readlink d3/d4/d9/l15 0 2024-09-16T08:57:04.523 INFO:tasks.workunit.client.1.smithi097.stdout:7/117: dwrite d4/d9/dc/f16 [0,4194304] 0 2024-09-16T08:57:04.523 INFO:tasks.workunit.client.1.smithi097.stdout:8/127: rename d8/dd/fe to d8/dd/d15/f29 0 2024-09-16T08:57:04.525 INFO:tasks.workunit.client.1.smithi097.stdout:4/145: rmdir d0/d7/df/d15/d1f 39 2024-09-16T08:57:04.526 INFO:tasks.workunit.client.1.smithi097.stdout:9/82: sync 2024-09-16T08:57:04.527 INFO:tasks.workunit.client.1.smithi097.stdout:1/68: dread f4 [0,4194304] 0 2024-09-16T08:57:04.528 INFO:tasks.workunit.client.1.smithi097.stdout:8/128: symlink d8/l2a 0 2024-09-16T08:57:04.530 INFO:tasks.workunit.client.1.smithi097.stdout:3/68: write d7/fa [2583543,70299] 0 2024-09-16T08:57:04.531 INFO:tasks.workunit.client.1.smithi097.stdout:3/69: fdatasync d7/de/ff 0 2024-09-16T08:57:04.531 INFO:tasks.workunit.client.1.smithi097.stdout:4/146: unlink d0/l1a 0 2024-09-16T08:57:04.535 INFO:tasks.workunit.client.1.smithi097.stdout:4/147: link d0/d7/df/f2c d0/f30 0 2024-09-16T08:57:04.535 INFO:tasks.workunit.client.1.smithi097.stdout:2/107: dread d3/d4/dd/f18 [0,4194304] 0 2024-09-16T08:57:04.536 INFO:tasks.workunit.client.1.smithi097.stdout:2/108: fsync d3/d4/f6 0 2024-09-16T08:57:04.538 INFO:tasks.workunit.client.1.smithi097.stdout:4/148: truncate d0/d7/df/f27 803903 0 2024-09-16T08:57:04.540 INFO:tasks.workunit.client.1.smithi097.stdout:4/149: mknod d0/d7/df/d15/d16/d26/c31 0 2024-09-16T08:57:04.548 INFO:tasks.workunit.client.1.smithi097.stdout:2/109: dread d3/d4/d9/ff [0,4194304] 0 2024-09-16T08:57:04.548 INFO:tasks.workunit.client.1.smithi097.stdout:2/110: chown d3/d4/f6 1323 1 2024-09-16T08:57:04.548 INFO:tasks.workunit.client.1.smithi097.stdout:2/111: readlink d3/d4/l5 0 2024-09-16T08:57:04.548 INFO:tasks.workunit.client.1.smithi097.stdout:2/112: chown d3/d4/dd/f18 43 1 2024-09-16T08:57:04.548 INFO:tasks.workunit.client.1.smithi097.stdout:2/113: chown d3/d4/d9/f12 208 1 2024-09-16T08:57:04.549 INFO:tasks.workunit.client.1.smithi097.stdout:2/114: write d3/d4/f6 [561150,83050] 0 2024-09-16T08:57:04.555 INFO:tasks.workunit.client.1.smithi097.stdout:2/115: dread d3/d4/d9/ff [4194304,4194304] 0 2024-09-16T08:57:04.555 INFO:tasks.workunit.client.1.smithi097.stdout:2/116: chown d3/d4/l5 45 1 2024-09-16T08:57:04.556 INFO:tasks.workunit.client.1.smithi097.stdout:2/117: symlink d3/d4/l1c 0 2024-09-16T08:57:04.558 INFO:tasks.workunit.client.1.smithi097.stdout:4/150: write d0/d7/f18 [1461647,44975] 0 2024-09-16T08:57:04.558 INFO:tasks.workunit.client.1.smithi097.stdout:4/151: creat d0/d7/f32 x:0 0 0 2024-09-16T08:57:04.558 INFO:tasks.workunit.client.1.smithi097.stdout:4/152: write d0/d7/f24 [787902,108768] 0 2024-09-16T08:57:04.559 INFO:tasks.workunit.client.1.smithi097.stdout:7/118: dwrite d4/d7/ff [0,4194304] 0 2024-09-16T08:57:04.560 INFO:tasks.workunit.client.1.smithi097.stdout:7/119: creat d4/f29 x:0 0 0 2024-09-16T08:57:04.560 INFO:tasks.workunit.client.1.smithi097.stdout:4/153: mknod d0/d7/c33 0 2024-09-16T08:57:04.561 INFO:tasks.workunit.client.1.smithi097.stdout:1/69: dwrite d5/f9 [0,4194304] 0 2024-09-16T08:57:04.562 INFO:tasks.workunit.client.1.smithi097.stdout:3/70: dwrite d7/de/ff [0,4194304] 0 2024-09-16T08:57:04.562 INFO:tasks.workunit.client.1.smithi097.stdout:9/83: dwrite d2/f6 [0,4194304] 0 2024-09-16T08:57:04.562 INFO:tasks.workunit.client.1.smithi097.stdout:9/84: fsync d2/fe 0 2024-09-16T08:57:04.562 INFO:tasks.workunit.client.1.smithi097.stdout:9/85: dread - d2/d5/f11 zero size 2024-09-16T08:57:04.563 INFO:tasks.workunit.client.1.smithi097.stdout:7/120: symlink d4/d12/l2a 0 2024-09-16T08:57:04.568 INFO:tasks.workunit.client.1.smithi097.stdout:1/70: mkdir d5/db/dc/d16 0 2024-09-16T08:57:04.568 INFO:tasks.workunit.client.1.smithi097.stdout:4/154: dread d0/d7/df/d15/d16/f1d [0,4194304] 0 2024-09-16T08:57:04.568 INFO:tasks.workunit.client.1.smithi097.stdout:4/155: chown d0/d7/fb 38962087 1 2024-09-16T08:57:04.572 INFO:tasks.workunit.client.1.smithi097.stdout:7/121: mkdir d4/d9/d2b 0 2024-09-16T08:57:04.572 INFO:tasks.workunit.client.1.smithi097.stdout:7/122: creat d4/d9/d1c/f2c x:0 0 0 2024-09-16T08:57:04.572 INFO:tasks.workunit.client.1.smithi097.stdout:7/123: chown d4/d7/f14 10366 1 2024-09-16T08:57:04.573 INFO:tasks.workunit.client.1.smithi097.stdout:7/124: write d4/d9/d1c/f28 [569237,127549] 0 2024-09-16T08:57:04.573 INFO:tasks.workunit.client.1.smithi097.stdout:7/125: write d4/d1b/f1e [1942447,126130] 0 2024-09-16T08:57:04.573 INFO:tasks.workunit.client.1.smithi097.stdout:7/126: creat d4/d9/d1c/f2d x:0 0 0 2024-09-16T08:57:04.573 INFO:tasks.workunit.client.1.smithi097.stdout:4/156: symlink d0/d7/df/d15/d16/d26/l34 0 2024-09-16T08:57:04.573 INFO:tasks.workunit.client.1.smithi097.stdout:4/157: readlink d0/d7/lc 0 2024-09-16T08:57:04.576 INFO:tasks.workunit.client.1.smithi097.stdout:7/127: creat d4/d9/d2b/f2e x:0 0 0 2024-09-16T08:57:04.576 INFO:tasks.workunit.client.1.smithi097.stdout:7/128: write d4/f17 [380255,18889] 0 2024-09-16T08:57:04.577 INFO:tasks.workunit.client.1.smithi097.stdout:4/158: mkdir d0/d35 0 2024-09-16T08:57:04.579 INFO:tasks.workunit.client.1.smithi097.stdout:7/129: mknod d4/d1b/c2f 0 2024-09-16T08:57:04.579 INFO:tasks.workunit.client.1.smithi097.stdout:1/71: dread d5/f9 [0,4194304] 0 2024-09-16T08:57:04.579 INFO:tasks.workunit.client.1.smithi097.stdout:1/72: write d5/f12 [595322,75821] 0 2024-09-16T08:57:04.579 INFO:tasks.workunit.client.1.smithi097.stdout:9/86: dwrite d2/d5/f18 [0,4194304] 0 2024-09-16T08:57:04.580 INFO:tasks.workunit.client.1.smithi097.stdout:9/87: creat d2/f1a x:0 0 0 2024-09-16T08:57:04.581 INFO:tasks.workunit.client.1.smithi097.stdout:7/130: mkdir d4/d7/d30 0 2024-09-16T08:57:04.581 INFO:tasks.workunit.client.1.smithi097.stdout:7/131: write d4/d7/f11 [1766892,90209] 0 2024-09-16T08:57:04.583 INFO:tasks.workunit.client.1.smithi097.stdout:1/73: mkdir d5/d17 0 2024-09-16T08:57:04.584 INFO:tasks.workunit.client.1.smithi097.stdout:1/74: creat d5/f18 x:0 0 0 2024-09-16T08:57:04.588 INFO:tasks.workunit.client.1.smithi097.stdout:7/132: mknod d4/d9/d2b/c31 0 2024-09-16T08:57:04.588 INFO:tasks.workunit.client.1.smithi097.stdout:7/133: fsync d4/f5 0 2024-09-16T08:57:04.588 INFO:tasks.workunit.client.1.smithi097.stdout:7/134: fdatasync d4/d9/dc/f16 0 2024-09-16T08:57:04.588 INFO:tasks.workunit.client.1.smithi097.stdout:7/135: creat d4/d7/f32 x:0 0 0 2024-09-16T08:57:04.588 INFO:tasks.workunit.client.1.smithi097.stdout:7/136: readlink d4/d12/l2a 0 2024-09-16T08:57:04.593 INFO:tasks.workunit.client.1.smithi097.stdout:3/71: truncate d7/de/ff 400985 0 2024-09-16T08:57:04.594 INFO:tasks.workunit.client.1.smithi097.stdout:4/159: dwrite d0/d7/df/d15/d16/f1d [0,4194304] 0 2024-09-16T08:57:04.596 INFO:tasks.workunit.client.1.smithi097.stdout:3/72: creat d7/f14 x:0 0 0 2024-09-16T08:57:04.597 INFO:tasks.workunit.client.1.smithi097.stdout:3/73: readlink d7/l11 0 2024-09-16T08:57:04.598 INFO:tasks.workunit.client.1.smithi097.stdout:4/160: mkdir d0/d36 0 2024-09-16T08:57:04.601 INFO:tasks.workunit.client.1.smithi097.stdout:1/75: rename d5/db/d11 to d5/db/dc/d19 0 2024-09-16T08:57:04.603 INFO:tasks.workunit.client.1.smithi097.stdout:4/161: link d0/f2a d0/d7/df/f37 0 2024-09-16T08:57:04.603 INFO:tasks.workunit.client.1.smithi097.stdout:4/162: chown d0/f2f 514539735 1 2024-09-16T08:57:04.603 INFO:tasks.workunit.client.1.smithi097.stdout:1/76: symlink d5/d14/l1a 0 2024-09-16T08:57:04.604 INFO:tasks.workunit.client.1.smithi097.stdout:3/74: mkdir d7/d15 0 2024-09-16T08:57:04.605 INFO:tasks.workunit.client.1.smithi097.stdout:4/163: creat d0/d35/f38 x:0 0 0 2024-09-16T08:57:04.606 INFO:tasks.workunit.client.1.smithi097.stdout:1/77: rename d5/d14/l1a to d5/l1b 0 2024-09-16T08:57:04.607 INFO:tasks.workunit.client.1.smithi097.stdout:3/75: unlink d7/f14 0 2024-09-16T08:57:04.609 INFO:tasks.workunit.client.1.smithi097.stdout:4/164: mkdir d0/d39 0 2024-09-16T08:57:04.611 INFO:tasks.workunit.client.1.smithi097.stdout:1/78: dread d5/db/dc/f10 [0,4194304] 0 2024-09-16T08:57:04.611 INFO:tasks.workunit.client.1.smithi097.stdout:1/79: chown d5/db/ff 1895384 1 2024-09-16T08:57:04.614 INFO:tasks.workunit.client.1.smithi097.stdout:4/165: creat d0/d23/f3a x:0 0 0 2024-09-16T08:57:04.616 INFO:tasks.workunit.client.1.smithi097.stdout:3/76: mknod d7/c16 0 2024-09-16T08:57:04.616 INFO:tasks.workunit.client.1.smithi097.stdout:3/77: fdatasync d7/fa 0 2024-09-16T08:57:04.616 INFO:tasks.workunit.client.1.smithi097.stdout:4/166: link d0/d7/df/d15/l17 d0/d7/df/l3b 0 2024-09-16T08:57:04.616 INFO:tasks.workunit.client.1.smithi097.stdout:4/167: stat d0/d7/f24 0 2024-09-16T08:57:04.617 INFO:tasks.workunit.client.1.smithi097.stdout:1/80: write f4 [2847924,91055] 0 2024-09-16T08:57:04.617 INFO:tasks.workunit.client.1.smithi097.stdout:3/78: creat d7/d15/f17 x:0 0 0 2024-09-16T08:57:04.622 INFO:tasks.workunit.client.1.smithi097.stdout:7/137: truncate d4/d7/f11 688954 0 2024-09-16T08:57:04.622 INFO:tasks.workunit.client.1.smithi097.stdout:7/138: chown d4/d9/d1c/f28 0 1 2024-09-16T08:57:04.622 INFO:tasks.workunit.client.1.smithi097.stdout:7/139: readlink d4/d9/l20 0 2024-09-16T08:57:04.625 INFO:tasks.workunit.client.1.smithi097.stdout:7/140: rename d4/d7/f14 to d4/d7/d19/f33 0 2024-09-16T08:57:04.627 INFO:tasks.workunit.client.1.smithi097.stdout:1/81: dread d5/db/dc/f10 [0,4194304] 0 2024-09-16T08:57:04.631 INFO:tasks.workunit.client.1.smithi097.stdout:1/82: rmdir d5 39 2024-09-16T08:57:04.631 INFO:tasks.workunit.client.1.smithi097.stdout:1/83: write d5/f12 [646802,26152] 0 2024-09-16T08:57:04.634 INFO:tasks.workunit.client.1.smithi097.stdout:1/84: rmdir d5/db/dc/d16 0 2024-09-16T08:57:04.636 INFO:tasks.workunit.client.1.smithi097.stdout:1/85: dread d5/db/ff [0,4194304] 0 2024-09-16T08:57:04.644 INFO:tasks.workunit.client.1.smithi097.stdout:7/141: dwrite d4/f29 [0,4194304] 0 2024-09-16T08:57:04.644 INFO:tasks.workunit.client.1.smithi097.stdout:7/142: fsync d4/d7/f32 0 2024-09-16T08:57:04.650 INFO:tasks.workunit.client.1.smithi097.stdout:1/86: dwrite d5/db/dc/d19/f15 [0,4194304] 0 2024-09-16T08:57:04.652 INFO:tasks.workunit.client.1.smithi097.stdout:1/87: rename d5/db/dc/d19/c13 to d5/db/dc/d19/c1c 0 2024-09-16T08:57:04.653 INFO:tasks.workunit.client.1.smithi097.stdout:1/88: write d5/f12 [761432,57699] 0 2024-09-16T08:57:04.653 INFO:tasks.workunit.client.1.smithi097.stdout:1/89: truncate d5/db/dc/f10 4440008 0 2024-09-16T08:57:04.653 INFO:tasks.workunit.client.1.smithi097.stdout:1/90: mkdir d5/d1d 0 2024-09-16T08:57:04.659 INFO:tasks.workunit.client.1.smithi097.stdout:1/91: dread d5/db/dc/f10 [0,4194304] 0 2024-09-16T08:57:04.660 INFO:tasks.workunit.client.1.smithi097.stdout:1/92: symlink d5/db/dc/l1e 0 2024-09-16T08:57:04.660 INFO:tasks.workunit.client.1.smithi097.stdout:1/93: symlink d5/db/dc/d19/l1f 0 2024-09-16T08:57:04.660 INFO:tasks.workunit.client.1.smithi097.stdout:1/94: write d5/db/dc/f10 [2182608,87281] 0 2024-09-16T08:57:04.661 INFO:tasks.workunit.client.1.smithi097.stdout:1/95: link f4 d5/db/f20 0 2024-09-16T08:57:04.661 INFO:tasks.workunit.client.1.smithi097.stdout:1/96: chown d5 69327080 1 2024-09-16T08:57:04.662 INFO:tasks.workunit.client.1.smithi097.stdout:1/97: rename d5/db/dc/f10 to d5/f21 0 2024-09-16T08:57:04.663 INFO:tasks.workunit.client.1.smithi097.stdout:1/98: link d5/db/dc/l1e d5/db/l22 0 2024-09-16T08:57:04.664 INFO:tasks.workunit.client.1.smithi097.stdout:1/99: rename d5/f18 to d5/db/f23 0 2024-09-16T08:57:04.664 INFO:tasks.workunit.client.1.smithi097.stdout:1/100: dread - d5/db/f23 zero size 2024-09-16T08:57:04.664 INFO:tasks.workunit.client.1.smithi097.stdout:1/101: chown d5/la 0 1 2024-09-16T08:57:04.665 INFO:tasks.workunit.client.1.smithi097.stdout:1/102: write d5/db/f20 [3914445,99701] 0 2024-09-16T08:57:04.670 INFO:tasks.workunit.client.1.smithi097.stdout:0/93: sync 2024-09-16T08:57:04.670 INFO:tasks.workunit.client.1.smithi097.stdout:5/72: sync 2024-09-16T08:57:04.670 INFO:tasks.workunit.client.1.smithi097.stdout:5/73: dread - db/f14 zero size 2024-09-16T08:57:04.670 INFO:tasks.workunit.client.1.smithi097.stdout:6/93: sync 2024-09-16T08:57:04.670 INFO:tasks.workunit.client.1.smithi097.stdout:6/94: creat d1/f1b x:0 0 0 2024-09-16T08:57:04.674 INFO:tasks.workunit.client.1.smithi097.stdout:1/103: dread d5/db/f20 [0,4194304] 0 2024-09-16T08:57:04.674 INFO:tasks.workunit.client.1.smithi097.stdout:1/104: chown d5/db/l22 841710 1 2024-09-16T08:57:04.674 INFO:tasks.workunit.client.1.smithi097.stdout:0/94: write f8 [1230841,95344] 0 2024-09-16T08:57:04.676 INFO:tasks.workunit.client.1.smithi097.stdout:0/95: creat da/d1a/f1c x:0 0 0 2024-09-16T08:57:04.677 INFO:tasks.workunit.client.1.smithi097.stdout:0/96: mknod da/df/d13/c1d 0 2024-09-16T08:57:04.679 INFO:tasks.workunit.client.1.smithi097.stdout:6/95: write d1/d3/f7 [732011,48366] 0 2024-09-16T08:57:04.680 INFO:tasks.workunit.client.1.smithi097.stdout:6/96: chown d1/d3/d16 1301199296 1 2024-09-16T08:57:04.680 INFO:tasks.workunit.client.1.smithi097.stdout:6/97: fdatasync d1/d3/d16/f1a 0 2024-09-16T08:57:04.680 INFO:tasks.workunit.client.1.smithi097.stdout:6/98: chown d1/l13 324 1 2024-09-16T08:57:04.680 INFO:tasks.workunit.client.1.smithi097.stdout:6/99: creat d1/d3/f1c x:0 0 0 2024-09-16T08:57:04.680 INFO:tasks.workunit.client.1.smithi097.stdout:0/97: mkdir da/d1e 0 2024-09-16T08:57:04.685 INFO:tasks.workunit.client.1.smithi097.stdout:6/100: rmdir d1/dc 39 2024-09-16T08:57:04.685 INFO:tasks.workunit.client.1.smithi097.stdout:6/101: creat d1/f1d x:0 0 0 2024-09-16T08:57:04.686 INFO:tasks.workunit.client.1.smithi097.stdout:0/98: mkdir da/df/d1f 0 2024-09-16T08:57:04.688 INFO:tasks.workunit.client.1.smithi097.stdout:5/74: dwrite db/f14 [0,4194304] 0 2024-09-16T08:57:04.688 INFO:tasks.workunit.client.1.smithi097.stdout:1/105: truncate d5/f12 738247 0 2024-09-16T08:57:04.689 INFO:tasks.workunit.client.1.smithi097.stdout:5/75: creat db/f15 x:0 0 0 2024-09-16T08:57:04.689 INFO:tasks.workunit.client.1.smithi097.stdout:1/106: chown d5/db/dc/d19 240871980 1 2024-09-16T08:57:04.689 INFO:tasks.workunit.client.1.smithi097.stdout:1/107: chown d5/db/dc/d19/l1f 11081 1 2024-09-16T08:57:04.689 INFO:tasks.workunit.client.1.smithi097.stdout:6/102: mkdir d1/dc/d1e 0 2024-09-16T08:57:04.693 INFO:tasks.workunit.client.1.smithi097.stdout:1/108: rmdir d5/d1d 0 2024-09-16T08:57:04.697 INFO:tasks.workunit.client.1.smithi097.stdout:5/76: dread f6 [0,4194304] 0 2024-09-16T08:57:04.699 INFO:tasks.workunit.client.1.smithi097.stdout:0/99: dread f5 [4194304,4194304] 0 2024-09-16T08:57:04.701 INFO:tasks.workunit.client.1.smithi097.stdout:0/100: rename f5 to da/d1e/f20 0 2024-09-16T08:57:04.704 INFO:tasks.workunit.client.1.smithi097.stdout:0/101: link da/df/d13/c17 da/df/d1f/c21 0 2024-09-16T08:57:04.704 INFO:tasks.workunit.client.1.smithi097.stdout:0/102: readlink l3 0 2024-09-16T08:57:04.710 INFO:tasks.workunit.client.1.smithi097.stdout:0/103: write da/d1e/f20 [6851647,60231] 0 2024-09-16T08:57:04.710 INFO:tasks.workunit.client.1.smithi097.stdout:0/104: dread - da/f1b zero size 2024-09-16T08:57:04.710 INFO:tasks.workunit.client.1.smithi097.stdout:0/105: dread - da/f1b zero size 2024-09-16T08:57:04.716 INFO:tasks.workunit.client.1.smithi097.stdout:6/103: dwrite d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:04.718 INFO:tasks.workunit.client.1.smithi097.stdout:6/104: unlink d1/d3/d16/f1a 0 2024-09-16T08:57:04.718 INFO:tasks.workunit.client.1.smithi097.stdout:6/105: write d1/f1d [531783,108027] 0 2024-09-16T08:57:04.718 INFO:tasks.workunit.client.1.smithi097.stdout:6/106: fdatasync d1/f1b 0 2024-09-16T08:57:04.718 INFO:tasks.workunit.client.1.smithi097.stdout:6/107: fdatasync d1/d3/f7 0 2024-09-16T08:57:04.718 INFO:tasks.workunit.client.1.smithi097.stdout:6/108: dread - d1/f1b zero size 2024-09-16T08:57:04.719 INFO:tasks.workunit.client.1.smithi097.stdout:6/109: rename d1/f1d to d1/d3/f1f 0 2024-09-16T08:57:04.719 INFO:tasks.workunit.client.1.smithi097.stdout:6/110: creat d1/d3/d16/f20 x:0 0 0 2024-09-16T08:57:04.719 INFO:tasks.workunit.client.1.smithi097.stdout:6/111: creat d1/d3/d16/f21 x:0 0 0 2024-09-16T08:57:04.725 INFO:tasks.workunit.client.1.smithi097.stdout:1/109: dwrite d5/db/f20 [0,4194304] 0 2024-09-16T08:57:04.725 INFO:tasks.workunit.client.1.smithi097.stdout:1/110: readlink d5/db/dc/d19/l1f 0 2024-09-16T08:57:04.725 INFO:tasks.workunit.client.1.smithi097.stdout:1/111: readlink d5/la 0 2024-09-16T08:57:04.726 INFO:tasks.workunit.client.1.smithi097.stdout:5/77: dwrite db/f13 [0,4194304] 0 2024-09-16T08:57:04.733 INFO:tasks.workunit.client.1.smithi097.stdout:5/78: getdents db 0 2024-09-16T08:57:04.734 INFO:tasks.workunit.client.1.smithi097.stdout:5/79: chown db/f14 1847982532 1 2024-09-16T08:57:04.734 INFO:tasks.workunit.client.1.smithi097.stdout:5/80: truncate f6 4358010 0 2024-09-16T08:57:04.735 INFO:tasks.workunit.client.1.smithi097.stdout:6/112: dread d1/d3/f1f [0,4194304] 0 2024-09-16T08:57:04.735 INFO:tasks.workunit.client.1.smithi097.stdout:6/113: stat d1/d3/d16 0 2024-09-16T08:57:04.736 INFO:tasks.workunit.client.1.smithi097.stdout:5/81: mkdir db/d16 0 2024-09-16T08:57:04.737 INFO:tasks.workunit.client.1.smithi097.stdout:6/114: rename d1/d3/d16/f20 to d1/f22 0 2024-09-16T08:57:04.738 INFO:tasks.workunit.client.1.smithi097.stdout:6/115: fsync d1/d3/f1c 0 2024-09-16T08:57:04.738 INFO:tasks.workunit.client.1.smithi097.stdout:6/116: stat d1/d3/d16/f21 0 2024-09-16T08:57:04.738 INFO:tasks.workunit.client.1.smithi097.stdout:6/117: dread d1/d3/f11 [0,4194304] 0 2024-09-16T08:57:04.739 INFO:tasks.workunit.client.1.smithi097.stdout:5/82: rename c2 to db/c17 0 2024-09-16T08:57:04.741 INFO:tasks.workunit.client.1.smithi097.stdout:5/83: rename db/d16 to db/d18 0 2024-09-16T08:57:04.741 INFO:tasks.workunit.client.1.smithi097.stdout:5/84: write fa [4330742,48840] 0 2024-09-16T08:57:04.742 INFO:tasks.workunit.client.1.smithi097.stdout:1/112: truncate f4 603838 0 2024-09-16T08:57:04.756 INFO:tasks.workunit.client.1.smithi097.stdout:6/118: dwrite d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:04.757 INFO:tasks.workunit.client.1.smithi097.stdout:6/119: mknod d1/c23 0 2024-09-16T08:57:04.757 INFO:tasks.workunit.client.1.smithi097.stdout:6/120: write d1/f1b [198205,84966] 0 2024-09-16T08:57:04.758 INFO:tasks.workunit.client.1.smithi097.stdout:6/121: mknod d1/c24 0 2024-09-16T08:57:04.758 INFO:tasks.workunit.client.1.smithi097.stdout:6/122: chown d1/d3/d16 7156 1 2024-09-16T08:57:04.758 INFO:tasks.workunit.client.1.smithi097.stdout:6/123: fsync d1/d3/d16/f21 0 2024-09-16T08:57:04.763 INFO:tasks.workunit.client.1.smithi097.stdout:5/85: dwrite db/fe [0,4194304] 0 2024-09-16T08:57:04.767 INFO:tasks.workunit.client.1.smithi097.stdout:1/113: dread d5/f21 [0,4194304] 0 2024-09-16T08:57:04.767 INFO:tasks.workunit.client.1.smithi097.stdout:1/114: readlink d5/db/dc/l1e 0 2024-09-16T08:57:04.767 INFO:tasks.workunit.client.1.smithi097.stdout:5/86: creat db/f19 x:0 0 0 2024-09-16T08:57:04.768 INFO:tasks.workunit.client.1.smithi097.stdout:1/115: creat d5/d17/f24 x:0 0 0 2024-09-16T08:57:04.772 INFO:tasks.workunit.client.1.smithi097.stdout:1/116: write d5/f9 [2355204,36385] 0 2024-09-16T08:57:04.772 INFO:tasks.workunit.client.1.smithi097.stdout:1/117: fsync d5/db/dc/d19/f15 0 2024-09-16T08:57:04.774 INFO:tasks.workunit.client.1.smithi097.stdout:1/118: dread d5/db/ff [0,4194304] 0 2024-09-16T08:57:04.775 INFO:tasks.workunit.client.1.smithi097.stdout:1/119: write d5/f21 [268489,114450] 0 2024-09-16T08:57:04.776 INFO:tasks.workunit.client.1.smithi097.stdout:1/120: truncate d5/db/ff 282993 0 2024-09-16T08:57:04.776 INFO:tasks.workunit.client.1.smithi097.stdout:1/121: read d5/f9 [206643,18002] 0 2024-09-16T08:57:04.777 INFO:tasks.workunit.client.1.smithi097.stdout:6/124: dwrite d1/d3/f1f [0,4194304] 0 2024-09-16T08:57:04.778 INFO:tasks.workunit.client.1.smithi097.stdout:6/125: symlink d1/dc/d1e/l25 0 2024-09-16T08:57:04.778 INFO:tasks.workunit.client.1.smithi097.stdout:6/126: write d1/d3/d16/f21 [904948,34216] 0 2024-09-16T08:57:04.778 INFO:tasks.workunit.client.1.smithi097.stdout:6/127: dread - d1/f22 zero size 2024-09-16T08:57:04.779 INFO:tasks.workunit.client.1.smithi097.stdout:6/128: write d1/d3/d16/f21 [443,109852] 0 2024-09-16T08:57:04.779 INFO:tasks.workunit.client.1.smithi097.stdout:6/129: write d1/f1b [1201023,7104] 0 2024-09-16T08:57:04.780 INFO:tasks.workunit.client.1.smithi097.stdout:1/122: write d5/db/ff [839692,61032] 0 2024-09-16T08:57:04.780 INFO:tasks.workunit.client.1.smithi097.stdout:1/123: readlink d5/la 0 2024-09-16T08:57:04.781 INFO:tasks.workunit.client.1.smithi097.stdout:1/124: mkdir d5/db/d25 0 2024-09-16T08:57:04.781 INFO:tasks.workunit.client.1.smithi097.stdout:1/125: creat d5/d17/f26 x:0 0 0 2024-09-16T08:57:04.783 INFO:tasks.workunit.client.1.smithi097.stdout:1/126: unlink d5/c7 0 2024-09-16T08:57:04.783 INFO:tasks.workunit.client.1.smithi097.stdout:1/127: stat d5/db/d25 0 2024-09-16T08:57:04.783 INFO:tasks.workunit.client.1.smithi097.stdout:1/128: chown d5/db/f20 6 1 2024-09-16T08:57:04.783 INFO:tasks.workunit.client.1.smithi097.stdout:1/129: truncate d5/d17/f26 599752 0 2024-09-16T08:57:04.783 INFO:tasks.workunit.client.1.smithi097.stdout:5/87: write db/fe [1335209,188] 0 2024-09-16T08:57:04.784 INFO:tasks.workunit.client.1.smithi097.stdout:5/88: unlink f0 0 2024-09-16T08:57:04.785 INFO:tasks.workunit.client.1.smithi097.stdout:6/130: dread d1/f1b [0,4194304] 0 2024-09-16T08:57:04.786 INFO:tasks.workunit.client.1.smithi097.stdout:6/131: dread d1/d3/f11 [0,4194304] 0 2024-09-16T08:57:04.787 INFO:tasks.workunit.client.1.smithi097.stdout:6/132: rename d1/c23 to d1/dc/d1e/c26 0 2024-09-16T08:57:04.788 INFO:tasks.workunit.client.1.smithi097.stdout:6/133: link d1/c15 d1/d3/df/c27 0 2024-09-16T08:57:04.791 INFO:tasks.workunit.client.1.smithi097.stdout:1/130: write d5/db/dc/d19/f15 [1347051,119687] 0 2024-09-16T08:57:04.807 INFO:tasks.workunit.client.1.smithi097.stdout:5/89: dwrite db/f14 [4194304,4194304] 0 2024-09-16T08:57:04.807 INFO:tasks.workunit.client.1.smithi097.stdout:1/131: dwrite d5/db/f23 [0,4194304] 0 2024-09-16T08:57:04.826 INFO:tasks.workunit.client.1.smithi097.stdout:5/90: dwrite db/f14 [4194304,4194304] 0 2024-09-16T08:57:04.827 INFO:tasks.workunit.client.1.smithi097.stdout:5/91: write f6 [5002685,10475] 0 2024-09-16T08:57:04.827 INFO:tasks.workunit.client.1.smithi097.stdout:5/92: chown f8 25243 1 2024-09-16T08:57:04.827 INFO:tasks.workunit.client.1.smithi097.stdout:5/93: mkdir db/d1a 0 2024-09-16T08:57:04.830 INFO:tasks.workunit.client.1.smithi097.stdout:1/132: dwrite d5/f12 [0,4194304] 0 2024-09-16T08:57:04.836 INFO:tasks.workunit.client.1.smithi097.stdout:5/94: read db/f13 [3821637,85224] 0 2024-09-16T08:57:04.836 INFO:tasks.workunit.client.1.smithi097.stdout:5/95: write fa [199059,88247] 0 2024-09-16T08:57:04.837 INFO:tasks.workunit.client.1.smithi097.stdout:5/96: mknod db/c1b 0 2024-09-16T08:57:04.839 INFO:tasks.workunit.client.1.smithi097.stdout:5/97: mkdir db/d18/d1c 0 2024-09-16T08:57:04.841 INFO:tasks.workunit.client.1.smithi097.stdout:5/98: rename f6 to db/f1d 0 2024-09-16T08:57:04.842 INFO:tasks.workunit.client.1.smithi097.stdout:5/99: symlink db/l1e 0 2024-09-16T08:57:04.843 INFO:tasks.workunit.client.1.smithi097.stdout:5/100: link db/f15 db/d1a/f1f 0 2024-09-16T08:57:04.844 INFO:tasks.workunit.client.1.smithi097.stdout:5/101: unlink db/f15 0 2024-09-16T08:57:04.845 INFO:tasks.workunit.client.1.smithi097.stdout:5/102: write fa [4414054,109135] 0 2024-09-16T08:57:04.845 INFO:tasks.workunit.client.1.smithi097.stdout:5/103: mkdir db/d18/d20 0 2024-09-16T08:57:04.846 INFO:tasks.workunit.client.1.smithi097.stdout:5/104: mknod db/d18/d1c/c21 0 2024-09-16T08:57:04.846 INFO:tasks.workunit.client.1.smithi097.stdout:5/105: dread - db/f10 zero size 2024-09-16T08:57:04.847 INFO:tasks.workunit.client.1.smithi097.stdout:5/106: mkdir db/d22 0 2024-09-16T08:57:04.848 INFO:tasks.workunit.client.1.smithi097.stdout:1/133: dwrite d5/d17/f26 [0,4194304] 0 2024-09-16T08:57:04.849 INFO:tasks.workunit.client.1.smithi097.stdout:5/107: rename db/lc to db/d18/d1c/l23 0 2024-09-16T08:57:04.849 INFO:tasks.workunit.client.1.smithi097.stdout:5/108: write db/d1a/f1f [706597,70688] 0 2024-09-16T08:57:04.850 INFO:tasks.workunit.client.1.smithi097.stdout:5/109: rmdir db/d18 39 2024-09-16T08:57:04.851 INFO:tasks.workunit.client.1.smithi097.stdout:5/110: truncate db/fe 1901473 0 2024-09-16T08:57:04.852 INFO:tasks.workunit.client.1.smithi097.stdout:1/134: dread d5/f9 [0,4194304] 0 2024-09-16T08:57:04.852 INFO:tasks.workunit.client.1.smithi097.stdout:5/111: rename db/d18/d1c/c21 to db/d1a/c24 0 2024-09-16T08:57:04.852 INFO:tasks.workunit.client.1.smithi097.stdout:5/112: write db/f14 [8428206,56370] 0 2024-09-16T08:57:04.854 INFO:tasks.workunit.client.1.smithi097.stdout:1/135: dread d5/db/f20 [0,4194304] 0 2024-09-16T08:57:04.854 INFO:tasks.workunit.client.1.smithi097.stdout:5/113: fdatasync db/d1a/f1f 0 2024-09-16T08:57:04.855 INFO:tasks.workunit.client.1.smithi097.stdout:5/114: creat db/f25 x:0 0 0 2024-09-16T08:57:04.855 INFO:tasks.workunit.client.1.smithi097.stdout:1/136: symlink d5/l27 0 2024-09-16T08:57:04.855 INFO:tasks.workunit.client.1.smithi097.stdout:1/137: read d5/f21 [120494,37224] 0 2024-09-16T08:57:04.859 INFO:tasks.workunit.client.1.smithi097.stdout:5/115: rename db/c17 to db/c26 0 2024-09-16T08:57:04.859 INFO:tasks.workunit.client.1.smithi097.stdout:5/116: write db/f1d [6009412,68778] 0 2024-09-16T08:57:04.860 INFO:tasks.workunit.client.1.smithi097.stdout:1/138: mknod d5/d17/c28 0 2024-09-16T08:57:04.861 INFO:tasks.workunit.client.1.smithi097.stdout:5/117: rename c9 to db/d18/c27 0 2024-09-16T08:57:04.864 INFO:tasks.workunit.client.1.smithi097.stdout:1/139: creat d5/db/dc/d19/f29 x:0 0 0 2024-09-16T08:57:04.866 INFO:tasks.workunit.client.1.smithi097.stdout:1/140: rename d5/db/ff to d5/db/dc/d19/f2a 0 2024-09-16T08:57:04.869 INFO:tasks.workunit.client.1.smithi097.stdout:1/141: rmdir d5/db 39 2024-09-16T08:57:04.874 INFO:tasks.workunit.client.1.smithi097.stdout:1/142: mknod d5/db/dc/d19/c2b 0 2024-09-16T08:57:04.876 INFO:tasks.workunit.client.1.smithi097.stdout:1/143: mkdir d5/d2c 0 2024-09-16T08:57:04.876 INFO:tasks.workunit.client.1.smithi097.stdout:5/118: dwrite db/f25 [0,4194304] 0 2024-09-16T08:57:04.877 INFO:tasks.workunit.client.1.smithi097.stdout:5/119: rename db to db/d18/d1c/d28 22 2024-09-16T08:57:04.878 INFO:tasks.workunit.client.1.smithi097.stdout:1/144: symlink d5/d2c/l2d 0 2024-09-16T08:57:04.881 INFO:tasks.workunit.client.1.smithi097.stdout:1/145: rmdir d5/db/dc 39 2024-09-16T08:57:04.881 INFO:tasks.workunit.client.1.smithi097.stdout:1/146: write d5/db/f20 [1457891,61896] 0 2024-09-16T08:57:04.881 INFO:tasks.workunit.client.1.smithi097.stdout:1/147: chown d5/l1b 8139982 1 2024-09-16T08:57:04.903 INFO:tasks.workunit.client.1.smithi097.stdout:5/120: dwrite db/f14 [4194304,4194304] 0 2024-09-16T08:57:04.906 INFO:tasks.workunit.client.1.smithi097.stdout:1/148: dwrite d5/d17/f26 [0,4194304] 0 2024-09-16T08:57:04.907 INFO:tasks.workunit.client.1.smithi097.stdout:1/149: creat d5/d2c/f2e x:0 0 0 2024-09-16T08:57:04.908 INFO:tasks.workunit.client.1.smithi097.stdout:1/150: rename d5/d17/c28 to d5/db/dc/d19/c2f 0 2024-09-16T08:57:04.909 INFO:tasks.workunit.client.1.smithi097.stdout:1/151: write d5/d2c/f2e [728409,108529] 0 2024-09-16T08:57:04.912 INFO:tasks.workunit.client.1.smithi097.stdout:5/121: write f8 [3899410,121093] 0 2024-09-16T08:57:04.914 INFO:tasks.workunit.client.1.smithi097.stdout:5/122: link fa db/d18/d20/f29 0 2024-09-16T08:57:04.926 INFO:tasks.workunit.client.1.smithi097.stdout:1/152: dwrite d5/d17/f24 [0,4194304] 0 2024-09-16T08:57:04.928 INFO:tasks.workunit.client.1.smithi097.stdout:9/88: sync 2024-09-16T08:57:04.928 INFO:tasks.workunit.client.1.smithi097.stdout:2/118: sync 2024-09-16T08:57:04.928 INFO:tasks.workunit.client.1.smithi097.stdout:9/89: creat d2/f1b x:0 0 0 2024-09-16T08:57:04.928 INFO:tasks.workunit.client.1.smithi097.stdout:4/168: sync 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:8/129: sync 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:4/169: stat d0/f2f 0 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:4/170: chown d0/d7/df/l22 29250524 1 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:1/153: rename d5/la to d5/d14/l30 0 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:1/154: stat d5/db 0 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:1/155: stat d5/l27 0 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:1/156: write d5/d2c/f2e [1381164,129300] 0 2024-09-16T08:57:04.929 INFO:tasks.workunit.client.1.smithi097.stdout:2/119: unlink d3/d4/d9/l15 0 2024-09-16T08:57:04.933 INFO:tasks.workunit.client.1.smithi097.stdout:4/171: truncate d0/d7/df/f37 351016 0 2024-09-16T08:57:04.933 INFO:tasks.workunit.client.1.smithi097.stdout:5/123: dwrite db/d18/d20/f29 [0,4194304] 0 2024-09-16T08:57:04.935 INFO:tasks.workunit.client.1.smithi097.stdout:1/157: unlink d5/c6 0 2024-09-16T08:57:04.936 INFO:tasks.workunit.client.1.smithi097.stdout:2/120: rmdir d3/d4/d8 39 2024-09-16T08:57:04.936 INFO:tasks.workunit.client.1.smithi097.stdout:2/121: creat d3/d4/d9/f1d x:0 0 0 2024-09-16T08:57:04.936 INFO:tasks.workunit.client.1.smithi097.stdout:2/122: write d3/d4/f6 [2525280,36649] 0 2024-09-16T08:57:04.936 INFO:tasks.workunit.client.1.smithi097.stdout:2/123: fsync d3/d4/dd/f18 0 2024-09-16T08:57:04.936 INFO:tasks.workunit.client.1.smithi097.stdout:2/124: write d3/d4/f6 [268351,97561] 0 2024-09-16T08:57:04.936 INFO:tasks.workunit.client.1.smithi097.stdout:2/125: fsync d3/d4/d9/ff 0 2024-09-16T08:57:04.936 INFO:tasks.workunit.client.1.smithi097.stdout:2/126: chown d3/d4/l5 2419 1 2024-09-16T08:57:04.937 INFO:tasks.workunit.client.1.smithi097.stdout:2/127: chown d3/d4/le 2588 1 2024-09-16T08:57:04.937 INFO:tasks.workunit.client.1.smithi097.stdout:2/128: chown d3/d4/d9/f12 7 1 2024-09-16T08:57:04.938 INFO:tasks.workunit.client.1.smithi097.stdout:9/90: rmdir d2 39 2024-09-16T08:57:04.939 INFO:tasks.workunit.client.1.smithi097.stdout:4/172: rename d0/d7/df/d15/d16/f19 to d0/d39/f3c 0 2024-09-16T08:57:04.940 INFO:tasks.workunit.client.1.smithi097.stdout:2/129: dread d3/d4/d8/f16 [0,4194304] 0 2024-09-16T08:57:04.941 INFO:tasks.workunit.client.1.smithi097.stdout:2/130: write d3/d4/d9/ff [5838977,76186] 0 2024-09-16T08:57:04.941 INFO:tasks.workunit.client.1.smithi097.stdout:2/131: write d3/d4/d9/f12 [1713999,115232] 0 2024-09-16T08:57:04.941 INFO:tasks.workunit.client.1.smithi097.stdout:2/132: creat d3/d4/d9/f1e x:0 0 0 2024-09-16T08:57:04.941 INFO:tasks.workunit.client.1.smithi097.stdout:2/133: creat d3/d4/dd/f1f x:0 0 0 2024-09-16T08:57:04.941 INFO:tasks.workunit.client.1.smithi097.stdout:5/124: symlink db/d18/d1c/l2a 0 2024-09-16T08:57:04.941 INFO:tasks.workunit.client.1.smithi097.stdout:9/91: mknod d2/d5/c1c 0 2024-09-16T08:57:04.941 INFO:tasks.workunit.client.1.smithi097.stdout:9/92: dread - d2/d5/f11 zero size 2024-09-16T08:57:04.942 INFO:tasks.workunit.client.1.smithi097.stdout:1/158: rename d5/d14/l30 to d5/db/d25/l31 0 2024-09-16T08:57:04.943 INFO:tasks.workunit.client.1.smithi097.stdout:1/159: dread - d5/db/dc/d19/f29 zero size 2024-09-16T08:57:04.943 INFO:tasks.workunit.client.1.smithi097.stdout:1/160: write d5/db/f20 [2107668,109500] 0 2024-09-16T08:57:04.943 INFO:tasks.workunit.client.1.smithi097.stdout:4/173: mkdir d0/d7/df/d15/d16/d26/d3d 0 2024-09-16T08:57:04.943 INFO:tasks.workunit.client.1.smithi097.stdout:4/174: creat d0/f3e x:0 0 0 2024-09-16T08:57:04.945 INFO:tasks.workunit.client.1.smithi097.stdout:5/125: mkdir db/d18/d20/d2b 0 2024-09-16T08:57:04.945 INFO:tasks.workunit.client.1.smithi097.stdout:5/126: chown db/d18/d20/f29 47 1 2024-09-16T08:57:04.946 INFO:tasks.workunit.client.1.smithi097.stdout:2/134: rename d3/d4/f6 to d3/d4/d9/f20 0 2024-09-16T08:57:04.949 INFO:tasks.workunit.client.1.smithi097.stdout:5/127: write db/f1d [2360190,108115] 0 2024-09-16T08:57:04.949 INFO:tasks.workunit.client.1.smithi097.stdout:4/175: getdents d0/d7/df/d15/d16/d29 0 2024-09-16T08:57:04.952 INFO:tasks.workunit.client.1.smithi097.stdout:8/130: dwrite d8/dd/d15/f29 [0,4194304] 0 2024-09-16T08:57:04.953 INFO:tasks.workunit.client.1.smithi097.stdout:4/176: mknod d0/d36/c3f 0 2024-09-16T08:57:04.953 INFO:tasks.workunit.client.1.smithi097.stdout:4/177: readlink d0/d7/df/d15/d16/d26/l34 0 2024-09-16T08:57:04.954 INFO:tasks.workunit.client.1.smithi097.stdout:5/128: dread db/fe [0,4194304] 0 2024-09-16T08:57:04.959 INFO:tasks.workunit.client.1.smithi097.stdout:4/178: mkdir d0/d7/df/d15/d40 0 2024-09-16T08:57:04.959 INFO:tasks.workunit.client.1.smithi097.stdout:4/179: write d0/d39/f3c [514217,20851] 0 2024-09-16T08:57:04.960 INFO:tasks.workunit.client.1.smithi097.stdout:5/129: dread db/f13 [0,4194304] 0 2024-09-16T08:57:04.961 INFO:tasks.workunit.client.1.smithi097.stdout:1/161: getdents d5/db/d25 0 2024-09-16T08:57:04.962 INFO:tasks.workunit.client.1.smithi097.stdout:5/130: truncate db/f14 1353622 0 2024-09-16T08:57:04.966 INFO:tasks.workunit.client.1.smithi097.stdout:1/162: dread d5/db/dc/d19/f15 [0,4194304] 0 2024-09-16T08:57:04.967 INFO:tasks.workunit.client.1.smithi097.stdout:1/163: write d5/f21 [1464020,71533] 0 2024-09-16T08:57:04.967 INFO:tasks.workunit.client.1.smithi097.stdout:1/164: chown d5/d2c/f2e 43484 1 2024-09-16T08:57:04.967 INFO:tasks.workunit.client.1.smithi097.stdout:1/165: readlink d5/d2c/l2d 0 2024-09-16T08:57:04.970 INFO:tasks.workunit.client.1.smithi097.stdout:1/166: dread d5/db/dc/d19/f15 [0,4194304] 0 2024-09-16T08:57:04.972 INFO:tasks.workunit.client.1.smithi097.stdout:1/167: read d5/f21 [1315285,108207] 0 2024-09-16T08:57:04.972 INFO:tasks.workunit.client.1.smithi097.stdout:1/168: creat d5/d17/f32 x:0 0 0 2024-09-16T08:57:04.973 INFO:tasks.workunit.client.1.smithi097.stdout:8/131: dwrite d8/f24 [0,4194304] 0 2024-09-16T08:57:04.975 INFO:tasks.workunit.client.1.smithi097.stdout:9/93: dwrite d2/fe [0,4194304] 0 2024-09-16T08:57:04.977 INFO:tasks.workunit.client.1.smithi097.stdout:1/169: rename d5/db/dc/d19/f2a to d5/db/dc/f33 0 2024-09-16T08:57:04.977 INFO:tasks.workunit.client.1.smithi097.stdout:1/170: readlink d5/db/l22 0 2024-09-16T08:57:04.977 INFO:tasks.workunit.client.1.smithi097.stdout:1/171: write d5/db/f20 [1053835,40552] 0 2024-09-16T08:57:04.978 INFO:tasks.workunit.client.1.smithi097.stdout:5/131: dread db/d18/d20/f29 [0,4194304] 0 2024-09-16T08:57:04.983 INFO:tasks.workunit.client.1.smithi097.stdout:9/94: truncate d2/d5/f18 2243038 0 2024-09-16T08:57:04.986 INFO:tasks.workunit.client.1.smithi097.stdout:9/95: unlink d2/d5/f16 0 2024-09-16T08:57:04.988 INFO:tasks.workunit.client.1.smithi097.stdout:9/96: mknod d2/d5/c1d 0 2024-09-16T08:57:04.989 INFO:tasks.workunit.client.1.smithi097.stdout:9/97: chown d2/c7 12 1 2024-09-16T08:57:04.990 INFO:tasks.workunit.client.1.smithi097.stdout:9/98: rename d2/d5/c1c to d2/d5/c1e 0 2024-09-16T08:57:04.991 INFO:tasks.workunit.client.1.smithi097.stdout:9/99: truncate d2/f4 2972855 0 2024-09-16T08:57:04.992 INFO:tasks.workunit.client.1.smithi097.stdout:1/172: dread d5/d17/f26 [0,4194304] 0 2024-09-16T08:57:05.000 INFO:tasks.workunit.client.1.smithi097.stdout:5/132: dwrite db/f19 [0,4194304] 0 2024-09-16T08:57:05.000 INFO:tasks.workunit.client.1.smithi097.stdout:5/133: readlink db/d18/d1c/l2a 0 2024-09-16T08:57:05.018 INFO:tasks.workunit.client.1.smithi097.stdout:1/173: dwrite d5/d2c/f2e [0,4194304] 0 2024-09-16T08:57:05.018 INFO:tasks.workunit.client.1.smithi097.stdout:1/174: write d5/f9 [3410686,123244] 0 2024-09-16T08:57:05.020 INFO:tasks.workunit.client.1.smithi097.stdout:1/175: rename d5/db/dc/d19/f15 to d5/db/f34 0 2024-09-16T08:57:05.020 INFO:tasks.workunit.client.1.smithi097.stdout:1/176: fdatasync d5/db/f23 0 2024-09-16T08:57:05.024 INFO:tasks.workunit.client.1.smithi097.stdout:9/100: dwrite d2/f4 [0,4194304] 0 2024-09-16T08:57:05.025 INFO:tasks.workunit.client.1.smithi097.stdout:9/101: rmdir d2 39 2024-09-16T08:57:05.026 INFO:tasks.workunit.client.1.smithi097.stdout:9/102: rename d2/d5/c19 to d2/c1f 0 2024-09-16T08:57:05.027 INFO:tasks.workunit.client.1.smithi097.stdout:9/103: symlink d2/d5/l20 0 2024-09-16T08:57:05.028 INFO:tasks.workunit.client.1.smithi097.stdout:9/104: creat d2/d5/f21 x:0 0 0 2024-09-16T08:57:05.029 INFO:tasks.workunit.client.1.smithi097.stdout:3/79: sync 2024-09-16T08:57:05.033 INFO:tasks.workunit.client.1.smithi097.stdout:5/134: dwrite db/d18/d20/f29 [4194304,4194304] 0 2024-09-16T08:57:05.037 INFO:tasks.workunit.client.1.smithi097.stdout:5/135: creat db/d1a/f2c x:0 0 0 2024-09-16T08:57:05.040 INFO:tasks.workunit.client.1.smithi097.stdout:5/136: creat db/d18/d20/d2b/f2d x:0 0 0 2024-09-16T08:57:05.041 INFO:tasks.workunit.client.1.smithi097.stdout:9/105: dread f0 [0,4194304] 0 2024-09-16T08:57:05.043 INFO:tasks.workunit.client.1.smithi097.stdout:9/106: getdents d2 0 2024-09-16T08:57:05.045 INFO:tasks.workunit.client.1.smithi097.stdout:9/107: symlink d2/l22 0 2024-09-16T08:57:05.046 INFO:tasks.workunit.client.1.smithi097.stdout:9/108: symlink d2/l23 0 2024-09-16T08:57:05.047 INFO:tasks.workunit.client.1.smithi097.stdout:9/109: mknod d2/c24 0 2024-09-16T08:57:05.049 INFO:tasks.workunit.client.1.smithi097.stdout:9/110: link f0 d2/f25 0 2024-09-16T08:57:05.049 INFO:tasks.workunit.client.1.smithi097.stdout:9/111: write f0 [6096369,22093] 0 2024-09-16T08:57:05.049 INFO:tasks.workunit.client.1.smithi097.stdout:9/112: write d2/f1a [328319,5499] 0 2024-09-16T08:57:05.050 INFO:tasks.workunit.client.1.smithi097.stdout:1/177: dwrite d5/db/f20 [0,4194304] 0 2024-09-16T08:57:05.055 INFO:tasks.workunit.client.1.smithi097.stdout:9/113: write d2/d5/f18 [487394,77746] 0 2024-09-16T08:57:05.057 INFO:tasks.workunit.client.1.smithi097.stdout:9/114: link d2/d5/c17 d2/d5/c26 0 2024-09-16T08:57:05.058 INFO:tasks.workunit.client.1.smithi097.stdout:9/115: readlink d2/l23 0 2024-09-16T08:57:05.058 INFO:tasks.workunit.client.1.smithi097.stdout:5/137: dwrite db/f14 [0,4194304] 0 2024-09-16T08:57:05.058 INFO:tasks.workunit.client.1.smithi097.stdout:5/138: creat db/f2e x:0 0 0 2024-09-16T08:57:05.058 INFO:tasks.workunit.client.1.smithi097.stdout:9/116: truncate d2/f4 3025451 0 2024-09-16T08:57:05.059 INFO:tasks.workunit.client.1.smithi097.stdout:5/139: unlink c3 0 2024-09-16T08:57:05.060 INFO:tasks.workunit.client.1.smithi097.stdout:9/117: symlink d2/l27 0 2024-09-16T08:57:05.061 INFO:tasks.workunit.client.1.smithi097.stdout:5/140: mknod db/d18/d20/d2b/c2f 0 2024-09-16T08:57:05.061 INFO:tasks.workunit.client.1.smithi097.stdout:5/141: write db/f2e [939806,3234] 0 2024-09-16T08:57:05.061 INFO:tasks.workunit.client.1.smithi097.stdout:5/142: truncate db/d1a/f2c 293843 0 2024-09-16T08:57:05.062 INFO:tasks.workunit.client.1.smithi097.stdout:9/118: creat d2/f28 x:0 0 0 2024-09-16T08:57:05.062 INFO:tasks.workunit.client.1.smithi097.stdout:9/119: creat d2/d5/f29 x:0 0 0 2024-09-16T08:57:05.062 INFO:tasks.workunit.client.1.smithi097.stdout:9/120: chown d2/f6 865238 1 2024-09-16T08:57:05.062 INFO:tasks.workunit.client.1.smithi097.stdout:9/121: creat d2/f2a x:0 0 0 2024-09-16T08:57:05.062 INFO:tasks.workunit.client.1.smithi097.stdout:9/122: write d2/f2a [548927,20027] 0 2024-09-16T08:57:05.063 INFO:tasks.workunit.client.1.smithi097.stdout:5/143: mknod db/d18/d1c/c30 0 2024-09-16T08:57:05.063 INFO:tasks.workunit.client.1.smithi097.stdout:5/144: chown db/l11 30419 1 2024-09-16T08:57:05.063 INFO:tasks.workunit.client.1.smithi097.stdout:5/145: chown db/f25 63277 1 2024-09-16T08:57:05.063 INFO:tasks.workunit.client.1.smithi097.stdout:5/146: readlink db/l11 0 2024-09-16T08:57:05.063 INFO:tasks.workunit.client.1.smithi097.stdout:5/147: dread db/d1a/f2c [0,4194304] 0 2024-09-16T08:57:05.067 INFO:tasks.workunit.client.1.smithi097.stdout:9/123: dread d2/f25 [0,4194304] 0 2024-09-16T08:57:05.071 INFO:tasks.workunit.client.1.smithi097.stdout:5/148: getdents db/d18/d20/d2b 0 2024-09-16T08:57:05.076 INFO:tasks.workunit.client.1.smithi097.stdout:5/149: truncate db/f14 542036 0 2024-09-16T08:57:05.077 INFO:tasks.workunit.client.1.smithi097.stdout:5/150: truncate db/f19 3202912 0 2024-09-16T08:57:05.078 INFO:tasks.workunit.client.1.smithi097.stdout:5/151: unlink db/f1d 0 2024-09-16T08:57:05.080 INFO:tasks.workunit.client.1.smithi097.stdout:9/124: truncate d2/f4 2442376 0 2024-09-16T08:57:05.080 INFO:tasks.workunit.client.1.smithi097.stdout:9/125: stat d2/d5/c1d 0 2024-09-16T08:57:05.082 INFO:tasks.workunit.client.1.smithi097.stdout:5/152: rename db/d1a/c24 to db/c31 0 2024-09-16T08:57:05.083 INFO:tasks.workunit.client.1.smithi097.stdout:9/126: rename d2/d5 to d2/d5/d2b 22 2024-09-16T08:57:05.084 INFO:tasks.workunit.client.1.smithi097.stdout:5/153: rename f8 to db/f32 0 2024-09-16T08:57:05.086 INFO:tasks.workunit.client.1.smithi097.stdout:5/154: mknod db/d18/d1c/c33 0 2024-09-16T08:57:05.087 INFO:tasks.workunit.client.1.smithi097.stdout:5/155: creat db/d18/f34 x:0 0 0 2024-09-16T08:57:05.093 INFO:tasks.workunit.client.1.smithi097.stdout:5/156: getdents db/d18/d1c 0 2024-09-16T08:57:05.093 INFO:tasks.workunit.client.1.smithi097.stdout:5/157: chown db/d18 608 1 2024-09-16T08:57:05.094 INFO:tasks.workunit.client.1.smithi097.stdout:5/158: read - db/d18/d20/d2b/f2d zero size 2024-09-16T08:57:05.094 INFO:tasks.workunit.client.1.smithi097.stdout:5/159: truncate db/f10 952422 0 2024-09-16T08:57:05.094 INFO:tasks.workunit.client.1.smithi097.stdout:5/160: stat db/l11 0 2024-09-16T08:57:05.101 INFO:tasks.workunit.client.1.smithi097.stdout:9/127: dwrite d2/f2a [0,4194304] 0 2024-09-16T08:57:05.104 INFO:tasks.workunit.client.1.smithi097.stdout:9/128: mkdir d2/d5/d2c 0 2024-09-16T08:57:05.105 INFO:tasks.workunit.client.1.smithi097.stdout:5/161: dread fa [0,4194304] 0 2024-09-16T08:57:05.107 INFO:tasks.workunit.client.1.smithi097.stdout:5/162: truncate db/f2e 563404 0 2024-09-16T08:57:05.109 INFO:tasks.workunit.client.1.smithi097.stdout:5/163: rename db/d18/d1c/c33 to db/d18/d20/c35 0 2024-09-16T08:57:05.110 INFO:tasks.workunit.client.1.smithi097.stdout:5/164: symlink db/d18/d20/d2b/l36 0 2024-09-16T08:57:05.110 INFO:tasks.workunit.client.1.smithi097.stdout:5/165: readlink db/d18/d1c/l23 0 2024-09-16T08:57:05.111 INFO:tasks.workunit.client.1.smithi097.stdout:5/166: symlink db/d18/d20/l37 0 2024-09-16T08:57:05.112 INFO:tasks.workunit.client.1.smithi097.stdout:5/167: creat db/d1a/f38 x:0 0 0 2024-09-16T08:57:05.113 INFO:tasks.workunit.client.1.smithi097.stdout:5/168: rename db/f25 to db/d18/d1c/f39 0 2024-09-16T08:57:05.116 INFO:tasks.workunit.client.1.smithi097.stdout:5/169: dread db/f14 [0,4194304] 0 2024-09-16T08:57:05.116 INFO:tasks.workunit.client.1.smithi097.stdout:5/170: write db/d18/d1c/f39 [4942929,91188] 0 2024-09-16T08:57:05.116 INFO:tasks.workunit.client.1.smithi097.stdout:5/171: dread - db/d18/d20/d2b/f2d zero size 2024-09-16T08:57:05.117 INFO:tasks.workunit.client.1.smithi097.stdout:5/172: creat db/d1a/f3a x:0 0 0 2024-09-16T08:57:05.119 INFO:tasks.workunit.client.1.smithi097.stdout:5/173: unlink db/f13 0 2024-09-16T08:57:05.119 INFO:tasks.workunit.client.1.smithi097.stdout:5/174: stat db/c1b 0 2024-09-16T08:57:05.119 INFO:tasks.workunit.client.1.smithi097.stdout:9/129: dwrite d2/d5/f21 [0,4194304] 0 2024-09-16T08:57:05.124 INFO:tasks.workunit.client.1.smithi097.stdout:9/130: write d2/fc [4366325,66751] 0 2024-09-16T08:57:05.124 INFO:tasks.workunit.client.1.smithi097.stdout:9/131: dread d2/d5/f18 [0,4194304] 0 2024-09-16T08:57:05.134 INFO:tasks.workunit.client.1.smithi097.stdout:9/132: write d2/d5/f21 [3460880,239] 0 2024-09-16T08:57:05.136 INFO:tasks.workunit.client.1.smithi097.stdout:5/175: dwrite db/f32 [4194304,4194304] 0 2024-09-16T08:57:05.136 INFO:tasks.workunit.client.1.smithi097.stdout:5/176: write db/f32 [3437495,49867] 0 2024-09-16T08:57:05.138 INFO:tasks.workunit.client.1.smithi097.stdout:9/133: truncate d2/fc 3011036 0 2024-09-16T08:57:05.138 INFO:tasks.workunit.client.1.smithi097.stdout:9/134: fsync d2/f1b 0 2024-09-16T08:57:05.139 INFO:tasks.workunit.client.1.smithi097.stdout:5/177: link db/d18/c27 db/d1a/c3b 0 2024-09-16T08:57:05.140 INFO:tasks.workunit.client.1.smithi097.stdout:5/178: rename db/c26 to db/d22/c3c 0 2024-09-16T08:57:05.141 INFO:tasks.workunit.client.1.smithi097.stdout:5/179: dread db/d1a/f2c [0,4194304] 0 2024-09-16T08:57:05.141 INFO:tasks.workunit.client.1.smithi097.stdout:5/180: chown db/d18/f34 0 1 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:7/143: sync 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:7/144: fdatasync d4/d7/f32 0 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:7/145: creat d4/d9/d1c/f34 x:0 0 0 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:7/146: write d4/d9/f24 [3447071,505] 0 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:7/147: creat d4/d1b/f35 x:0 0 0 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:0/106: sync 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:0/107: creat da/d1a/f22 x:0 0 0 2024-09-16T08:57:05.148 INFO:tasks.workunit.client.1.smithi097.stdout:6/134: sync 2024-09-16T08:57:05.149 INFO:tasks.workunit.client.1.smithi097.stdout:6/135: stat d1/d3/f7 0 2024-09-16T08:57:05.149 INFO:tasks.workunit.client.1.smithi097.stdout:9/135: dread d2/d5/f21 [0,4194304] 0 2024-09-16T08:57:05.149 INFO:tasks.workunit.client.1.smithi097.stdout:5/181: dread fa [0,4194304] 0 2024-09-16T08:57:05.149 INFO:tasks.workunit.client.1.smithi097.stdout:5/182: chown db/d18/d20/d2b/l36 4653267 1 2024-09-16T08:57:05.150 INFO:tasks.workunit.client.1.smithi097.stdout:7/148: mkdir d4/d1b/d36 0 2024-09-16T08:57:05.151 INFO:tasks.workunit.client.1.smithi097.stdout:0/108: stat da/df/d1f/c21 0 2024-09-16T08:57:05.151 INFO:tasks.workunit.client.1.smithi097.stdout:0/109: write da/d1a/f1c [620439,69534] 0 2024-09-16T08:57:05.151 INFO:tasks.workunit.client.1.smithi097.stdout:6/136: mknod d1/c28 0 2024-09-16T08:57:05.151 INFO:tasks.workunit.client.1.smithi097.stdout:6/137: dread - d1/f22 zero size 2024-09-16T08:57:05.152 INFO:tasks.workunit.client.1.smithi097.stdout:6/138: chown d1/f12 205795 1 2024-09-16T08:57:05.153 INFO:tasks.workunit.client.1.smithi097.stdout:7/149: mknod d4/d9/dc/c37 0 2024-09-16T08:57:05.154 INFO:tasks.workunit.client.1.smithi097.stdout:7/150: creat d4/d9/d1c/f38 x:0 0 0 2024-09-16T08:57:05.154 INFO:tasks.workunit.client.1.smithi097.stdout:6/139: dread d1/d3/d16/f21 [0,4194304] 0 2024-09-16T08:57:05.154 INFO:tasks.workunit.client.1.smithi097.stdout:0/110: mknod da/d1a/c23 0 2024-09-16T08:57:05.155 INFO:tasks.workunit.client.1.smithi097.stdout:5/183: rmdir db 39 2024-09-16T08:57:05.155 INFO:tasks.workunit.client.1.smithi097.stdout:5/184: write db/d18/f34 [274917,58876] 0 2024-09-16T08:57:05.157 INFO:tasks.workunit.client.1.smithi097.stdout:7/151: symlink d4/d12/l39 0 2024-09-16T08:57:05.157 INFO:tasks.workunit.client.1.smithi097.stdout:6/140: truncate d1/f1b 126010 0 2024-09-16T08:57:05.159 INFO:tasks.workunit.client.1.smithi097.stdout:9/136: truncate d2/d5/f21 2400079 0 2024-09-16T08:57:05.160 INFO:tasks.workunit.client.1.smithi097.stdout:7/152: link d4/d9/dc/f16 d4/d1b/d36/f3a 0 2024-09-16T08:57:05.160 INFO:tasks.workunit.client.1.smithi097.stdout:5/185: write db/d1a/f1f [132330,97622] 0 2024-09-16T08:57:05.161 INFO:tasks.workunit.client.1.smithi097.stdout:6/141: rename d1/dc/cd to d1/dc/c29 0 2024-09-16T08:57:05.161 INFO:tasks.workunit.client.1.smithi097.stdout:6/142: truncate d1/d3/f1f 4376607 0 2024-09-16T08:57:05.162 INFO:tasks.workunit.client.1.smithi097.stdout:0/111: rmdir da/df/d13 39 2024-09-16T08:57:05.162 INFO:tasks.workunit.client.1.smithi097.stdout:0/112: fdatasync da/f15 0 2024-09-16T08:57:05.163 INFO:tasks.workunit.client.1.smithi097.stdout:9/137: rmdir d2/d5/d2c 0 2024-09-16T08:57:05.165 INFO:tasks.workunit.client.1.smithi097.stdout:7/153: creat d4/d12/d1f/f3b x:0 0 0 2024-09-16T08:57:05.167 INFO:tasks.workunit.client.1.smithi097.stdout:6/143: rename d1/d3/d16/c19 to d1/d3/df/c2a 0 2024-09-16T08:57:05.167 INFO:tasks.workunit.client.1.smithi097.stdout:6/144: creat d1/d3/f2b x:0 0 0 2024-09-16T08:57:05.167 INFO:tasks.workunit.client.1.smithi097.stdout:6/145: creat d1/d3/f2c x:0 0 0 2024-09-16T08:57:05.168 INFO:tasks.workunit.client.1.smithi097.stdout:6/146: read - d1/f22 zero size 2024-09-16T08:57:05.168 INFO:tasks.workunit.client.1.smithi097.stdout:0/113: rename da/d1e to da/d24 0 2024-09-16T08:57:05.171 INFO:tasks.workunit.client.1.smithi097.stdout:9/138: dread d2/f6 [0,4194304] 0 2024-09-16T08:57:05.171 INFO:tasks.workunit.client.1.smithi097.stdout:6/147: creat d1/d3/df/f2d x:0 0 0 2024-09-16T08:57:05.172 INFO:tasks.workunit.client.1.smithi097.stdout:9/139: unlink d2/l27 0 2024-09-16T08:57:05.175 INFO:tasks.workunit.client.1.smithi097.stdout:9/140: unlink d2/d5/f18 0 2024-09-16T08:57:05.175 INFO:tasks.workunit.client.1.smithi097.stdout:9/141: rename d2 to d2/d2d 22 2024-09-16T08:57:05.177 INFO:tasks.workunit.client.1.smithi097.stdout:9/142: truncate d2/fe 207995 0 2024-09-16T08:57:05.178 INFO:tasks.workunit.client.1.smithi097.stdout:9/143: mknod d2/c2e 0 2024-09-16T08:57:05.178 INFO:tasks.workunit.client.1.smithi097.stdout:9/144: creat d2/d5/f2f x:0 0 0 2024-09-16T08:57:05.182 INFO:tasks.workunit.client.1.smithi097.stdout:7/154: dwrite d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:05.185 INFO:tasks.workunit.client.1.smithi097.stdout:6/148: dread d1/d3/f1f [0,4194304] 0 2024-09-16T08:57:05.185 INFO:tasks.workunit.client.1.smithi097.stdout:6/149: creat d1/d3/d16/f2e x:0 0 0 2024-09-16T08:57:05.185 INFO:tasks.workunit.client.1.smithi097.stdout:6/150: creat d1/d3/f2f x:0 0 0 2024-09-16T08:57:05.188 INFO:tasks.workunit.client.1.smithi097.stdout:0/114: dwrite da/d24/f20 [0,4194304] 0 2024-09-16T08:57:05.188 INFO:tasks.workunit.client.1.smithi097.stdout:0/115: readlink da/df/l11 0 2024-09-16T08:57:05.189 INFO:tasks.workunit.client.1.smithi097.stdout:6/151: unlink d1/d3/f2b 0 2024-09-16T08:57:05.189 INFO:tasks.workunit.client.1.smithi097.stdout:5/186: truncate db/d1a/f1f 74075 0 2024-09-16T08:57:05.190 INFO:tasks.workunit.client.1.smithi097.stdout:0/116: read da/df/f16 [1355616,85559] 0 2024-09-16T08:57:05.192 INFO:tasks.workunit.client.1.smithi097.stdout:6/152: mknod d1/dc/d1e/c30 0 2024-09-16T08:57:05.194 INFO:tasks.workunit.client.1.smithi097.stdout:6/153: link d1/dc/d1e/c26 d1/d3/c31 0 2024-09-16T08:57:05.203 INFO:tasks.workunit.client.1.smithi097.stdout:0/117: dread da/d1a/f1c [0,4194304] 0 2024-09-16T08:57:05.204 INFO:tasks.workunit.client.1.smithi097.stdout:9/145: dwrite d2/d5/f21 [0,4194304] 0 2024-09-16T08:57:05.204 INFO:tasks.workunit.client.1.smithi097.stdout:9/146: stat d2/l15 0 2024-09-16T08:57:05.205 INFO:tasks.workunit.client.1.smithi097.stdout:9/147: mkdir d2/d30 0 2024-09-16T08:57:05.215 INFO:tasks.workunit.client.1.smithi097.stdout:0/118: write f1 [3408467,40422] 0 2024-09-16T08:57:05.216 INFO:tasks.workunit.client.1.smithi097.stdout:0/119: rename da/d1a/c23 to da/c25 0 2024-09-16T08:57:05.216 INFO:tasks.workunit.client.1.smithi097.stdout:0/120: truncate da/d1a/f22 883558 0 2024-09-16T08:57:05.218 INFO:tasks.workunit.client.1.smithi097.stdout:0/121: dread da/d1a/f22 [0,4194304] 0 2024-09-16T08:57:05.218 INFO:tasks.workunit.client.1.smithi097.stdout:9/148: write d2/f4 [2012887,89774] 0 2024-09-16T08:57:05.219 INFO:tasks.workunit.client.1.smithi097.stdout:9/149: dread - d2/d5/fb zero size 2024-09-16T08:57:05.219 INFO:tasks.workunit.client.1.smithi097.stdout:9/150: chown d2/c1f 1 1 2024-09-16T08:57:05.219 INFO:tasks.workunit.client.1.smithi097.stdout:9/151: creat d2/f31 x:0 0 0 2024-09-16T08:57:05.219 INFO:tasks.workunit.client.1.smithi097.stdout:0/122: mkdir da/d1a/d26 0 2024-09-16T08:57:05.221 INFO:tasks.workunit.client.1.smithi097.stdout:5/187: dwrite db/f14 [0,4194304] 0 2024-09-16T08:57:05.221 INFO:tasks.workunit.client.1.smithi097.stdout:5/188: dread - db/d1a/f38 zero size 2024-09-16T08:57:05.221 INFO:tasks.workunit.client.1.smithi097.stdout:9/152: creat d2/d30/f32 x:0 0 0 2024-09-16T08:57:05.221 INFO:tasks.workunit.client.1.smithi097.stdout:5/189: creat db/d18/f3d x:0 0 0 2024-09-16T08:57:05.221 INFO:tasks.workunit.client.1.smithi097.stdout:5/190: mknod db/d18/d1c/c3e 0 2024-09-16T08:57:05.221 INFO:tasks.workunit.client.1.smithi097.stdout:5/191: chown db/d18 29189910 1 2024-09-16T08:57:05.221 INFO:tasks.workunit.client.1.smithi097.stdout:5/192: chown db/d18/d1c/l2a 680007 1 2024-09-16T08:57:05.223 INFO:tasks.workunit.client.1.smithi097.stdout:7/155: dwrite d4/d7/f11 [0,4194304] 0 2024-09-16T08:57:05.224 INFO:tasks.workunit.client.1.smithi097.stdout:6/154: dwrite d1/d3/f11 [0,4194304] 0 2024-09-16T08:57:05.224 INFO:tasks.workunit.client.1.smithi097.stdout:6/155: chown d1/d3/df/c2a 644 1 2024-09-16T08:57:05.224 INFO:tasks.workunit.client.1.smithi097.stdout:6/156: creat d1/d3/df/f32 x:0 0 0 2024-09-16T08:57:05.225 INFO:tasks.workunit.client.1.smithi097.stdout:6/157: stat d1/c9 0 2024-09-16T08:57:05.229 INFO:tasks.workunit.client.1.smithi097.stdout:0/123: dread f8 [0,4194304] 0 2024-09-16T08:57:05.234 INFO:tasks.workunit.client.1.smithi097.stdout:0/124: read da/f15 [714114,66374] 0 2024-09-16T08:57:05.234 INFO:tasks.workunit.client.1.smithi097.stdout:0/125: chown da 47025665 1 2024-09-16T08:57:05.234 INFO:tasks.workunit.client.1.smithi097.stdout:0/126: truncate da/f1b 949777 0 2024-09-16T08:57:05.234 INFO:tasks.workunit.client.1.smithi097.stdout:0/127: fsync f6 0 2024-09-16T08:57:05.234 INFO:tasks.workunit.client.1.smithi097.stdout:0/128: creat da/f27 x:0 0 0 2024-09-16T08:57:05.235 INFO:tasks.workunit.client.1.smithi097.stdout:6/158: symlink d1/dc/d1e/l33 0 2024-09-16T08:57:05.235 INFO:tasks.workunit.client.1.smithi097.stdout:6/159: read - d1/d3/df/f2d zero size 2024-09-16T08:57:05.236 INFO:tasks.workunit.client.1.smithi097.stdout:6/160: stat d1/dc/d1e/c26 0 2024-09-16T08:57:05.237 INFO:tasks.workunit.client.1.smithi097.stdout:6/161: chown d1/dc/c29 42 1 2024-09-16T08:57:05.238 INFO:tasks.workunit.client.1.smithi097.stdout:7/156: dread d4/d9/dc/f16 [0,4194304] 0 2024-09-16T08:57:05.238 INFO:tasks.workunit.client.1.smithi097.stdout:7/157: chown d4/d7 428 1 2024-09-16T08:57:05.238 INFO:tasks.workunit.client.1.smithi097.stdout:7/158: readlink d4/d12/l39 0 2024-09-16T08:57:05.239 INFO:tasks.workunit.client.1.smithi097.stdout:6/162: rename d1/d3/f1f to d1/d3/f34 0 2024-09-16T08:57:05.239 INFO:tasks.workunit.client.1.smithi097.stdout:6/163: chown d1/d3 6594925 1 2024-09-16T08:57:05.239 INFO:tasks.workunit.client.1.smithi097.stdout:6/164: stat d1/dc/d1e/l33 0 2024-09-16T08:57:05.239 INFO:tasks.workunit.client.1.smithi097.stdout:6/165: truncate d1/d3/f7 4224878 0 2024-09-16T08:57:05.241 INFO:tasks.workunit.client.1.smithi097.stdout:7/159: rename d4/d9/dc/c37 to d4/d9/d2b/c3c 0 2024-09-16T08:57:05.241 INFO:tasks.workunit.client.1.smithi097.stdout:7/160: chown d4/d9/d1c/f28 146 1 2024-09-16T08:57:05.241 INFO:tasks.workunit.client.1.smithi097.stdout:7/161: fdatasync d4/d12/d1f/f3b 0 2024-09-16T08:57:05.243 INFO:tasks.workunit.client.1.smithi097.stdout:7/162: rename d4/f29 to d4/d12/d1f/d22/f3d 0 2024-09-16T08:57:05.243 INFO:tasks.workunit.client.1.smithi097.stdout:7/163: write d4/f17 [595291,29656] 0 2024-09-16T08:57:05.247 INFO:tasks.workunit.client.1.smithi097.stdout:9/153: dwrite d2/f2a [0,4194304] 0 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:5/193: dwrite db/f32 [4194304,4194304] 0 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:3/80: sync 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:4/180: sync 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:4/181: write d0/f2f [1013471,34773] 0 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:1/178: sync 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:8/132: sync 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:2/135: sync 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:2/136: truncate d3/d4/dd/f1f 257571 0 2024-09-16T08:57:05.251 INFO:tasks.workunit.client.1.smithi097.stdout:1/179: write d5/db/dc/d19/f29 [1040500,106206] 0 2024-09-16T08:57:05.252 INFO:tasks.workunit.client.1.smithi097.stdout:0/129: dread f1 [0,4194304] 0 2024-09-16T08:57:05.252 INFO:tasks.workunit.client.1.smithi097.stdout:0/130: chown da/df/d1f 1543199 1 2024-09-16T08:57:05.252 INFO:tasks.workunit.client.1.smithi097.stdout:0/131: fsync f8 0 2024-09-16T08:57:05.253 INFO:tasks.workunit.client.1.smithi097.stdout:5/194: link db/d18/d1c/l2a db/d18/d20/l3f 0 2024-09-16T08:57:05.255 INFO:tasks.workunit.client.1.smithi097.stdout:4/182: rename d0/d7/f18 to d0/d7/df/d15/f41 0 2024-09-16T08:57:05.255 INFO:tasks.workunit.client.1.smithi097.stdout:4/183: fdatasync d0/d7/df/f27 0 2024-09-16T08:57:05.255 INFO:tasks.workunit.client.1.smithi097.stdout:4/184: rename d0/d7/df to d0/d7/df/d15/d42 22 2024-09-16T08:57:05.256 INFO:tasks.workunit.client.1.smithi097.stdout:8/133: symlink d8/dd/l2b 0 2024-09-16T08:57:05.256 INFO:tasks.workunit.client.1.smithi097.stdout:8/134: stat d8 0 2024-09-16T08:57:05.257 INFO:tasks.workunit.client.1.smithi097.stdout:7/164: dread d4/fb [0,4194304] 0 2024-09-16T08:57:05.257 INFO:tasks.workunit.client.1.smithi097.stdout:1/180: creat d5/f35 x:0 0 0 2024-09-16T08:57:05.257 INFO:tasks.workunit.client.1.smithi097.stdout:0/132: link da/df/d1f/c21 da/df/c28 0 2024-09-16T08:57:05.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: mgrmap e22: smithi097.mpashf(active, since 1.2239s) 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: Checking for cert/key for grafana.smithi031 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: Migrating grafana.smithi031 cert to cert store 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: Migrating grafana.smithi031 key to cert store 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:57:05.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:57:05.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:57:05.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/trash_purge_schedule"}]: dispatch 2024-09-16T08:57:05.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:04 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/trash_purge_schedule"}]: dispatch 2024-09-16T08:57:05.263 INFO:tasks.workunit.client.1.smithi097.stdout:3/81: rmdir d7 39 2024-09-16T08:57:05.263 INFO:tasks.workunit.client.1.smithi097.stdout:3/82: write d7/fa [3818505,96942] 0 2024-09-16T08:57:05.264 INFO:tasks.workunit.client.1.smithi097.stdout:3/83: truncate d7/de/ff 1289349 0 2024-09-16T08:57:05.264 INFO:tasks.workunit.client.1.smithi097.stdout:3/84: write d7/d15/f17 [714662,94027] 0 2024-09-16T08:57:05.264 INFO:tasks.workunit.client.1.smithi097.stdout:4/185: mknod d0/d23/c43 0 2024-09-16T08:57:05.265 INFO:tasks.workunit.client.1.smithi097.stdout:7/165: truncate d4/d7/d19/f33 7041002 0 2024-09-16T08:57:05.265 INFO:tasks.workunit.client.1.smithi097.stdout:7/166: fdatasync f2 0 2024-09-16T08:57:05.266 INFO:tasks.workunit.client.1.smithi097.stdout:1/181: mknod d5/db/c36 0 2024-09-16T08:57:05.267 INFO:tasks.workunit.client.1.smithi097.stdout:1/182: creat d5/f37 x:0 0 0 2024-09-16T08:57:05.267 INFO:tasks.workunit.client.1.smithi097.stdout:1/183: creat d5/db/dc/d19/f38 x:0 0 0 2024-09-16T08:57:05.268 INFO:tasks.workunit.client.1.smithi097.stdout:2/137: dread d3/d4/d9/f20 [0,4194304] 0 2024-09-16T08:57:05.268 INFO:tasks.workunit.client.1.smithi097.stdout:6/166: dwrite d1/f22 [0,4194304] 0 2024-09-16T08:57:05.268 INFO:tasks.workunit.client.1.smithi097.stdout:6/167: fsync d1/d3/d16/f21 0 2024-09-16T08:57:05.269 INFO:tasks.workunit.client.1.smithi097.stdout:6/168: chown d1/d3/df/c18 2548 1 2024-09-16T08:57:05.269 INFO:tasks.workunit.client.1.smithi097.stdout:6/169: stat d1/d3/d16/f2e 0 2024-09-16T08:57:05.269 INFO:tasks.workunit.client.1.smithi097.stdout:6/170: write d1/d3/df/f32 [929388,52449] 0 2024-09-16T08:57:05.269 INFO:tasks.workunit.client.1.smithi097.stdout:6/171: readlink d1/dc/d1e/l25 0 2024-09-16T08:57:05.270 INFO:tasks.workunit.client.1.smithi097.stdout:4/186: creat d0/d7/df/d15/d16/d26/d3d/f44 x:0 0 0 2024-09-16T08:57:05.271 INFO:tasks.workunit.client.1.smithi097.stdout:8/135: rmdir d8/dd/d15 39 2024-09-16T08:57:05.275 INFO:tasks.workunit.client.1.smithi097.stdout:1/184: dread d5/db/f23 [0,4194304] 0 2024-09-16T08:57:05.279 INFO:tasks.workunit.client.1.smithi097.stdout:9/154: dwrite d2/d5/fb [0,4194304] 0 2024-09-16T08:57:05.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/85: mknod d7/c18 0 2024-09-16T08:57:05.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/86: fdatasync d7/de/ff 0 2024-09-16T08:57:05.281 INFO:tasks.workunit.client.1.smithi097.stdout:2/138: getdents d3/d4/d8 0 2024-09-16T08:57:05.282 INFO:tasks.workunit.client.1.smithi097.stdout:5/195: dwrite db/f14 [0,4194304] 0 2024-09-16T08:57:05.283 INFO:tasks.workunit.client.1.smithi097.stdout:7/167: dread d4/d7/ff [0,4194304] 0 2024-09-16T08:57:05.283 INFO:tasks.workunit.client.1.smithi097.stdout:4/187: mknod d0/d7/df/d15/c45 0 2024-09-16T08:57:05.284 INFO:tasks.workunit.client.1.smithi097.stdout:8/136: mkdir d8/dd/d15/d2c 0 2024-09-16T08:57:05.284 INFO:tasks.workunit.client.1.smithi097.stdout:1/185: symlink d5/db/dc/l39 0 2024-09-16T08:57:05.284 INFO:tasks.workunit.client.1.smithi097.stdout:1/186: write d5/d17/f26 [4220535,117587] 0 2024-09-16T08:57:05.288 INFO:tasks.workunit.client.1.smithi097.stdout:2/139: creat d3/d4/f21 x:0 0 0 2024-09-16T08:57:05.290 INFO:tasks.workunit.client.1.smithi097.stdout:6/172: dwrite d1/d3/f34 [0,4194304] 0 2024-09-16T08:57:05.291 INFO:tasks.workunit.client.1.smithi097.stdout:5/196: creat db/f40 x:0 0 0 2024-09-16T08:57:05.291 INFO:tasks.workunit.client.1.smithi097.stdout:7/168: dread d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:05.291 INFO:tasks.workunit.client.1.smithi097.stdout:7/169: creat d4/d9/d2b/f3e x:0 0 0 2024-09-16T08:57:05.291 INFO:tasks.workunit.client.1.smithi097.stdout:7/170: truncate d4/d12/d1f/f21 1169128 0 2024-09-16T08:57:05.300 INFO:tasks.workunit.client.1.smithi097.stdout:4/188: rename d0/d7/df/d15/d16/d26/c31 to d0/d7/df/d15/d16/d26/c46 0 2024-09-16T08:57:05.301 INFO:tasks.workunit.client.1.smithi097.stdout:8/137: truncate d8/dd/f19 2101558 0 2024-09-16T08:57:05.301 INFO:tasks.workunit.client.1.smithi097.stdout:8/138: stat d8/l2a 0 2024-09-16T08:57:05.302 INFO:tasks.workunit.client.1.smithi097.stdout:1/187: symlink d5/db/dc/d19/l3a 0 2024-09-16T08:57:05.305 INFO:tasks.workunit.client.1.smithi097.stdout:9/155: rmdir d2 39 2024-09-16T08:57:05.306 INFO:tasks.workunit.client.1.smithi097.stdout:9/156: stat f0 0 2024-09-16T08:57:05.308 INFO:tasks.workunit.client.1.smithi097.stdout:3/87: rename d7/de/ff to d7/f19 0 2024-09-16T08:57:05.309 INFO:tasks.workunit.client.1.smithi097.stdout:2/140: truncate d3/d4/d9/f12 1745682 0 2024-09-16T08:57:05.310 INFO:tasks.workunit.client.1.smithi097.stdout:6/173: mkdir d1/dc/d35 0 2024-09-16T08:57:05.311 INFO:tasks.workunit.client.1.smithi097.stdout:6/174: creat d1/d3/df/f36 x:0 0 0 2024-09-16T08:57:05.311 INFO:tasks.workunit.client.1.smithi097.stdout:6/175: dread - d1/d3/f2c zero size 2024-09-16T08:57:05.311 INFO:tasks.workunit.client.1.smithi097.stdout:6/176: write d1/d3/f2f [758316,129225] 0 2024-09-16T08:57:05.311 INFO:tasks.workunit.client.1.smithi097.stdout:6/177: dread - d1/d3/df/f36 zero size 2024-09-16T08:57:05.311 INFO:tasks.workunit.client.1.smithi097.stdout:5/197: creat db/d1a/f41 x:0 0 0 2024-09-16T08:57:05.314 INFO:tasks.workunit.client.1.smithi097.stdout:7/171: symlink d4/d9/d2b/l3f 0 2024-09-16T08:57:05.314 INFO:tasks.workunit.client.1.smithi097.stdout:7/172: write d4/d9/d2b/f3e [31303,107801] 0 2024-09-16T08:57:05.316 INFO:tasks.workunit.client.1.smithi097.stdout:4/189: creat d0/d36/f47 x:0 0 0 2024-09-16T08:57:05.318 INFO:tasks.workunit.client.1.smithi097.stdout:9/157: symlink d2/d5/l33 0 2024-09-16T08:57:05.318 INFO:tasks.workunit.client.1.smithi097.stdout:2/141: dread d3/d4/d9/ff [4194304,4194304] 0 2024-09-16T08:57:05.319 INFO:tasks.workunit.client.1.smithi097.stdout:2/142: readlink d3/d4/le 0 2024-09-16T08:57:05.319 INFO:tasks.workunit.client.1.smithi097.stdout:2/143: fdatasync d3/d4/d8/f16 0 2024-09-16T08:57:05.319 INFO:tasks.workunit.client.1.smithi097.stdout:3/88: symlink d7/de/l1a 0 2024-09-16T08:57:05.319 INFO:tasks.workunit.client.1.smithi097.stdout:3/89: truncate d7/f19 1320174 0 2024-09-16T08:57:05.319 INFO:tasks.workunit.client.1.smithi097.stdout:3/90: chown d7 321804401 1 2024-09-16T08:57:05.320 INFO:tasks.workunit.client.1.smithi097.stdout:3/91: chown d7/c18 6088 1 2024-09-16T08:57:05.322 INFO:tasks.workunit.client.1.smithi097.stdout:6/178: getdents d1/dc/d1e 0 2024-09-16T08:57:05.323 INFO:tasks.workunit.client.1.smithi097.stdout:4/190: symlink d0/d7/l48 0 2024-09-16T08:57:05.323 INFO:tasks.workunit.client.1.smithi097.stdout:9/158: mknod d2/c34 0 2024-09-16T08:57:05.323 INFO:tasks.workunit.client.1.smithi097.stdout:5/198: dread fa [0,4194304] 0 2024-09-16T08:57:05.323 INFO:tasks.workunit.client.1.smithi097.stdout:4/191: mknod d0/d7/df/d15/d16/d29/c49 0 2024-09-16T08:57:05.324 INFO:tasks.workunit.client.1.smithi097.stdout:9/159: creat d2/d5/f35 x:0 0 0 2024-09-16T08:57:05.326 INFO:tasks.workunit.client.1.smithi097.stdout:5/199: mkdir db/d18/d1c/d42 0 2024-09-16T08:57:05.326 INFO:tasks.workunit.client.1.smithi097.stdout:5/200: truncate db/d1a/f38 428834 0 2024-09-16T08:57:05.328 INFO:tasks.workunit.client.1.smithi097.stdout:5/201: dread db/f2e [0,4194304] 0 2024-09-16T08:57:05.330 INFO:tasks.workunit.client.1.smithi097.stdout:5/202: rmdir db/d18/d20/d2b 39 2024-09-16T08:57:05.331 INFO:tasks.workunit.client.1.smithi097.stdout:2/144: dread d3/d4/d8/f16 [0,4194304] 0 2024-09-16T08:57:05.331 INFO:tasks.workunit.client.1.smithi097.stdout:5/203: rename db/d18/d1c/l23 to db/d22/l43 0 2024-09-16T08:57:05.333 INFO:tasks.workunit.client.1.smithi097.stdout:2/145: rmdir d3/d4/d8 39 2024-09-16T08:57:05.333 INFO:tasks.workunit.client.1.smithi097.stdout:5/204: symlink db/d18/d1c/l44 0 2024-09-16T08:57:05.335 INFO:tasks.workunit.client.1.smithi097.stdout:5/205: mknod db/c45 0 2024-09-16T08:57:05.337 INFO:tasks.workunit.client.1.smithi097.stdout:5/206: mkdir db/d18/d1c/d42/d46 0 2024-09-16T08:57:05.338 INFO:tasks.workunit.client.1.smithi097.stdout:5/207: creat db/d18/d1c/d42/d46/f47 x:0 0 0 2024-09-16T08:57:05.339 INFO:tasks.workunit.client.1.smithi097.stdout:5/208: mkdir db/d18/d1c/d42/d48 0 2024-09-16T08:57:05.347 INFO:tasks.workunit.client.1.smithi097.stdout:1/188: dwrite d5/db/dc/d19/f29 [0,4194304] 0 2024-09-16T08:57:05.347 INFO:tasks.workunit.client.1.smithi097.stdout:1/189: stat d5 0 2024-09-16T08:57:05.347 INFO:tasks.workunit.client.1.smithi097.stdout:1/190: fdatasync d5/db/dc/d19/f38 0 2024-09-16T08:57:05.365 INFO:tasks.workunit.client.1.smithi097.stdout:7/173: dwrite d4/d9/d1c/f34 [0,4194304] 0 2024-09-16T08:57:05.367 INFO:tasks.workunit.client.1.smithi097.stdout:7/174: creat d4/d1b/d36/f40 x:0 0 0 2024-09-16T08:57:05.367 INFO:tasks.workunit.client.1.smithi097.stdout:7/175: truncate d4/d9/d2b/f3e 750590 0 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: mgrmap e22: smithi097.mpashf(active, since 1.2239s) 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: Checking for cert/key for grafana.smithi031 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: Migrating grafana.smithi031 cert to cert store 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: Migrating grafana.smithi031 key to cert store 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:05.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:57:05.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:57:05.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:57:05.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/trash_purge_schedule"}]: dispatch 2024-09-16T08:57:05.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:04 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi097.mpashf/trash_purge_schedule"}]: dispatch 2024-09-16T08:57:05.375 INFO:tasks.workunit.client.1.smithi097.stdout:6/179: dwrite d1/d3/f2c [0,4194304] 0 2024-09-16T08:57:05.375 INFO:tasks.workunit.client.1.smithi097.stdout:6/180: dread - d1/d3/d16/f2e zero size 2024-09-16T08:57:05.375 INFO:tasks.workunit.client.1.smithi097.stdout:6/181: creat d1/f37 x:0 0 0 2024-09-16T08:57:05.375 INFO:tasks.workunit.client.1.smithi097.stdout:6/182: dread - d1/d3/df/f2d zero size 2024-09-16T08:57:05.376 INFO:tasks.workunit.client.1.smithi097.stdout:3/92: dwrite d7/fa [0,4194304] 0 2024-09-16T08:57:05.376 INFO:tasks.workunit.client.1.smithi097.stdout:3/93: stat d7/de 0 2024-09-16T08:57:05.376 INFO:tasks.workunit.client.1.smithi097.stdout:3/94: stat d7/fa 0 2024-09-16T08:57:05.377 INFO:tasks.workunit.client.1.smithi097.stdout:2/146: dwrite d3/d4/d9/f12 [0,4194304] 0 2024-09-16T08:57:05.377 INFO:tasks.workunit.client.1.smithi097.stdout:5/209: dwrite db/d18/d20/d2b/f2d [0,4194304] 0 2024-09-16T08:57:05.378 INFO:tasks.workunit.client.1.smithi097.stdout:4/192: dwrite d0/f3e [0,4194304] 0 2024-09-16T08:57:05.378 INFO:tasks.workunit.client.1.smithi097.stdout:4/193: dread - d0/fd zero size 2024-09-16T08:57:05.378 INFO:tasks.workunit.client.1.smithi097.stdout:4/194: stat d0/d7/df/d15/d16 0 2024-09-16T08:57:05.378 INFO:tasks.workunit.client.1.smithi097.stdout:1/191: dwrite d5/d17/f32 [0,4194304] 0 2024-09-16T08:57:05.380 INFO:tasks.workunit.client.1.smithi097.stdout:7/176: write d4/f5 [3816432,7689] 0 2024-09-16T08:57:05.380 INFO:tasks.workunit.client.1.smithi097.stdout:3/95: write d7/f19 [158348,16515] 0 2024-09-16T08:57:05.382 INFO:tasks.workunit.client.0.smithi031.stderr:++ readlink -f fsstress 2024-09-16T08:57:05.382 INFO:tasks.workunit.client.1.smithi097.stdout:2/147: unlink d3/c13 0 2024-09-16T08:57:05.382 INFO:tasks.workunit.client.1.smithi097.stdout:2/148: write d3/d4/f21 [275373,80704] 0 2024-09-16T08:57:05.383 INFO:tasks.workunit.client.0.smithi031.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-16T08:57:05.383 INFO:tasks.workunit.client.0.smithi031.stderr:+ popd 2024-09-16T08:57:05.383 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-16T08:57:05.383 INFO:tasks.workunit.client.0.smithi031.stderr:+ popd 2024-09-16T08:57:05.384 INFO:tasks.workunit.client.0.smithi031.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-16T08:57:05.384 INFO:tasks.workunit.client.0.smithi031.stderr:++ mktemp -d -p . 2024-09-16T08:57:05.385 INFO:tasks.workunit.client.1.smithi097.stdout:6/183: write d1/d3/f7 [3962783,40906] 0 2024-09-16T08:57:05.385 INFO:tasks.workunit.client.1.smithi097.stdout:5/210: mknod db/d18/d1c/d42/d48/c49 0 2024-09-16T08:57:05.386 INFO:tasks.workunit.client.0.smithi031.stderr:+ T=./tmp.sDrlXt8OLN 2024-09-16T08:57:05.386 INFO:tasks.workunit.client.0.smithi031.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.sDrlXt8OLN -l 1 -n 1000 -p 10 -v 2024-09-16T08:57:05.389 INFO:tasks.workunit.client.1.smithi097.stdout:9/160: getdents d2/d5 0 2024-09-16T08:57:05.391 INFO:tasks.workunit.client.1.smithi097.stdout:2/149: dread d3/d4/d9/ff [4194304,4194304] 0 2024-09-16T08:57:05.391 INFO:tasks.workunit.client.0.smithi031.stdout:seed = 1726090137 2024-09-16T08:57:05.391 INFO:tasks.workunit.client.1.smithi097.stdout:1/192: write d5/d17/f32 [951426,12404] 0 2024-09-16T08:57:05.391 INFO:tasks.workunit.client.1.smithi097.stdout:1/193: chown d5/db/l22 382 1 2024-09-16T08:57:05.391 INFO:tasks.workunit.client.1.smithi097.stdout:1/194: fsync d5/db/f23 0 2024-09-16T08:57:05.391 INFO:tasks.workunit.client.1.smithi097.stdout:1/195: dread - d5/f35 zero size 2024-09-16T08:57:05.392 INFO:tasks.workunit.client.1.smithi097.stdout:4/195: symlink d0/d7/df/l4a 0 2024-09-16T08:57:05.392 INFO:tasks.workunit.client.1.smithi097.stdout:4/196: fsync d0/d7/f24 0 2024-09-16T08:57:05.394 INFO:tasks.workunit.client.1.smithi097.stdout:7/177: unlink d4/c6 0 2024-09-16T08:57:05.398 INFO:tasks.workunit.client.1.smithi097.stdout:3/96: symlink d7/d15/l1b 0 2024-09-16T08:57:05.399 INFO:tasks.workunit.client.0.smithi031.stdout:0/0: creat f0 x:0 0 0 2024-09-16T08:57:05.399 INFO:tasks.workunit.client.0.smithi031.stdout:0/1: dread - f0 zero size 2024-09-16T08:57:05.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/0: link - no file 2024-09-16T08:57:05.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/1: chown . 21749 1 2024-09-16T08:57:05.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/2: dread - no filename 2024-09-16T08:57:05.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/3: link - no file 2024-09-16T08:57:05.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/4: unlink - no file 2024-09-16T08:57:05.400 INFO:tasks.workunit.client.0.smithi031.stdout:2/5: dwrite - no filename 2024-09-16T08:57:05.401 INFO:tasks.workunit.client.0.smithi031.stdout:2/6: chown . 127644 1 2024-09-16T08:57:05.401 INFO:tasks.workunit.client.0.smithi031.stdout:2/7: fsync - no filename 2024-09-16T08:57:05.401 INFO:tasks.workunit.client.0.smithi031.stdout:1/0: creat f0 x:0 0 0 2024-09-16T08:57:05.402 INFO:tasks.workunit.client.0.smithi031.stdout:0/2: mknod c1 0 2024-09-16T08:57:05.404 INFO:tasks.workunit.client.0.smithi031.stdout:3/0: readlink - no filename 2024-09-16T08:57:05.404 INFO:tasks.workunit.client.0.smithi031.stdout:3/1: dwrite - no filename 2024-09-16T08:57:05.404 INFO:tasks.workunit.client.0.smithi031.stdout:3/2: getdents . 0 2024-09-16T08:57:05.404 INFO:tasks.workunit.client.0.smithi031.stdout:3/3: chown . 373125794 1 2024-09-16T08:57:05.404 INFO:tasks.workunit.client.0.smithi031.stdout:3/4: rmdir - no directory 2024-09-16T08:57:05.404 INFO:tasks.workunit.client.1.smithi097.stdout:6/184: creat d1/dc/f38 x:0 0 0 2024-09-16T08:57:05.405 INFO:tasks.workunit.client.0.smithi031.stdout:2/8: creat f0 x:0 0 0 2024-09-16T08:57:05.406 INFO:tasks.workunit.client.0.smithi031.stdout:1/1: mknod c1 0 2024-09-16T08:57:05.407 INFO:tasks.workunit.client.0.smithi031.stdout:1/2: dread - f0 zero size 2024-09-16T08:57:05.407 INFO:tasks.workunit.client.1.smithi097.stdout:9/161: link d2/d5/l33 d2/d5/l36 0 2024-09-16T08:57:05.408 INFO:tasks.workunit.client.0.smithi031.stdout:0/3: mknod c2 0 2024-09-16T08:57:05.408 INFO:tasks.workunit.client.0.smithi031.stdout:0/4: creat f3 x:0 0 0 2024-09-16T08:57:05.408 INFO:tasks.workunit.client.1.smithi097.stdout:9/162: read d2/f1a [162252,18650] 0 2024-09-16T08:57:05.409 INFO:tasks.workunit.client.0.smithi031.stdout:4/0: getdents . 0 2024-09-16T08:57:05.409 INFO:tasks.workunit.client.0.smithi031.stdout:4/1: getdents . 0 2024-09-16T08:57:05.409 INFO:tasks.workunit.client.0.smithi031.stdout:4/2: rmdir - no directory 2024-09-16T08:57:05.409 INFO:tasks.workunit.client.0.smithi031.stdout:4/3: rename - no filename 2024-09-16T08:57:05.410 INFO:tasks.workunit.client.1.smithi097.stdout:2/150: symlink d3/d4/d9/l22 0 2024-09-16T08:57:05.410 INFO:tasks.workunit.client.0.smithi031.stdout:3/5: mknod c0 0 2024-09-16T08:57:05.411 INFO:tasks.workunit.client.0.smithi031.stdout:2/9: rename f0 to f1 0 2024-09-16T08:57:05.413 INFO:tasks.workunit.client.1.smithi097.stdout:1/196: mkdir d5/d2c/d3b 0 2024-09-16T08:57:05.413 INFO:tasks.workunit.client.1.smithi097.stdout:1/197: chown d5/db/c36 979675 1 2024-09-16T08:57:05.414 INFO:tasks.workunit.client.0.smithi031.stdout:1/3: mkdir d2 0 2024-09-16T08:57:05.415 INFO:tasks.workunit.client.1.smithi097.stdout:4/197: creat d0/d7/df/d15/d16/f4b x:0 0 0 2024-09-16T08:57:05.415 INFO:tasks.workunit.client.1.smithi097.stdout:4/198: chown d0/f3e 3871950 1 2024-09-16T08:57:05.415 INFO:tasks.workunit.client.1.smithi097.stdout:4/199: fdatasync d0/d7/fb 0 2024-09-16T08:57:05.418 INFO:tasks.workunit.client.1.smithi097.stdout:7/178: getdents d4/d7/d19 0 2024-09-16T08:57:05.418 INFO:tasks.workunit.client.1.smithi097.stdout:7/179: fdatasync d4/d9/dc/f16 0 2024-09-16T08:57:05.422 INFO:tasks.workunit.client.0.smithi031.stdout:4/4: mkdir d0 0 2024-09-16T08:57:05.422 INFO:tasks.workunit.client.0.smithi031.stdout:4/5: fsync - no filename 2024-09-16T08:57:05.422 INFO:tasks.workunit.client.1.smithi097.stdout:6/185: rename d1/dc/l17 to d1/d3/d16/l39 0 2024-09-16T08:57:05.423 INFO:tasks.workunit.client.0.smithi031.stdout:3/6: creat f1 x:0 0 0 2024-09-16T08:57:05.423 INFO:tasks.workunit.client.0.smithi031.stdout:3/7: chown c0 4 1 2024-09-16T08:57:05.425 INFO:tasks.workunit.client.1.smithi097.stdout:9/163: mknod d2/c37 0 2024-09-16T08:57:05.425 INFO:tasks.workunit.client.1.smithi097.stdout:9/164: truncate d2/d5/f11 1946 0 2024-09-16T08:57:05.425 INFO:tasks.workunit.client.1.smithi097.stdout:9/165: write d2/d30/f32 [248211,124543] 0 2024-09-16T08:57:05.425 INFO:tasks.workunit.client.1.smithi097.stdout:7/180: dread d4/d1b/f1e [0,4194304] 0 2024-09-16T08:57:05.425 INFO:tasks.workunit.client.0.smithi031.stdout:1/4: creat d2/f3 x:0 0 0 2024-09-16T08:57:05.427 INFO:tasks.workunit.client.1.smithi097.stdout:1/198: unlink d5/d17/f32 0 2024-09-16T08:57:05.427 INFO:tasks.workunit.client.1.smithi097.stdout:1/199: fdatasync f4 0 2024-09-16T08:57:05.427 INFO:tasks.workunit.client.1.smithi097.stdout:1/200: chown d5/db/dc/d19/f38 2039 1 2024-09-16T08:57:05.427 INFO:tasks.workunit.client.1.smithi097.stdout:1/201: write d5/f9 [4711774,107699] 0 2024-09-16T08:57:05.427 INFO:tasks.workunit.client.0.smithi031.stdout:0/5: rename f3 to f4 0 2024-09-16T08:57:05.428 INFO:tasks.workunit.client.0.smithi031.stdout:0/6: dread - f4 zero size 2024-09-16T08:57:05.428 INFO:tasks.workunit.client.0.smithi031.stdout:0/7: write f0 [773044,13172] 0 2024-09-16T08:57:05.428 INFO:tasks.workunit.client.1.smithi097.stdout:4/200: mkdir d0/d36/d4c 0 2024-09-16T08:57:05.428 INFO:tasks.workunit.client.1.smithi097.stdout:4/201: dread - d0/d35/f38 zero size 2024-09-16T08:57:05.430 INFO:tasks.workunit.client.1.smithi097.stdout:2/151: dwrite d3/d4/dd/f1f [0,4194304] 0 2024-09-16T08:57:05.430 INFO:tasks.workunit.client.0.smithi031.stdout:2/10: dwrite f1 [0,4194304] 0 2024-09-16T08:57:05.431 INFO:tasks.workunit.client.0.smithi031.stdout:2/11: creat f2 x:0 0 0 2024-09-16T08:57:05.431 INFO:tasks.workunit.client.0.smithi031.stdout:2/12: creat f3 x:0 0 0 2024-09-16T08:57:05.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/0: truncate - no filename 2024-09-16T08:57:05.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/1: dread - no filename 2024-09-16T08:57:05.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/2: dwrite - no filename 2024-09-16T08:57:05.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/3: dwrite - no filename 2024-09-16T08:57:05.431 INFO:tasks.workunit.client.0.smithi031.stdout:6/4: chown . 2522 1 2024-09-16T08:57:05.433 INFO:tasks.workunit.client.0.smithi031.stdout:5/0: symlink l0 0 2024-09-16T08:57:05.435 INFO:tasks.workunit.client.1.smithi097.stdout:3/97: link d7/de/l10 d7/l1c 0 2024-09-16T08:57:05.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/6: creat d0/f1 x:0 0 0 2024-09-16T08:57:05.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/7: dread - d0/f1 zero size 2024-09-16T08:57:05.437 INFO:tasks.workunit.client.1.smithi097.stdout:6/186: creat d1/dc/d1e/f3a x:0 0 0 2024-09-16T08:57:05.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/8: mknod c2 0 2024-09-16T08:57:05.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/9: stat c2 0 2024-09-16T08:57:05.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/10: chown c2 162684 1 2024-09-16T08:57:05.439 INFO:tasks.workunit.client.1.smithi097.stdout:9/166: unlink d2/d5/c17 0 2024-09-16T08:57:05.439 INFO:tasks.workunit.client.0.smithi031.stdout:1/5: mkdir d2/d4 0 2024-09-16T08:57:05.440 INFO:tasks.workunit.client.1.smithi097.stdout:7/181: unlink d4/f17 0 2024-09-16T08:57:05.441 INFO:tasks.workunit.client.0.smithi031.stdout:0/8: mknod c5 0 2024-09-16T08:57:05.442 INFO:tasks.workunit.client.1.smithi097.stdout:1/202: creat d5/db/f3c x:0 0 0 2024-09-16T08:57:05.442 INFO:tasks.workunit.client.1.smithi097.stdout:1/203: chown d5/db/dc/l39 0 1 2024-09-16T08:57:05.443 INFO:tasks.workunit.client.1.smithi097.stdout:2/152: dwrite d3/d4/d9/ff [0,4194304] 0 2024-09-16T08:57:05.444 INFO:tasks.workunit.client.1.smithi097.stdout:2/153: dread - d3/d4/d9/f1d zero size 2024-09-16T08:57:05.444 INFO:tasks.workunit.client.1.smithi097.stdout:4/202: rename d0/l1 to d0/d7/l4d 0 2024-09-16T08:57:05.444 INFO:tasks.workunit.client.1.smithi097.stdout:4/203: write d0/d7/df/d15/d16/d26/d3d/f44 [1029882,17181] 0 2024-09-16T08:57:05.447 INFO:tasks.workunit.client.1.smithi097.stdout:0/133: sync 2024-09-16T08:57:05.454 INFO:tasks.workunit.client.1.smithi097.stdout:1/204: write d5/d17/f26 [3076660,187] 0 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/0: chown . 11391248 1 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/1: rename - no filename 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/2: rename - no filename 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/3: dwrite - no filename 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/4: fsync - no filename 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/5: stat - no entries 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/6: link - no file 2024-09-16T08:57:05.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/7: chown . 235552 1 2024-09-16T08:57:05.456 INFO:tasks.workunit.client.0.smithi031.stdout:7/8: chown . 4119707 1 2024-09-16T08:57:05.457 INFO:tasks.workunit.client.0.smithi031.stdout:6/5: mkdir d0 0 2024-09-16T08:57:05.458 INFO:tasks.workunit.client.1.smithi097.stdout:3/98: rmdir d7/de 39 2024-09-16T08:57:05.461 INFO:tasks.workunit.client.0.smithi031.stdout:5/1: creat f1 x:0 0 0 2024-09-16T08:57:05.461 INFO:tasks.workunit.client.0.smithi031.stdout:5/2: readlink l0 0 2024-09-16T08:57:05.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/187: link d1/c24 d1/d3/d16/c3b 0 2024-09-16T08:57:05.465 INFO:tasks.workunit.client.0.smithi031.stdout:4/8: mkdir d0/d2 0 2024-09-16T08:57:05.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/167: creat d2/f38 x:0 0 0 2024-09-16T08:57:05.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/168: chown d2/c37 242113648 1 2024-09-16T08:57:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:3/11: mkdir d3 0 2024-09-16T08:57:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:3/12: write f1 [488188,115491] 0 2024-09-16T08:57:05.475 INFO:tasks.workunit.client.0.smithi031.stdout:3/13: chown c2 183 1 2024-09-16T08:57:05.478 INFO:tasks.workunit.client.1.smithi097.stdout:7/182: mkdir d4/d1b/d41 0 2024-09-16T08:57:05.478 INFO:tasks.workunit.client.0.smithi031.stdout:1/6: creat d2/d4/f5 x:0 0 0 2024-09-16T08:57:05.483 INFO:tasks.workunit.client.1.smithi097.stdout:0/134: rename da/df/d1f/c21 to da/c29 0 2024-09-16T08:57:05.485 INFO:tasks.workunit.client.0.smithi031.stdout:0/9: mkdir d6 0 2024-09-16T08:57:05.486 INFO:tasks.workunit.client.0.smithi031.stdout:0/10: chown f4 2 1 2024-09-16T08:57:05.489 INFO:tasks.workunit.client.0.smithi031.stdout:2/13: rename f3 to f4 0 2024-09-16T08:57:05.489 INFO:tasks.workunit.client.0.smithi031.stdout:2/14: readlink - no filename 2024-09-16T08:57:05.492 INFO:tasks.workunit.client.1.smithi097.stdout:6/188: truncate d1/d3/f2f 400194 0 2024-09-16T08:57:05.492 INFO:tasks.workunit.client.1.smithi097.stdout:6/189: chown d1/d3/d16 17999 1 2024-09-16T08:57:05.492 INFO:tasks.workunit.client.1.smithi097.stdout:6/190: truncate d1/d3/df/f32 1263365 0 2024-09-16T08:57:05.495 INFO:tasks.workunit.client.1.smithi097.stdout:7/183: symlink d4/l42 0 2024-09-16T08:57:05.495 INFO:tasks.workunit.client.0.smithi031.stdout:7/9: symlink l0 0 2024-09-16T08:57:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:7/10: write - no filename 2024-09-16T08:57:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:5/3: mkdir d2 0 2024-09-16T08:57:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:5/4: dread - f1 zero size 2024-09-16T08:57:05.496 INFO:tasks.workunit.client.0.smithi031.stdout:5/5: rename d2 to d2/d3 22 2024-09-16T08:57:05.497 INFO:tasks.workunit.client.0.smithi031.stdout:5/6: write f1 [855557,60422] 0 2024-09-16T08:57:05.500 INFO:tasks.workunit.client.0.smithi031.stdout:4/9: symlink d0/l3 0 2024-09-16T08:57:05.502 INFO:tasks.workunit.client.1.smithi097.stdout:0/135: rename da/f27 to da/df/d1f/f2a 0 2024-09-16T08:57:05.503 INFO:tasks.workunit.client.1.smithi097.stdout:3/99: mkdir d7/d1d 0 2024-09-16T08:57:05.506 INFO:tasks.workunit.client.0.smithi031.stdout:7/11: mknod c1 0 2024-09-16T08:57:05.506 INFO:tasks.workunit.client.0.smithi031.stdout:7/12: dwrite - no filename 2024-09-16T08:57:05.508 INFO:tasks.workunit.client.1.smithi097.stdout:6/191: rmdir d1/dc/d35 0 2024-09-16T08:57:05.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/7: creat d2/f4 x:0 0 0 2024-09-16T08:57:05.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/8: creat d2/f5 x:0 0 0 2024-09-16T08:57:05.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/9: creat d2/f6 x:0 0 0 2024-09-16T08:57:05.510 INFO:tasks.workunit.client.1.smithi097.stdout:7/184: mkdir d4/d1b/d36/d43 0 2024-09-16T08:57:05.511 INFO:tasks.workunit.client.0.smithi031.stdout:4/10: mknod d0/d2/c4 0 2024-09-16T08:57:05.517 INFO:tasks.workunit.client.0.smithi031.stdout:7/13: mkdir d2 0 2024-09-16T08:57:05.517 INFO:tasks.workunit.client.0.smithi031.stdout:7/14: dwrite - no filename 2024-09-16T08:57:05.517 INFO:tasks.workunit.client.0.smithi031.stdout:7/15: dwrite - no filename 2024-09-16T08:57:05.523 INFO:tasks.workunit.client.0.smithi031.stdout:7/16: creat d2/f3 x:0 0 0 2024-09-16T08:57:05.525 INFO:tasks.workunit.client.1.smithi097.stdout:2/154: dwrite d3/d4/dd/f18 [0,4194304] 0 2024-09-16T08:57:05.526 INFO:tasks.workunit.client.1.smithi097.stdout:1/205: dwrite d5/db/f23 [4194304,4194304] 0 2024-09-16T08:57:05.526 INFO:tasks.workunit.client.1.smithi097.stdout:1/206: read - d5/db/dc/d19/f38 zero size 2024-09-16T08:57:05.528 INFO:tasks.workunit.client.0.smithi031.stdout:7/17: rename d2/f3 to d2/f4 0 2024-09-16T08:57:05.528 INFO:tasks.workunit.client.0.smithi031.stdout:7/18: stat l0 0 2024-09-16T08:57:05.529 INFO:tasks.workunit.client.0.smithi031.stdout:5/10: rename d2/f6 to d2/f7 0 2024-09-16T08:57:05.529 INFO:tasks.workunit.client.0.smithi031.stdout:5/11: truncate d2/f4 1045237 0 2024-09-16T08:57:05.530 INFO:tasks.workunit.client.1.smithi097.stdout:1/207: truncate d5/f21 635904 0 2024-09-16T08:57:05.530 INFO:tasks.workunit.client.1.smithi097.stdout:1/208: stat d5/db/dc/d19/c2b 0 2024-09-16T08:57:05.531 INFO:tasks.workunit.client.0.smithi031.stdout:7/19: mknod d2/c5 0 2024-09-16T08:57:05.531 INFO:tasks.workunit.client.0.smithi031.stdout:7/20: fdatasync d2/f4 0 2024-09-16T08:57:05.531 INFO:tasks.workunit.client.0.smithi031.stdout:7/21: fdatasync d2/f4 0 2024-09-16T08:57:05.531 INFO:tasks.workunit.client.0.smithi031.stdout:7/22: write d2/f4 [1039307,44528] 0 2024-09-16T08:57:05.531 INFO:tasks.workunit.client.0.smithi031.stdout:7/23: chown d2/f4 30 1 2024-09-16T08:57:05.534 INFO:tasks.workunit.client.0.smithi031.stdout:5/12: unlink l0 0 2024-09-16T08:57:05.534 INFO:tasks.workunit.client.0.smithi031.stdout:7/24: symlink d2/l6 0 2024-09-16T08:57:05.535 INFO:tasks.workunit.client.0.smithi031.stdout:7/25: write d2/f4 [2020343,100208] 0 2024-09-16T08:57:05.535 INFO:tasks.workunit.client.0.smithi031.stdout:7/26: readlink l0 0 2024-09-16T08:57:05.535 INFO:tasks.workunit.client.0.smithi031.stdout:7/27: chown d2/c5 1 1 2024-09-16T08:57:05.535 INFO:tasks.workunit.client.0.smithi031.stdout:7/28: stat d2/l6 0 2024-09-16T08:57:05.538 INFO:tasks.workunit.client.0.smithi031.stdout:7/29: mknod d2/c7 0 2024-09-16T08:57:05.539 INFO:tasks.workunit.client.0.smithi031.stdout:7/30: truncate d2/f4 2142022 0 2024-09-16T08:57:05.543 INFO:tasks.workunit.client.1.smithi097.stdout:1/209: rmdir d5 39 2024-09-16T08:57:05.551 INFO:tasks.workunit.client.0.smithi031.stdout:7/31: unlink c1 0 2024-09-16T08:57:05.553 INFO:tasks.workunit.client.1.smithi097.stdout:7/185: dread d4/d7/d19/f33 [0,4194304] 0 2024-09-16T08:57:05.565 INFO:tasks.workunit.client.1.smithi097.stdout:2/155: dread d3/d4/d8/f16 [0,4194304] 0 2024-09-16T08:57:05.565 INFO:tasks.workunit.client.1.smithi097.stdout:2/156: write d3/d4/d9/f1e [45492,69133] 0 2024-09-16T08:57:05.629 INFO:tasks.workunit.client.1.smithi097.stdout:6/192: dwrite d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:05.634 INFO:tasks.workunit.client.1.smithi097.stdout:6/193: mkdir d1/dc/d3c 0 2024-09-16T08:57:05.634 INFO:tasks.workunit.client.1.smithi097.stdout:6/194: write d1/dc/d1e/f3a [440152,105798] 0 2024-09-16T08:57:05.634 INFO:tasks.workunit.client.1.smithi097.stdout:6/195: dread - d1/d3/df/f36 zero size 2024-09-16T08:57:05.635 INFO:tasks.workunit.client.0.smithi031.stdout:3/14: dread f1 [0,4194304] 0 2024-09-16T08:57:05.638 INFO:tasks.workunit.client.0.smithi031.stdout:3/15: dread f1 [0,4194304] 0 2024-09-16T08:57:05.642 INFO:tasks.workunit.client.1.smithi097.stdout:6/196: rename d1/f1b to d1/d3/f3d 0 2024-09-16T08:57:05.651 INFO:tasks.workunit.client.1.smithi097.stdout:6/197: mkdir d1/d3/d3e 0 2024-09-16T08:57:05.661 INFO:tasks.workunit.client.1.smithi097.stdout:9/169: getdents d2/d5 0 2024-09-16T08:57:05.666 INFO:tasks.workunit.client.1.smithi097.stdout:6/198: symlink d1/dc/d1e/l3f 0 2024-09-16T08:57:05.669 INFO:tasks.workunit.client.0.smithi031.stdout:5/13: fsync d2/f4 0 2024-09-16T08:57:05.669 INFO:tasks.workunit.client.0.smithi031.stdout:5/14: dread - d2/f7 zero size 2024-09-16T08:57:05.671 INFO:tasks.workunit.client.1.smithi097.stdout:4/204: dwrite d0/d7/f24 [0,4194304] 0 2024-09-16T08:57:05.671 INFO:tasks.workunit.client.1.smithi097.stdout:9/170: rename d2/d5/lf to d2/d5/l39 0 2024-09-16T08:57:05.672 INFO:tasks.workunit.client.0.smithi031.stdout:3/16: dwrite f1 [0,4194304] 0 2024-09-16T08:57:05.674 INFO:tasks.workunit.client.1.smithi097.stdout:6/199: symlink d1/d3/d16/l40 0 2024-09-16T08:57:05.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/11: truncate f0 47164 0 2024-09-16T08:57:05.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/12: write f4 [815036,44466] 0 2024-09-16T08:57:05.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/13: readlink - no filename 2024-09-16T08:57:05.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/14: getdents d6 0 2024-09-16T08:57:05.686 INFO:tasks.workunit.client.1.smithi097.stdout:0/136: dwrite f8 [0,4194304] 0 2024-09-16T08:57:05.686 INFO:tasks.workunit.client.1.smithi097.stdout:2/157: dwrite d3/d4/dd/f1f [0,4194304] 0 2024-09-16T08:57:05.686 INFO:tasks.workunit.client.1.smithi097.stdout:0/137: creat da/d1a/f2b x:0 0 0 2024-09-16T08:57:05.686 INFO:tasks.workunit.client.1.smithi097.stdout:0/138: chown da 926 1 2024-09-16T08:57:05.689 INFO:tasks.workunit.client.0.smithi031.stdout:8/0: write - no filename 2024-09-16T08:57:05.700 INFO:tasks.workunit.client.0.smithi031.stdout:7/32: dread d2/f4 [0,4194304] 0 2024-09-16T08:57:05.705 INFO:tasks.workunit.client.1.smithi097.stdout:4/205: rename d0/d7/df/d15/d16/d26/c46 to d0/d7/c4e 0 2024-09-16T08:57:05.706 INFO:tasks.workunit.client.1.smithi097.stdout:6/200: dwrite d1/d3/f3d [0,4194304] 0 2024-09-16T08:57:05.706 INFO:tasks.workunit.client.1.smithi097.stdout:3/100: dwrite d7/d15/f17 [0,4194304] 0 2024-09-16T08:57:05.706 INFO:tasks.workunit.client.1.smithi097.stdout:3/101: getdents d7/d1d 0 2024-09-16T08:57:05.706 INFO:tasks.workunit.client.1.smithi097.stdout:9/171: mkdir d2/d5/d3a 0 2024-09-16T08:57:05.707 INFO:tasks.workunit.client.1.smithi097.stdout:4/206: dread d0/f2a [0,4194304] 0 2024-09-16T08:57:05.713 INFO:tasks.workunit.client.1.smithi097.stdout:1/210: write d5/f21 [361853,63514] 0 2024-09-16T08:57:05.713 INFO:tasks.workunit.client.1.smithi097.stdout:0/139: link da/c25 da/d24/c2c 0 2024-09-16T08:57:05.718 INFO:tasks.workunit.client.1.smithi097.stdout:6/201: mknod d1/dc/d3c/c41 0 2024-09-16T08:57:05.720 INFO:tasks.workunit.client.1.smithi097.stdout:1/211: dread d5/d17/f26 [0,4194304] 0 2024-09-16T08:57:05.720 INFO:tasks.workunit.client.1.smithi097.stdout:1/212: chown d5/db/dc/f33 343426 1 2024-09-16T08:57:05.722 INFO:tasks.workunit.client.1.smithi097.stdout:2/158: dwrite d3/d4/d9/ff [4194304,4194304] 0 2024-09-16T08:57:05.722 INFO:tasks.workunit.client.0.smithi031.stdout:7/33: dwrite d2/f4 [0,4194304] 0 2024-09-16T08:57:05.725 INFO:tasks.workunit.client.1.smithi097.stdout:3/102: creat d7/de/f1e x:0 0 0 2024-09-16T08:57:05.726 INFO:tasks.workunit.client.1.smithi097.stdout:9/172: creat d2/d30/f3b x:0 0 0 2024-09-16T08:57:05.726 INFO:tasks.workunit.client.1.smithi097.stdout:9/173: fsync d2/d5/f29 0 2024-09-16T08:57:05.727 INFO:tasks.workunit.client.1.smithi097.stdout:0/140: mknod da/df/c2d 0 2024-09-16T08:57:05.728 INFO:tasks.workunit.client.1.smithi097.stdout:0/141: write da/f1b [208006,91429] 0 2024-09-16T08:57:05.729 INFO:tasks.workunit.client.1.smithi097.stdout:6/202: link d1/c9 d1/d3/d16/c42 0 2024-09-16T08:57:05.729 INFO:tasks.workunit.client.1.smithi097.stdout:6/203: chown d1/d3/f11 2 1 2024-09-16T08:57:05.733 INFO:tasks.workunit.client.1.smithi097.stdout:3/103: mknod d7/d15/c1f 0 2024-09-16T08:57:05.735 INFO:tasks.workunit.client.1.smithi097.stdout:9/174: mkdir d2/d5/d3a/d3c 0 2024-09-16T08:57:05.735 INFO:tasks.workunit.client.1.smithi097.stdout:9/175: readlink d2/l10 0 2024-09-16T08:57:05.737 INFO:tasks.workunit.client.1.smithi097.stdout:6/204: symlink d1/d3/d3e/l43 0 2024-09-16T08:57:05.737 INFO:tasks.workunit.client.1.smithi097.stdout:6/205: write d1/dc/f38 [32928,126644] 0 2024-09-16T08:57:05.737 INFO:tasks.workunit.client.1.smithi097.stdout:6/206: chown d1/d3 461 1 2024-09-16T08:57:05.740 INFO:tasks.workunit.client.1.smithi097.stdout:9/176: creat d2/d5/d3a/f3d x:0 0 0 2024-09-16T08:57:05.740 INFO:tasks.workunit.client.1.smithi097.stdout:9/177: mknod d2/d5/c3e 0 2024-09-16T08:57:05.741 INFO:tasks.workunit.client.1.smithi097.stdout:2/159: dwrite d3/d4/dd/f18 [0,4194304] 0 2024-09-16T08:57:05.745 INFO:tasks.workunit.client.1.smithi097.stdout:0/142: dwrite da/d24/f20 [0,4194304] 0 2024-09-16T08:57:05.749 INFO:tasks.workunit.client.1.smithi097.stdout:3/104: write d7/d15/f17 [1402961,107959] 0 2024-09-16T08:57:05.749 INFO:tasks.workunit.client.1.smithi097.stdout:3/105: stat l5 0 2024-09-16T08:57:05.753 INFO:tasks.workunit.client.1.smithi097.stdout:3/106: mkdir d7/d15/d20 0 2024-09-16T08:57:05.756 INFO:tasks.workunit.client.1.smithi097.stdout:3/107: rename d7/f19 to d7/f21 0 2024-09-16T08:57:05.757 INFO:tasks.workunit.client.1.smithi097.stdout:3/108: chown d7/de 1103 1 2024-09-16T08:57:05.757 INFO:tasks.workunit.client.1.smithi097.stdout:0/143: dread da/d24/f20 [0,4194304] 0 2024-09-16T08:57:05.758 INFO:tasks.workunit.client.1.smithi097.stdout:9/178: dwrite d2/f38 [0,4194304] 0 2024-09-16T08:57:05.758 INFO:tasks.workunit.client.1.smithi097.stdout:9/179: fsync d2/d5/fa 0 2024-09-16T08:57:05.758 INFO:tasks.workunit.client.1.smithi097.stdout:9/180: fdatasync d2/d5/f29 0 2024-09-16T08:57:05.758 INFO:tasks.workunit.client.1.smithi097.stdout:9/181: chown d2/c24 14097 1 2024-09-16T08:57:05.759 INFO:tasks.workunit.client.1.smithi097.stdout:3/109: fsync d7/d15/f17 0 2024-09-16T08:57:05.781 INFO:tasks.workunit.client.1.smithi097.stdout:9/182: dwrite d2/f13 [0,4194304] 0 2024-09-16T08:57:05.781 INFO:tasks.workunit.client.1.smithi097.stdout:9/183: truncate d2/d5/fa 684222 0 2024-09-16T08:57:05.786 INFO:tasks.workunit.client.1.smithi097.stdout:9/184: mkdir d2/d3f 0 2024-09-16T08:57:05.786 INFO:tasks.workunit.client.1.smithi097.stdout:9/185: write d2/d30/f3b [952214,15649] 0 2024-09-16T08:57:05.801 INFO:tasks.workunit.client.1.smithi097.stdout:5/211: sync 2024-09-16T08:57:05.801 INFO:tasks.workunit.client.1.smithi097.stdout:8/139: sync 2024-09-16T08:57:05.801 INFO:tasks.workunit.client.1.smithi097.stdout:8/140: readlink d8/dd/d15/l20 0 2024-09-16T08:57:05.802 INFO:tasks.workunit.client.1.smithi097.stdout:5/212: symlink db/l4a 0 2024-09-16T08:57:05.804 INFO:tasks.workunit.client.1.smithi097.stdout:9/186: dwrite d2/f1b [0,4194304] 0 2024-09-16T08:57:05.809 INFO:tasks.workunit.client.0.smithi031.stdout:3/17: creat d3/f4 x:0 0 0 2024-09-16T08:57:05.814 INFO:tasks.workunit.client.1.smithi097.stdout:9/187: unlink d2/f1b 0 2024-09-16T08:57:05.817 INFO:tasks.workunit.client.1.smithi097.stdout:0/144: fsync da/d1a/f2b 0 2024-09-16T08:57:05.820 INFO:tasks.workunit.client.1.smithi097.stdout:0/145: mknod da/d1a/d26/c2e 0 2024-09-16T08:57:05.821 INFO:tasks.workunit.client.0.smithi031.stdout:0/15: link f4 d6/f7 0 2024-09-16T08:57:05.821 INFO:tasks.workunit.client.0.smithi031.stdout:0/16: fdatasync f4 0 2024-09-16T08:57:05.821 INFO:tasks.workunit.client.0.smithi031.stdout:0/17: write f0 [50187,72497] 0 2024-09-16T08:57:05.823 INFO:tasks.workunit.client.1.smithi097.stdout:0/146: unlink c2 0 2024-09-16T08:57:05.826 INFO:tasks.workunit.client.1.smithi097.stdout:0/147: creat da/df/d13/f2f x:0 0 0 2024-09-16T08:57:05.826 INFO:tasks.workunit.client.1.smithi097.stdout:0/148: fdatasync f8 0 2024-09-16T08:57:05.827 INFO:tasks.workunit.client.0.smithi031.stdout:1/7: symlink d2/d4/l6 0 2024-09-16T08:57:05.828 INFO:tasks.workunit.client.1.smithi097.stdout:9/188: write d2/fc [388075,6295] 0 2024-09-16T08:57:05.828 INFO:tasks.workunit.client.1.smithi097.stdout:9/189: stat d2/l23 0 2024-09-16T08:57:05.828 INFO:tasks.workunit.client.0.smithi031.stdout:8/1: mknod c0 0 2024-09-16T08:57:05.828 INFO:tasks.workunit.client.0.smithi031.stdout:8/2: dread - no filename 2024-09-16T08:57:05.829 INFO:tasks.workunit.client.1.smithi097.stdout:0/149: link da/df/c28 da/df/c30 0 2024-09-16T08:57:05.829 INFO:tasks.workunit.client.1.smithi097.stdout:0/150: readlink l3 0 2024-09-16T08:57:05.831 INFO:tasks.workunit.client.1.smithi097.stdout:5/213: dwrite db/d18/d20/f29 [0,4194304] 0 2024-09-16T08:57:05.832 INFO:tasks.workunit.client.1.smithi097.stdout:5/214: chown c7 750155 1 2024-09-16T08:57:05.832 INFO:tasks.workunit.client.1.smithi097.stdout:8/141: dwrite d8/dd/d15/f29 [0,4194304] 0 2024-09-16T08:57:05.833 INFO:tasks.workunit.client.0.smithi031.stdout:7/34: mkdir d2/d8 0 2024-09-16T08:57:05.833 INFO:tasks.workunit.client.0.smithi031.stdout:7/35: fsync d2/f4 0 2024-09-16T08:57:05.833 INFO:tasks.workunit.client.0.smithi031.stdout:5/15: rmdir d2 39 2024-09-16T08:57:05.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/16: write d2/f7 [569833,12241] 0 2024-09-16T08:57:05.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/17: chown d2/f7 4170 1 2024-09-16T08:57:05.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/18: fsync d2/f5 0 2024-09-16T08:57:05.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/19: write d2/f7 [987011,45627] 0 2024-09-16T08:57:05.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/20: readlink - no filename 2024-09-16T08:57:05.835 INFO:tasks.workunit.client.1.smithi097.stdout:5/215: getdents db/d22 0 2024-09-16T08:57:05.840 INFO:tasks.workunit.client.1.smithi097.stdout:5/216: creat db/d22/f4b x:0 0 0 2024-09-16T08:57:05.841 INFO:tasks.workunit.client.0.smithi031.stdout:1/8: link d2/d4/l6 d2/d4/l7 0 2024-09-16T08:57:05.842 INFO:tasks.workunit.client.1.smithi097.stdout:2/160: truncate d3/d4/d9/ff 6726511 0 2024-09-16T08:57:05.842 INFO:tasks.workunit.client.1.smithi097.stdout:5/217: dread db/d1a/f1f [0,4194304] 0 2024-09-16T08:57:05.846 INFO:tasks.workunit.client.0.smithi031.stdout:7/36: rename d2/l6 to d2/l9 0 2024-09-16T08:57:05.846 INFO:tasks.workunit.client.0.smithi031.stdout:7/37: stat l0 0 2024-09-16T08:57:05.846 INFO:tasks.workunit.client.0.smithi031.stdout:7/38: chown d2/d8 108945275 1 2024-09-16T08:57:05.846 INFO:tasks.workunit.client.0.smithi031.stdout:7/39: chown l0 69597 1 2024-09-16T08:57:05.848 INFO:tasks.workunit.client.0.smithi031.stdout:5/21: creat d2/f8 x:0 0 0 2024-09-16T08:57:05.848 INFO:tasks.workunit.client.0.smithi031.stdout:5/22: readlink - no filename 2024-09-16T08:57:05.851 INFO:tasks.workunit.client.0.smithi031.stdout:1/9: symlink d2/d4/l8 0 2024-09-16T08:57:05.853 INFO:tasks.workunit.client.0.smithi031.stdout:5/23: symlink d2/l9 0 2024-09-16T08:57:05.857 INFO:tasks.workunit.client.0.smithi031.stdout:5/24: unlink d2/f4 0 2024-09-16T08:57:05.860 INFO:tasks.workunit.client.0.smithi031.stdout:5/25: unlink f1 0 2024-09-16T08:57:05.860 INFO:tasks.workunit.client.0.smithi031.stdout:5/26: truncate d2/f7 1773904 0 2024-09-16T08:57:05.860 INFO:tasks.workunit.client.1.smithi097.stdout:5/218: dread fa [0,4194304] 0 2024-09-16T08:57:05.860 INFO:tasks.workunit.client.1.smithi097.stdout:5/219: write db/d1a/f41 [902573,54542] 0 2024-09-16T08:57:05.864 INFO:tasks.workunit.client.0.smithi031.stdout:0/18: dwrite d6/f7 [0,4194304] 0 2024-09-16T08:57:05.866 INFO:tasks.workunit.client.0.smithi031.stdout:0/19: link f0 d6/f8 0 2024-09-16T08:57:05.866 INFO:tasks.workunit.client.0.smithi031.stdout:7/40: write d2/f4 [2344966,88595] 0 2024-09-16T08:57:05.868 INFO:tasks.workunit.client.0.smithi031.stdout:7/41: mknod d2/d8/ca 0 2024-09-16T08:57:05.871 INFO:tasks.workunit.client.1.smithi097.stdout:3/110: fsync d7/f21 0 2024-09-16T08:57:05.873 INFO:tasks.workunit.client.1.smithi097.stdout:9/190: dwrite d2/f25 [0,4194304] 0 2024-09-16T08:57:05.873 INFO:tasks.workunit.client.1.smithi097.stdout:9/191: read - d2/d5/f2f zero size 2024-09-16T08:57:05.873 INFO:tasks.workunit.client.1.smithi097.stdout:3/111: creat d7/d15/d20/f22 x:0 0 0 2024-09-16T08:57:05.873 INFO:tasks.workunit.client.1.smithi097.stdout:3/112: creat d7/de/f23 x:0 0 0 2024-09-16T08:57:05.874 INFO:tasks.workunit.client.1.smithi097.stdout:3/113: fsync d7/fa 0 2024-09-16T08:57:05.874 INFO:tasks.workunit.client.1.smithi097.stdout:3/114: write d7/de/f23 [764499,43685] 0 2024-09-16T08:57:05.874 INFO:tasks.workunit.client.1.smithi097.stdout:3/115: write d7/d15/d20/f22 [560799,36579] 0 2024-09-16T08:57:05.876 INFO:tasks.workunit.client.1.smithi097.stdout:3/116: symlink d7/de/l24 0 2024-09-16T08:57:05.878 INFO:tasks.workunit.client.1.smithi097.stdout:3/117: rename d7/l1c to d7/l25 0 2024-09-16T08:57:05.878 INFO:tasks.workunit.client.1.smithi097.stdout:3/118: fdatasync d7/f21 0 2024-09-16T08:57:05.879 INFO:tasks.workunit.client.1.smithi097.stdout:3/119: read d7/de/f23 [608484,118825] 0 2024-09-16T08:57:05.881 INFO:tasks.workunit.client.1.smithi097.stdout:3/120: mknod d7/c26 0 2024-09-16T08:57:05.881 INFO:tasks.workunit.client.1.smithi097.stdout:3/121: fdatasync d7/fa 0 2024-09-16T08:57:05.881 INFO:tasks.workunit.client.1.smithi097.stdout:0/151: dwrite da/d1a/f2b [0,4194304] 0 2024-09-16T08:57:05.881 INFO:tasks.workunit.client.1.smithi097.stdout:0/152: chown da/df/d13/cd 497338160 1 2024-09-16T08:57:05.883 INFO:tasks.workunit.client.1.smithi097.stdout:8/142: dwrite d8/f1e [0,4194304] 0 2024-09-16T08:57:05.883 INFO:tasks.workunit.client.1.smithi097.stdout:8/143: fdatasync d8/fc 0 2024-09-16T08:57:05.883 INFO:tasks.workunit.client.1.smithi097.stdout:8/144: write d8/dd/d15/f22 [246217,42951] 0 2024-09-16T08:57:05.885 INFO:tasks.workunit.client.1.smithi097.stdout:8/145: mkdir d8/dd/d15/d16/d2d 0 2024-09-16T08:57:05.886 INFO:tasks.workunit.client.1.smithi097.stdout:3/122: rmdir d7/d1d 0 2024-09-16T08:57:05.887 INFO:tasks.workunit.client.1.smithi097.stdout:8/146: symlink d8/dd/d15/d16/l2e 0 2024-09-16T08:57:05.889 INFO:tasks.workunit.client.1.smithi097.stdout:8/147: creat d8/dd/f2f x:0 0 0 2024-09-16T08:57:05.889 INFO:tasks.workunit.client.1.smithi097.stdout:8/148: fdatasync d8/dd/f19 0 2024-09-16T08:57:05.889 INFO:tasks.workunit.client.1.smithi097.stdout:9/192: write d2/f13 [2448611,127359] 0 2024-09-16T08:57:05.889 INFO:tasks.workunit.client.1.smithi097.stdout:8/149: rename d8/dd/d15/d16/d2d to d8/dd/d15/d16/d2d/d30 22 2024-09-16T08:57:05.889 INFO:tasks.workunit.client.1.smithi097.stdout:9/193: stat d2/c24 0 2024-09-16T08:57:05.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/10: dwrite d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:05.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/11: chown d2/d4 166495 1 2024-09-16T08:57:05.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/12: dread - d2/f3 zero size 2024-09-16T08:57:05.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/13: truncate f0 771894 0 2024-09-16T08:57:05.890 INFO:tasks.workunit.client.0.smithi031.stdout:1/14: write d2/f3 [590964,83752] 0 2024-09-16T08:57:05.890 INFO:tasks.workunit.client.1.smithi097.stdout:2/161: dwrite d3/d4/f21 [0,4194304] 0 2024-09-16T08:57:05.893 INFO:tasks.workunit.client.1.smithi097.stdout:0/153: dread f6 [0,4194304] 0 2024-09-16T08:57:05.898 INFO:tasks.workunit.client.0.smithi031.stdout:5/27: dwrite d2/f8 [0,4194304] 0 2024-09-16T08:57:05.898 INFO:tasks.workunit.client.0.smithi031.stdout:5/28: chown d2/l9 181975404 1 2024-09-16T08:57:05.898 INFO:tasks.workunit.client.0.smithi031.stdout:5/29: creat d2/fa x:0 0 0 2024-09-16T08:57:05.899 INFO:tasks.workunit.client.1.smithi097.stdout:3/123: dread d7/fa [0,4194304] 0 2024-09-16T08:57:05.899 INFO:tasks.workunit.client.1.smithi097.stdout:3/124: creat d7/d15/d20/f27 x:0 0 0 2024-09-16T08:57:05.899 INFO:tasks.workunit.client.0.smithi031.stdout:0/20: dwrite f4 [0,4194304] 0 2024-09-16T08:57:05.900 INFO:tasks.workunit.client.1.smithi097.stdout:0/154: symlink da/df/l31 0 2024-09-16T08:57:05.901 INFO:tasks.workunit.client.1.smithi097.stdout:8/150: dread d8/f1e [0,4194304] 0 2024-09-16T08:57:05.901 INFO:tasks.workunit.client.1.smithi097.stdout:8/151: chown d8/f24 16262 1 2024-09-16T08:57:05.904 INFO:tasks.workunit.client.0.smithi031.stdout:1/15: write d2/d4/f5 [1793946,8240] 0 2024-09-16T08:57:05.904 INFO:tasks.workunit.client.0.smithi031.stdout:1/16: write d2/f3 [1379527,131029] 0 2024-09-16T08:57:05.906 INFO:tasks.workunit.client.1.smithi097.stdout:0/155: rename da/d1a/d26/c2e to da/df/d1f/c32 0 2024-09-16T08:57:05.907 INFO:tasks.workunit.client.0.smithi031.stdout:0/21: creat d6/f9 x:0 0 0 2024-09-16T08:57:05.909 INFO:tasks.workunit.client.0.smithi031.stdout:5/30: dread d2/f8 [0,4194304] 0 2024-09-16T08:57:05.909 INFO:tasks.workunit.client.0.smithi031.stdout:5/31: creat d2/fb x:0 0 0 2024-09-16T08:57:05.909 INFO:tasks.workunit.client.0.smithi031.stdout:5/32: fdatasync d2/fb 0 2024-09-16T08:57:05.909 INFO:tasks.workunit.client.1.smithi097.stdout:3/125: mkdir d7/d28 0 2024-09-16T08:57:05.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/152: rename d8/dd/d15/f22 to d8/f31 0 2024-09-16T08:57:05.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/153: chown f7 34652 1 2024-09-16T08:57:05.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/154: chown d8/f24 2109543286 1 2024-09-16T08:57:05.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/155: dread - d8/dd/f26 zero size 2024-09-16T08:57:05.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/156: chown c0 6086 1 2024-09-16T08:57:05.915 INFO:tasks.workunit.client.1.smithi097.stdout:3/126: unlink d7/c18 0 2024-09-16T08:57:05.915 INFO:tasks.workunit.client.1.smithi097.stdout:3/127: write d7/d15/d20/f27 [488355,34292] 0 2024-09-16T08:57:05.916 INFO:tasks.workunit.client.0.smithi031.stdout:5/33: mkdir d2/dc 0 2024-09-16T08:57:05.918 INFO:tasks.workunit.client.0.smithi031.stdout:5/34: creat d2/dc/fd x:0 0 0 2024-09-16T08:57:05.918 INFO:tasks.workunit.client.0.smithi031.stdout:5/35: chown d2/f8 1 1 2024-09-16T08:57:05.919 INFO:tasks.workunit.client.0.smithi031.stdout:5/36: chown d2/f7 21131497 1 2024-09-16T08:57:05.919 INFO:tasks.workunit.client.0.smithi031.stdout:5/37: dread - d2/fb zero size 2024-09-16T08:57:05.922 INFO:tasks.workunit.client.0.smithi031.stdout:3/18: getdents d3 0 2024-09-16T08:57:05.922 INFO:tasks.workunit.client.1.smithi097.stdout:8/157: write d8/f24 [103096,82860] 0 2024-09-16T08:57:05.924 INFO:tasks.workunit.client.0.smithi031.stdout:3/19: creat d3/f5 x:0 0 0 2024-09-16T08:57:05.927 INFO:tasks.workunit.client.1.smithi097.stdout:9/194: dwrite d2/fe [0,4194304] 0 2024-09-16T08:57:05.927 INFO:tasks.workunit.client.1.smithi097.stdout:9/195: write d2/f28 [196746,21536] 0 2024-09-16T08:57:05.930 INFO:tasks.workunit.client.0.smithi031.stdout:3/20: mknod d3/c6 0 2024-09-16T08:57:05.931 INFO:tasks.workunit.client.0.smithi031.stdout:3/21: mknod d3/c7 0 2024-09-16T08:57:05.932 INFO:tasks.workunit.client.0.smithi031.stdout:3/22: creat d3/f8 x:0 0 0 2024-09-16T08:57:05.933 INFO:tasks.workunit.client.1.smithi097.stdout:9/196: dread f0 [4194304,4194304] 0 2024-09-16T08:57:05.937 INFO:tasks.workunit.client.0.smithi031.stdout:8/3: getdents . 0 2024-09-16T08:57:05.937 INFO:tasks.workunit.client.0.smithi031.stdout:8/4: write - no filename 2024-09-16T08:57:05.937 INFO:tasks.workunit.client.0.smithi031.stdout:8/5: dwrite - no filename 2024-09-16T08:57:05.937 INFO:tasks.workunit.client.0.smithi031.stdout:8/6: dread - no filename 2024-09-16T08:57:05.937 INFO:tasks.workunit.client.0.smithi031.stdout:8/7: write - no filename 2024-09-16T08:57:05.939 INFO:tasks.workunit.client.0.smithi031.stdout:8/8: rename c0 to c1 0 2024-09-16T08:57:05.947 INFO:tasks.workunit.client.0.smithi031.stdout:7/42: unlink d2/l9 0 2024-09-16T08:57:05.955 INFO:tasks.workunit.client.0.smithi031.stdout:0/22: dwrite d6/f9 [0,4194304] 0 2024-09-16T08:57:05.955 INFO:tasks.workunit.client.0.smithi031.stdout:0/23: chown c2 808 1 2024-09-16T08:57:05.955 INFO:tasks.workunit.client.0.smithi031.stdout:0/24: stat c2 0 2024-09-16T08:57:05.955 INFO:tasks.workunit.client.0.smithi031.stdout:0/25: creat d6/fa x:0 0 0 2024-09-16T08:57:05.956 INFO:tasks.workunit.client.0.smithi031.stdout:0/26: write d6/fa [817512,57560] 0 2024-09-16T08:57:05.956 INFO:tasks.workunit.client.1.smithi097.stdout:0/156: dwrite f8 [0,4194304] 0 2024-09-16T08:57:05.956 INFO:tasks.workunit.client.0.smithi031.stdout:1/17: dwrite f0 [0,4194304] 0 2024-09-16T08:57:05.957 INFO:tasks.workunit.client.0.smithi031.stdout:5/38: dwrite d2/dc/fd [0,4194304] 0 2024-09-16T08:57:05.957 INFO:tasks.workunit.client.1.smithi097.stdout:9/197: dwrite d2/d5/f29 [0,4194304] 0 2024-09-16T08:57:05.959 INFO:tasks.workunit.client.0.smithi031.stdout:5/39: write d2/f8 [4733888,6038] 0 2024-09-16T08:57:05.960 INFO:tasks.workunit.client.0.smithi031.stdout:5/40: creat d2/fe x:0 0 0 2024-09-16T08:57:05.960 INFO:tasks.workunit.client.0.smithi031.stdout:5/41: creat d2/dc/ff x:0 0 0 2024-09-16T08:57:05.962 INFO:tasks.workunit.client.0.smithi031.stdout:0/27: rename c2 to d6/cb 0 2024-09-16T08:57:05.962 INFO:tasks.workunit.client.0.smithi031.stdout:0/28: creat d6/fc x:0 0 0 2024-09-16T08:57:05.967 INFO:tasks.workunit.client.1.smithi097.stdout:8/158: getdents d8 0 2024-09-16T08:57:05.978 INFO:tasks.workunit.client.0.smithi031.stdout:8/9: link c1 c2 0 2024-09-16T08:57:05.978 INFO:tasks.workunit.client.0.smithi031.stdout:8/10: truncate - no filename 2024-09-16T08:57:05.978 INFO:tasks.workunit.client.0.smithi031.stdout:8/11: dread - no filename 2024-09-16T08:57:05.978 INFO:tasks.workunit.client.0.smithi031.stdout:8/12: dwrite - no filename 2024-09-16T08:57:05.980 INFO:tasks.workunit.client.0.smithi031.stdout:7/43: truncate d2/f4 586255 0 2024-09-16T08:57:05.980 INFO:tasks.workunit.client.0.smithi031.stdout:7/44: readlink l0 0 2024-09-16T08:57:05.985 INFO:tasks.workunit.client.0.smithi031.stdout:5/42: mknod d2/dc/c10 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/43: creat d2/f11 x:0 0 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/44: creat d2/dc/f12 x:0 0 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/45: creat d2/f13 x:0 0 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/46: truncate d2/dc/fd 4805041 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/47: write d2/f13 [5385,85014] 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/48: dread - d2/fe zero size 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/49: write d2/f5 [946427,77371] 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.0.smithi031.stdout:5/50: write d2/dc/ff [254090,8998] 0 2024-09-16T08:57:05.986 INFO:tasks.workunit.client.1.smithi097.stdout:3/128: dwrite d7/d15/d20/f27 [0,4194304] 0 2024-09-16T08:57:05.987 INFO:tasks.workunit.client.0.smithi031.stdout:1/18: getdents d2/d4 0 2024-09-16T08:57:05.987 INFO:tasks.workunit.client.0.smithi031.stdout:1/19: rename d2 to d2/d4/d9 22 2024-09-16T08:57:05.987 INFO:tasks.workunit.client.0.smithi031.stdout:1/20: write d2/d4/f5 [2282824,117713] 0 2024-09-16T08:57:05.988 INFO:tasks.workunit.client.0.smithi031.stdout:1/21: chown d2/d4/l7 32672 1 2024-09-16T08:57:05.988 INFO:tasks.workunit.client.0.smithi031.stdout:8/13: stat c2 0 2024-09-16T08:57:05.988 INFO:tasks.workunit.client.1.smithi097.stdout:8/159: dwrite d8/dd/f2f [0,4194304] 0 2024-09-16T08:57:05.988 INFO:tasks.workunit.client.1.smithi097.stdout:8/160: stat d8 0 2024-09-16T08:57:05.988 INFO:tasks.workunit.client.1.smithi097.stdout:9/198: dwrite d2/d5/f11 [0,4194304] 0 2024-09-16T08:57:05.991 INFO:tasks.workunit.client.0.smithi031.stdout:7/45: unlink d2/c7 0 2024-09-16T08:57:05.993 INFO:tasks.workunit.client.1.smithi097.stdout:3/129: mkdir d7/de/d29 0 2024-09-16T08:57:05.993 INFO:tasks.workunit.client.1.smithi097.stdout:9/199: dread d2/f13 [0,4194304] 0 2024-09-16T08:57:05.994 INFO:tasks.workunit.client.0.smithi031.stdout:0/29: fsync d6/fa 0 2024-09-16T08:57:05.994 INFO:tasks.workunit.client.0.smithi031.stdout:0/30: dread - d6/fc zero size 2024-09-16T08:57:06.000 INFO:tasks.workunit.client.1.smithi097.stdout:8/161: rename d8/dd/l2b to d8/dd/d15/d16/d2d/l32 0 2024-09-16T08:57:06.002 INFO:tasks.workunit.client.0.smithi031.stdout:5/51: unlink d2/f8 0 2024-09-16T08:57:06.002 INFO:tasks.workunit.client.0.smithi031.stdout:5/52: dread - d2/fb zero size 2024-09-16T08:57:06.002 INFO:tasks.workunit.client.0.smithi031.stdout:5/53: dread - d2/fb zero size 2024-09-16T08:57:06.002 INFO:tasks.workunit.client.0.smithi031.stdout:5/54: chown d2/f5 295374618 1 2024-09-16T08:57:06.002 INFO:tasks.workunit.client.0.smithi031.stdout:5/55: chown d2/f13 149861513 1 2024-09-16T08:57:06.006 INFO:tasks.workunit.client.1.smithi097.stdout:3/130: creat d7/de/d29/f2a x:0 0 0 2024-09-16T08:57:06.006 INFO:tasks.workunit.client.1.smithi097.stdout:3/131: chown d7/d15/d20/f22 25 1 2024-09-16T08:57:06.008 INFO:tasks.workunit.client.0.smithi031.stdout:1/22: creat d2/d4/fa x:0 0 0 2024-09-16T08:57:06.009 INFO:tasks.workunit.client.1.smithi097.stdout:9/200: symlink d2/d5/l40 0 2024-09-16T08:57:06.011 INFO:tasks.workunit.client.1.smithi097.stdout:2/162: truncate d3/d4/d9/ff 2097597 0 2024-09-16T08:57:06.011 INFO:tasks.workunit.client.0.smithi031.stdout:7/46: rmdir d2/d8 39 2024-09-16T08:57:06.014 INFO:tasks.workunit.client.0.smithi031.stdout:0/31: unlink f0 0 2024-09-16T08:57:06.015 INFO:tasks.workunit.client.0.smithi031.stdout:0/32: stat d6/fc 0 2024-09-16T08:57:06.016 INFO:tasks.workunit.client.1.smithi097.stdout:8/162: link d8/dd/d15/d16/c1b d8/dd/c33 0 2024-09-16T08:57:06.019 INFO:tasks.workunit.client.1.smithi097.stdout:3/132: creat d7/f2b x:0 0 0 2024-09-16T08:57:06.019 INFO:tasks.workunit.client.1.smithi097.stdout:3/133: chown d7/f21 53 1 2024-09-16T08:57:06.020 INFO:tasks.workunit.client.0.smithi031.stdout:7/47: dread d2/f4 [0,4194304] 0 2024-09-16T08:57:06.021 INFO:tasks.workunit.client.0.smithi031.stdout:5/56: unlink d2/f7 0 2024-09-16T08:57:06.022 INFO:tasks.workunit.client.1.smithi097.stdout:9/201: creat d2/d3f/f41 x:0 0 0 2024-09-16T08:57:06.025 INFO:tasks.workunit.client.1.smithi097.stdout:2/163: rename d3/d4/d9/f20 to d3/f23 0 2024-09-16T08:57:06.026 INFO:tasks.workunit.client.0.smithi031.stdout:8/14: link c2 c3 0 2024-09-16T08:57:06.026 INFO:tasks.workunit.client.0.smithi031.stdout:8/15: chown c3 1111 1 2024-09-16T08:57:06.027 INFO:tasks.workunit.client.1.smithi097.stdout:3/134: rename d7/de/d29/f2a to d7/de/d29/f2c 0 2024-09-16T08:57:06.027 INFO:tasks.workunit.client.1.smithi097.stdout:3/135: write d7/d15/d20/f22 [851907,23081] 0 2024-09-16T08:57:06.027 INFO:tasks.workunit.client.1.smithi097.stdout:3/136: creat d7/de/d29/f2d x:0 0 0 2024-09-16T08:57:06.027 INFO:tasks.workunit.client.1.smithi097.stdout:3/137: chown d7/d15/c1f 5296 1 2024-09-16T08:57:06.028 INFO:tasks.workunit.client.1.smithi097.stdout:3/138: write d7/d15/d20/f22 [269238,102337] 0 2024-09-16T08:57:06.028 INFO:tasks.workunit.client.0.smithi031.stdout:0/33: rename c5 to d6/cd 0 2024-09-16T08:57:06.029 INFO:tasks.workunit.client.1.smithi097.stdout:9/202: rename d2/c37 to d2/c42 0 2024-09-16T08:57:06.030 INFO:tasks.workunit.client.1.smithi097.stdout:9/203: write d2/f6 [5339258,107237] 0 2024-09-16T08:57:06.031 INFO:tasks.workunit.client.0.smithi031.stdout:7/48: link d2/f4 d2/fb 0 2024-09-16T08:57:06.033 INFO:tasks.workunit.client.1.smithi097.stdout:9/204: fdatasync d2/f6 0 2024-09-16T08:57:06.033 INFO:tasks.workunit.client.1.smithi097.stdout:9/205: write d2/d3f/f41 [724722,49306] 0 2024-09-16T08:57:06.033 INFO:tasks.workunit.client.1.smithi097.stdout:9/206: creat d2/d3f/f43 x:0 0 0 2024-09-16T08:57:06.033 INFO:tasks.workunit.client.1.smithi097.stdout:8/163: rename d8/f23 to d8/dd/d15/f34 0 2024-09-16T08:57:06.033 INFO:tasks.workunit.client.1.smithi097.stdout:2/164: rename d3 to d3/d1b/d24 22 2024-09-16T08:57:06.033 INFO:tasks.workunit.client.0.smithi031.stdout:7/49: dread d2/fb [0,4194304] 0 2024-09-16T08:57:06.040 INFO:tasks.workunit.client.1.smithi097.stdout:9/207: rename d2/l10 to d2/d5/d3a/d3c/l44 0 2024-09-16T08:57:06.040 INFO:tasks.workunit.client.1.smithi097.stdout:9/208: chown d2/d5/fb 8273044 1 2024-09-16T08:57:06.040 INFO:tasks.workunit.client.1.smithi097.stdout:9/209: write d2/d3f/f43 [860359,34509] 0 2024-09-16T08:57:06.040 INFO:tasks.workunit.client.1.smithi097.stdout:9/210: creat d2/d30/f45 x:0 0 0 2024-09-16T08:57:06.040 INFO:tasks.workunit.client.1.smithi097.stdout:9/211: stat d2/c2e 0 2024-09-16T08:57:06.040 INFO:tasks.workunit.client.1.smithi097.stdout:9/212: creat d2/f46 x:0 0 0 2024-09-16T08:57:06.041 INFO:tasks.workunit.client.0.smithi031.stdout:7/50: link d2/c5 d2/d8/cc 0 2024-09-16T08:57:06.044 INFO:tasks.workunit.client.1.smithi097.stdout:8/164: unlink f7 0 2024-09-16T08:57:06.044 INFO:tasks.workunit.client.1.smithi097.stdout:8/165: readlink d8/dd/l1a 0 2024-09-16T08:57:06.044 INFO:tasks.workunit.client.1.smithi097.stdout:8/166: fdatasync d8/dd/f1f 0 2024-09-16T08:57:06.044 INFO:tasks.workunit.client.1.smithi097.stdout:8/167: fsync d8/dd/f19 0 2024-09-16T08:57:06.044 INFO:tasks.workunit.client.1.smithi097.stdout:8/168: readlink d8/dd/d15/l1d 0 2024-09-16T08:57:06.044 INFO:tasks.workunit.client.1.smithi097.stdout:3/139: dwrite d7/d15/d20/f22 [0,4194304] 0 2024-09-16T08:57:06.045 INFO:tasks.workunit.client.1.smithi097.stdout:2/165: creat d3/d4/d8/f25 x:0 0 0 2024-09-16T08:57:06.045 INFO:tasks.workunit.client.1.smithi097.stdout:2/166: truncate d3/d4/d8/f25 998233 0 2024-09-16T08:57:06.045 INFO:tasks.workunit.client.0.smithi031.stdout:1/23: getdents d2/d4 0 2024-09-16T08:57:06.048 INFO:tasks.workunit.client.0.smithi031.stdout:1/24: mknod d2/d4/cb 0 2024-09-16T08:57:06.052 INFO:tasks.workunit.client.0.smithi031.stdout:1/25: write d2/d4/f5 [2005254,99102] 0 2024-09-16T08:57:06.053 INFO:tasks.workunit.client.1.smithi097.stdout:9/213: rename d2/d5/f2f to d2/d30/f47 0 2024-09-16T08:57:06.053 INFO:tasks.workunit.client.1.smithi097.stdout:9/214: chown d2/d5/d3a 210 1 2024-09-16T08:57:06.054 INFO:tasks.workunit.client.1.smithi097.stdout:8/169: creat d8/dd/d15/d16/d2d/f35 x:0 0 0 2024-09-16T08:57:06.055 INFO:tasks.workunit.client.0.smithi031.stdout:1/26: truncate d2/f3 410072 0 2024-09-16T08:57:06.059 INFO:tasks.workunit.client.1.smithi097.stdout:7/186: sync 2024-09-16T08:57:06.059 INFO:tasks.workunit.client.1.smithi097.stdout:7/187: stat d4/c18 0 2024-09-16T08:57:06.059 INFO:tasks.workunit.client.1.smithi097.stdout:7/188: stat d4/d9/l20 0 2024-09-16T08:57:06.060 INFO:tasks.workunit.client.0.smithi031.stdout:1/27: fdatasync d2/d4/f5 0 2024-09-16T08:57:06.061 INFO:tasks.workunit.client.1.smithi097.stdout:3/140: mknod d7/de/c2e 0 2024-09-16T08:57:06.061 INFO:tasks.workunit.client.1.smithi097.stdout:3/141: chown d7/de/d29/f2d 11 1 2024-09-16T08:57:06.061 INFO:tasks.workunit.client.1.smithi097.stdout:3/142: read - d7/de/f1e zero size 2024-09-16T08:57:06.062 INFO:tasks.workunit.client.1.smithi097.stdout:8/170: creat d8/dd/d15/d16/f36 x:0 0 0 2024-09-16T08:57:06.064 INFO:tasks.workunit.client.1.smithi097.stdout:3/143: creat d7/d28/f2f x:0 0 0 2024-09-16T08:57:06.064 INFO:tasks.workunit.client.1.smithi097.stdout:3/144: readlink d7/l12 0 2024-09-16T08:57:06.064 INFO:tasks.workunit.client.1.smithi097.stdout:3/145: truncate d7/f21 1762096 0 2024-09-16T08:57:06.065 INFO:tasks.workunit.client.0.smithi031.stdout:1/28: dread d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:06.065 INFO:tasks.workunit.client.0.smithi031.stdout:7/51: fsync d2/fb 0 2024-09-16T08:57:06.066 INFO:tasks.workunit.client.0.smithi031.stdout:7/52: dread d2/fb [0,4194304] 0 2024-09-16T08:57:06.066 INFO:tasks.workunit.client.1.smithi097.stdout:8/171: mknod d8/dd/d15/d16/d2d/c37 0 2024-09-16T08:57:06.067 INFO:tasks.workunit.client.0.smithi031.stdout:1/29: stat d2/d4/l7 0 2024-09-16T08:57:06.068 INFO:tasks.workunit.client.0.smithi031.stdout:7/53: unlink d2/d8/cc 0 2024-09-16T08:57:06.068 INFO:tasks.workunit.client.0.smithi031.stdout:7/54: write d2/f4 [898224,24929] 0 2024-09-16T08:57:06.070 INFO:tasks.workunit.client.1.smithi097.stdout:8/172: dread d8/f1e [0,4194304] 0 2024-09-16T08:57:06.071 INFO:tasks.workunit.client.1.smithi097.stdout:8/173: symlink d8/dd/l38 0 2024-09-16T08:57:06.075 INFO:tasks.workunit.client.1.smithi097.stdout:9/215: getdents d2 0 2024-09-16T08:57:06.092 INFO:tasks.workunit.client.0.smithi031.stdout:1/30: dwrite d2/d4/fa [0,4194304] 0 2024-09-16T08:57:06.093 INFO:tasks.workunit.client.1.smithi097.stdout:9/216: dwrite f0 [4194304,4194304] 0 2024-09-16T08:57:06.093 INFO:tasks.workunit.client.1.smithi097.stdout:9/217: dread - d2/d30/f47 zero size 2024-09-16T08:57:06.094 INFO:tasks.workunit.client.0.smithi031.stdout:1/31: link d2/d4/cb d2/cc 0 2024-09-16T08:57:06.095 INFO:tasks.workunit.client.0.smithi031.stdout:1/32: mknod d2/cd 0 2024-09-16T08:57:06.096 INFO:tasks.workunit.client.0.smithi031.stdout:1/33: getdents d2/d4 0 2024-09-16T08:57:06.096 INFO:tasks.workunit.client.0.smithi031.stdout:1/34: chown d2/d4/fa 13867 1 2024-09-16T08:57:06.097 INFO:tasks.workunit.client.0.smithi031.stdout:1/35: mkdir d2/de 0 2024-09-16T08:57:06.097 INFO:tasks.workunit.client.0.smithi031.stdout:1/36: chown f0 14399 1 2024-09-16T08:57:06.098 INFO:tasks.workunit.client.0.smithi031.stdout:1/37: mknod d2/d4/cf 0 2024-09-16T08:57:06.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/38: rename d2/d4/cf to d2/de/c10 0 2024-09-16T08:57:06.111 INFO:tasks.workunit.client.1.smithi097.stdout:9/218: dwrite d2/f28 [0,4194304] 0 2024-09-16T08:57:06.111 INFO:tasks.workunit.client.1.smithi097.stdout:9/219: write d2/f1a [1083120,107716] 0 2024-09-16T08:57:06.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:05 smithi031 ceph-mon[27844]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:57:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:05 smithi031 ceph-mon[27844]: Deploying cephadm binary to smithi097 2024-09-16T08:57:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:05 smithi031 ceph-mon[27844]: Deploying cephadm binary to smithi031 2024-09-16T08:57:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:05 smithi031 ceph-mon[27844]: pgmap v4: 65 pgs: 65 active+clean; 355 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:57:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:05 smithi031 ceph-mon[27844]: mgrmap e23: smithi097.mpashf(active, since 2s) 2024-09-16T08:57:06.131 INFO:tasks.workunit.client.0.smithi031.stdout:7/55: fsync d2/f4 0 2024-09-16T08:57:06.131 INFO:tasks.workunit.client.0.smithi031.stdout:7/56: truncate d2/f4 1001730 0 2024-09-16T08:57:06.131 INFO:tasks.workunit.client.0.smithi031.stdout:7/57: readlink l0 0 2024-09-16T08:57:06.131 INFO:tasks.workunit.client.0.smithi031.stdout:1/39: dread d2/f3 [0,4194304] 0 2024-09-16T08:57:06.131 INFO:tasks.workunit.client.0.smithi031.stdout:1/40: write d2/d4/f5 [211489,25299] 0 2024-09-16T08:57:06.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/58: creat d2/d8/fd x:0 0 0 2024-09-16T08:57:06.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/59: read - d2/d8/fd zero size 2024-09-16T08:57:06.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/60: stat d2/f4 0 2024-09-16T08:57:06.133 INFO:tasks.workunit.client.1.smithi097.stdout:4/207: sync 2024-09-16T08:57:06.134 INFO:tasks.workunit.client.0.smithi031.stdout:1/41: symlink d2/d4/l11 0 2024-09-16T08:57:06.134 INFO:tasks.workunit.client.0.smithi031.stdout:1/42: chown c1 171 1 2024-09-16T08:57:06.134 INFO:tasks.workunit.client.1.smithi097.stdout:1/213: sync 2024-09-16T08:57:06.134 INFO:tasks.workunit.client.1.smithi097.stdout:1/214: write d5/db/f34 [4692588,80883] 0 2024-09-16T08:57:06.135 INFO:tasks.workunit.client.1.smithi097.stdout:5/220: sync 2024-09-16T08:57:06.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/61: link d2/d8/fd d2/d8/fe 0 2024-09-16T08:57:06.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/62: read d2/fb [611019,17385] 0 2024-09-16T08:57:06.137 INFO:tasks.workunit.client.0.smithi031.stdout:1/43: creat d2/d4/f12 x:0 0 0 2024-09-16T08:57:06.140 INFO:tasks.workunit.client.0.smithi031.stdout:7/63: unlink d2/d8/ca 0 2024-09-16T08:57:06.144 INFO:tasks.workunit.client.1.smithi097.stdout:8/174: dwrite d8/dd/d15/f28 [0,4194304] 0 2024-09-16T08:57:06.145 INFO:tasks.workunit.client.1.smithi097.stdout:8/175: truncate d8/f10 571009 0 2024-09-16T08:57:06.145 INFO:tasks.workunit.client.0.smithi031.stdout:7/64: rename d2/f4 to d2/ff 0 2024-09-16T08:57:06.145 INFO:tasks.workunit.client.0.smithi031.stdout:7/65: readlink l0 0 2024-09-16T08:57:06.149 INFO:tasks.workunit.client.1.smithi097.stdout:5/221: dread db/d18/d1c/f39 [0,4194304] 0 2024-09-16T08:57:06.149 INFO:tasks.workunit.client.0.smithi031.stdout:7/66: symlink d2/d8/l10 0 2024-09-16T08:57:06.151 INFO:tasks.workunit.client.1.smithi097.stdout:8/176: mknod d8/dd/d15/d16/d2d/c39 0 2024-09-16T08:57:06.152 INFO:tasks.workunit.client.1.smithi097.stdout:4/208: dwrite d0/fe [0,4194304] 0 2024-09-16T08:57:06.152 INFO:tasks.workunit.client.1.smithi097.stdout:4/209: chown d0/f2f 4147466 1 2024-09-16T08:57:06.152 INFO:tasks.workunit.client.1.smithi097.stdout:4/210: creat d0/f4f x:0 0 0 2024-09-16T08:57:06.154 INFO:tasks.workunit.client.1.smithi097.stdout:5/222: creat db/d18/d1c/d42/f4c x:0 0 0 2024-09-16T08:57:06.156 INFO:tasks.workunit.client.1.smithi097.stdout:8/177: unlink c2 0 2024-09-16T08:57:06.161 INFO:tasks.workunit.client.1.smithi097.stdout:0/157: sync 2024-09-16T08:57:06.161 INFO:tasks.workunit.client.1.smithi097.stdout:6/207: sync 2024-09-16T08:57:06.161 INFO:tasks.workunit.client.1.smithi097.stdout:0/158: chown da/df/d1f 7 1 2024-09-16T08:57:06.162 INFO:tasks.workunit.client.1.smithi097.stdout:0/159: write da/f15 [736567,23545] 0 2024-09-16T08:57:06.162 INFO:tasks.workunit.client.1.smithi097.stdout:0/160: write da/df/d1f/f2a [31355,46778] 0 2024-09-16T08:57:06.162 INFO:tasks.workunit.client.0.smithi031.stdout:7/67: getdents d2/d8 0 2024-09-16T08:57:06.163 INFO:tasks.workunit.client.0.smithi031.stdout:7/68: mkdir d2/d8/d11 0 2024-09-16T08:57:06.164 INFO:tasks.workunit.client.0.smithi031.stdout:7/69: mknod d2/d8/d11/c12 0 2024-09-16T08:57:06.164 INFO:tasks.workunit.client.0.smithi031.stdout:7/70: readlink l0 0 2024-09-16T08:57:06.165 INFO:tasks.workunit.client.0.smithi031.stdout:7/71: rename d2/d8/d11 to d2/d8/d11/d13 22 2024-09-16T08:57:06.165 INFO:tasks.workunit.client.1.smithi097.stdout:0/161: dread da/d24/f20 [0,4194304] 0 2024-09-16T08:57:06.165 INFO:tasks.workunit.client.0.smithi031.stdout:7/72: creat d2/d8/d11/f14 x:0 0 0 2024-09-16T08:57:06.167 INFO:tasks.workunit.client.1.smithi097.stdout:0/162: unlink da/df/l11 0 2024-09-16T08:57:06.168 INFO:tasks.workunit.client.1.smithi097.stdout:5/223: dread db/f32 [4194304,4194304] 0 2024-09-16T08:57:06.168 INFO:tasks.workunit.client.1.smithi097.stdout:5/224: chown db/d18/d1c/d42/d46/f47 159222 1 2024-09-16T08:57:06.169 INFO:tasks.workunit.client.1.smithi097.stdout:0/163: creat da/d24/f33 x:0 0 0 2024-09-16T08:57:06.169 INFO:tasks.workunit.client.0.smithi031.stdout:7/73: getdents d2 0 2024-09-16T08:57:06.170 INFO:tasks.workunit.client.1.smithi097.stdout:5/225: unlink db/d18/d20/l37 0 2024-09-16T08:57:06.171 INFO:tasks.workunit.client.0.smithi031.stdout:7/74: chown d2/d8/fe 213619 1 2024-09-16T08:57:06.171 INFO:tasks.workunit.client.0.smithi031.stdout:7/75: fdatasync d2/fb 0 2024-09-16T08:57:06.171 INFO:tasks.workunit.client.1.smithi097.stdout:0/164: mknod da/d1a/d26/c34 0 2024-09-16T08:57:06.171 INFO:tasks.workunit.client.1.smithi097.stdout:5/226: dread db/f10 [0,4194304] 0 2024-09-16T08:57:06.172 INFO:tasks.workunit.client.0.smithi031.stdout:7/76: rename d2/d8/d11/c12 to d2/c15 0 2024-09-16T08:57:06.173 INFO:tasks.workunit.client.1.smithi097.stdout:6/208: dread d1/d3/f3d [0,4194304] 0 2024-09-16T08:57:06.173 INFO:tasks.workunit.client.1.smithi097.stdout:6/209: chown d1/d3/l8 39029 1 2024-09-16T08:57:06.173 INFO:tasks.workunit.client.1.smithi097.stdout:6/210: chown d1/f37 3058779 1 2024-09-16T08:57:06.173 INFO:tasks.workunit.client.1.smithi097.stdout:0/165: symlink da/df/l35 0 2024-09-16T08:57:06.174 INFO:tasks.workunit.client.1.smithi097.stdout:4/211: dwrite d0/d7/df/d15/d16/d26/d3d/f44 [0,4194304] 0 2024-09-16T08:57:06.174 INFO:tasks.workunit.client.0.smithi031.stdout:1/44: dwrite d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:06.174 INFO:tasks.workunit.client.0.smithi031.stdout:1/45: fsync d2/d4/fa 0 2024-09-16T08:57:06.174 INFO:tasks.workunit.client.0.smithi031.stdout:7/77: symlink d2/d8/d11/l16 0 2024-09-16T08:57:06.176 INFO:tasks.workunit.client.1.smithi097.stdout:0/166: rename da/c29 to da/c36 0 2024-09-16T08:57:06.176 INFO:tasks.workunit.client.1.smithi097.stdout:0/167: fsync da/df/d13/f2f 0 2024-09-16T08:57:06.177 INFO:tasks.workunit.client.0.smithi031.stdout:7/78: symlink d2/l17 0 2024-09-16T08:57:06.181 INFO:tasks.workunit.client.0.smithi031.stdout:7/79: creat d2/f18 x:0 0 0 2024-09-16T08:57:06.181 INFO:tasks.workunit.client.0.smithi031.stdout:7/80: stat d2/l17 0 2024-09-16T08:57:06.183 INFO:tasks.workunit.client.1.smithi097.stdout:4/212: link d0/d7/df/f1e d0/d7/df/d15/d16/f50 0 2024-09-16T08:57:06.183 INFO:tasks.workunit.client.1.smithi097.stdout:4/213: truncate d0/d7/df/d15/d16/f50 1032460 0 2024-09-16T08:57:06.183 INFO:tasks.workunit.client.1.smithi097.stdout:4/214: write d0/d7/df/d15/f11 [3090664,58679] 0 2024-09-16T08:57:06.183 INFO:tasks.workunit.client.1.smithi097.stdout:4/215: creat d0/d36/f51 x:0 0 0 2024-09-16T08:57:06.185 INFO:tasks.workunit.client.1.smithi097.stdout:0/168: mkdir da/df/d37 0 2024-09-16T08:57:06.185 INFO:tasks.workunit.client.1.smithi097.stdout:0/169: rmdir da/d1a/d26 39 2024-09-16T08:57:06.187 INFO:tasks.workunit.client.1.smithi097.stdout:0/170: link da/df/d13/c18 da/d1a/d26/c38 0 2024-09-16T08:57:06.187 INFO:tasks.workunit.client.1.smithi097.stdout:0/171: creat da/df/d37/f39 x:0 0 0 2024-09-16T08:57:06.193 INFO:tasks.workunit.client.0.smithi031.stdout:1/46: dwrite d2/d4/fa [4194304,4194304] 0 2024-09-16T08:57:06.194 INFO:tasks.workunit.client.1.smithi097.stdout:6/211: dwrite d1/d3/f11 [0,4194304] 0 2024-09-16T08:57:06.194 INFO:tasks.workunit.client.1.smithi097.stdout:6/212: dread - d1/d3/df/f36 zero size 2024-09-16T08:57:06.195 INFO:tasks.workunit.client.0.smithi031.stdout:1/47: rmdir d2/d4 39 2024-09-16T08:57:06.195 INFO:tasks.workunit.client.0.smithi031.stdout:1/48: truncate d2/d4/f12 946666 0 2024-09-16T08:57:06.198 INFO:tasks.workunit.client.1.smithi097.stdout:0/172: write da/d1a/f2b [3855927,85783] 0 2024-09-16T08:57:06.200 INFO:tasks.workunit.client.1.smithi097.stdout:0/173: truncate da/d24/f20 291457 0 2024-09-16T08:57:06.200 INFO:tasks.workunit.client.1.smithi097.stdout:0/174: chown f6 1 1 2024-09-16T08:57:06.206 INFO:tasks.workunit.client.0.smithi031.stdout:1/49: dread d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:06.206 INFO:tasks.workunit.client.0.smithi031.stdout:1/50: fsync d2/d4/f12 0 2024-09-16T08:57:06.207 INFO:tasks.workunit.client.1.smithi097.stdout:4/216: dwrite d0/f2f [0,4194304] 0 2024-09-16T08:57:06.207 INFO:tasks.workunit.client.1.smithi097.stdout:4/217: readlink d0/d7/l4d 0 2024-09-16T08:57:06.208 INFO:tasks.workunit.client.1.smithi097.stdout:0/175: dread f6 [0,4194304] 0 2024-09-16T08:57:06.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/51: rmdir d2/d4 39 2024-09-16T08:57:06.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/52: fsync d2/f3 0 2024-09-16T08:57:06.210 INFO:tasks.workunit.client.1.smithi097.stdout:0/176: rmdir da/d1a/d26 39 2024-09-16T08:57:06.213 INFO:tasks.workunit.client.0.smithi031.stdout:1/53: dread d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:06.213 INFO:tasks.workunit.client.1.smithi097.stdout:4/218: rename d0/d7/df/d15/d16 to d0/d7/d52 0 2024-09-16T08:57:06.214 INFO:tasks.workunit.client.1.smithi097.stdout:6/213: dwrite d1/d3/f2f [0,4194304] 0 2024-09-16T08:57:06.214 INFO:tasks.workunit.client.1.smithi097.stdout:6/214: stat d1/dc/d1e 0 2024-09-16T08:57:06.215 INFO:tasks.workunit.client.1.smithi097.stdout:4/219: symlink d0/d7/d52/d26/d3d/l53 0 2024-09-16T08:57:06.215 INFO:tasks.workunit.client.1.smithi097.stdout:4/220: creat d0/d36/f54 x:0 0 0 2024-09-16T08:57:06.236 INFO:tasks.workunit.client.1.smithi097.stdout:0/177: dwrite da/df/d13/f2f [0,4194304] 0 2024-09-16T08:57:06.236 INFO:tasks.workunit.client.1.smithi097.stdout:0/178: creat da/d1a/f3a x:0 0 0 2024-09-16T08:57:06.236 INFO:tasks.workunit.client.1.smithi097.stdout:6/215: dwrite d1/d3/f34 [0,4194304] 0 2024-09-16T08:57:06.237 INFO:tasks.workunit.client.1.smithi097.stdout:6/216: dread - d1/d3/d16/f2e zero size 2024-09-16T08:57:06.237 INFO:tasks.workunit.client.1.smithi097.stdout:0/179: truncate da/df/f16 1640148 0 2024-09-16T08:57:06.238 INFO:tasks.workunit.client.1.smithi097.stdout:0/180: readlink l3 0 2024-09-16T08:57:06.238 INFO:tasks.workunit.client.1.smithi097.stdout:6/217: creat d1/dc/d3c/f44 x:0 0 0 2024-09-16T08:57:06.239 INFO:tasks.workunit.client.1.smithi097.stdout:0/181: mkdir da/df/d1f/d3b 0 2024-09-16T08:57:06.241 INFO:tasks.workunit.client.1.smithi097.stdout:6/218: rename d1/dc/d1e/l25 to d1/d3/df/l45 0 2024-09-16T08:57:06.243 INFO:tasks.workunit.client.1.smithi097.stdout:0/182: rename f1 to da/df/d1f/f3c 0 2024-09-16T08:57:06.244 INFO:tasks.workunit.client.1.smithi097.stdout:0/183: stat da/df/d1f/c32 0 2024-09-16T08:57:06.244 INFO:tasks.workunit.client.1.smithi097.stdout:0/184: write da/f1b [1777415,24739] 0 2024-09-16T08:57:06.249 INFO:tasks.workunit.client.1.smithi097.stdout:0/185: dread f6 [0,4194304] 0 2024-09-16T08:57:06.250 INFO:tasks.workunit.client.1.smithi097.stdout:0/186: mkdir da/df/d3d 0 2024-09-16T08:57:06.250 INFO:tasks.workunit.client.1.smithi097.stdout:0/187: chown da/df/d37 3056989 1 2024-09-16T08:57:06.250 INFO:tasks.workunit.client.1.smithi097.stdout:6/219: dread d1/d3/f2f [0,4194304] 0 2024-09-16T08:57:06.252 INFO:tasks.workunit.client.1.smithi097.stdout:0/188: unlink da/f15 0 2024-09-16T08:57:06.259 INFO:tasks.workunit.client.1.smithi097.stdout:6/220: write d1/f22 [3519943,97963] 0 2024-09-16T08:57:06.261 INFO:tasks.workunit.client.1.smithi097.stdout:6/221: dread d1/d3/df/f32 [0,4194304] 0 2024-09-16T08:57:06.261 INFO:tasks.workunit.client.1.smithi097.stdout:6/222: chown d1/d3/df/c27 12062158 1 2024-09-16T08:57:06.261 INFO:tasks.workunit.client.1.smithi097.stdout:6/223: write d1/d3/d16/f21 [1763936,68774] 0 2024-09-16T08:57:06.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:05 smithi097 ceph-mon[35478]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:57:06.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:05 smithi097 ceph-mon[35478]: Deploying cephadm binary to smithi097 2024-09-16T08:57:06.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:05 smithi097 ceph-mon[35478]: Deploying cephadm binary to smithi031 2024-09-16T08:57:06.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:05 smithi097 ceph-mon[35478]: pgmap v4: 65 pgs: 65 active+clean; 355 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:57:06.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:05 smithi097 ceph-mon[35478]: mgrmap e23: smithi097.mpashf(active, since 2s) 2024-09-16T08:57:06.262 INFO:tasks.workunit.client.1.smithi097.stdout:6/224: unlink d1/dc/c29 0 2024-09-16T08:57:06.268 INFO:tasks.workunit.client.1.smithi097.stdout:2/167: sync 2024-09-16T08:57:06.268 INFO:tasks.workunit.client.1.smithi097.stdout:2/168: truncate d3/d4/d9/f1d 759180 0 2024-09-16T08:57:06.268 INFO:tasks.workunit.client.1.smithi097.stdout:7/189: sync 2024-09-16T08:57:06.268 INFO:tasks.workunit.client.1.smithi097.stdout:3/146: sync 2024-09-16T08:57:06.268 INFO:tasks.workunit.client.1.smithi097.stdout:7/190: dread - d4/d9/d1c/f2c zero size 2024-09-16T08:57:06.269 INFO:tasks.workunit.client.1.smithi097.stdout:2/169: truncate d3/d4/d9/f12 1433807 0 2024-09-16T08:57:06.272 INFO:tasks.workunit.client.1.smithi097.stdout:3/147: mkdir d7/d28/d30 0 2024-09-16T08:57:06.273 INFO:tasks.workunit.client.1.smithi097.stdout:2/170: creat d3/d1b/f26 x:0 0 0 2024-09-16T08:57:06.275 INFO:tasks.workunit.client.1.smithi097.stdout:7/191: dread d4/d9/f24 [0,4194304] 0 2024-09-16T08:57:06.275 INFO:tasks.workunit.client.1.smithi097.stdout:7/192: write d4/d1b/f1e [2090411,66922] 0 2024-09-16T08:57:06.276 INFO:tasks.workunit.client.1.smithi097.stdout:3/148: rename d7/d15/c1f to d7/de/c31 0 2024-09-16T08:57:06.280 INFO:tasks.workunit.client.1.smithi097.stdout:6/225: dwrite d1/f37 [0,4194304] 0 2024-09-16T08:57:06.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/149: dread d7/f21 [0,4194304] 0 2024-09-16T08:57:06.281 INFO:tasks.workunit.client.1.smithi097.stdout:6/226: write d1/d3/d16/f2e [326064,93644] 0 2024-09-16T08:57:06.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/150: creat d7/de/d29/f32 x:0 0 0 2024-09-16T08:57:06.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/151: write d7/f21 [246269,123249] 0 2024-09-16T08:57:06.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/152: chown d7/de/l1a 63 1 2024-09-16T08:57:06.283 INFO:tasks.workunit.client.1.smithi097.stdout:6/227: rmdir d1/d3/d3e 39 2024-09-16T08:57:06.283 INFO:tasks.workunit.client.1.smithi097.stdout:6/228: fdatasync d1/d3/f7 0 2024-09-16T08:57:06.285 INFO:tasks.workunit.client.1.smithi097.stdout:6/229: getdents d1/d3/d16 0 2024-09-16T08:57:06.286 INFO:tasks.workunit.client.1.smithi097.stdout:2/171: write d3/d4/d8/f16 [1153051,60206] 0 2024-09-16T08:57:06.286 INFO:tasks.workunit.client.1.smithi097.stdout:6/230: symlink d1/d3/l46 0 2024-09-16T08:57:06.288 INFO:tasks.workunit.client.1.smithi097.stdout:2/172: creat d3/f27 x:0 0 0 2024-09-16T08:57:06.290 INFO:tasks.workunit.client.1.smithi097.stdout:7/193: dwrite d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:06.291 INFO:tasks.workunit.client.1.smithi097.stdout:2/173: unlink d3/d4/le 0 2024-09-16T08:57:06.291 INFO:tasks.workunit.client.1.smithi097.stdout:2/174: fdatasync d3/d4/dd/f1f 0 2024-09-16T08:57:06.292 INFO:tasks.workunit.client.1.smithi097.stdout:7/194: creat d4/d1b/d36/d43/f44 x:0 0 0 2024-09-16T08:57:06.295 INFO:tasks.workunit.client.1.smithi097.stdout:2/175: unlink d3/d4/d9/f12 0 2024-09-16T08:57:06.297 INFO:tasks.workunit.client.1.smithi097.stdout:7/195: symlink d4/d9/d1c/l45 0 2024-09-16T08:57:06.297 INFO:tasks.workunit.client.1.smithi097.stdout:7/196: creat d4/d1b/d36/f46 x:0 0 0 2024-09-16T08:57:06.300 INFO:tasks.workunit.client.1.smithi097.stdout:7/197: rename d4/d9/dc/ld to d4/l47 0 2024-09-16T08:57:06.300 INFO:tasks.workunit.client.1.smithi097.stdout:7/198: creat d4/d9/dc/f48 x:0 0 0 2024-09-16T08:57:06.300 INFO:tasks.workunit.client.1.smithi097.stdout:7/199: chown d4/d9/d1c/f2c 27 1 2024-09-16T08:57:06.304 INFO:tasks.workunit.client.1.smithi097.stdout:7/200: rename d4/d9/d2b/f3e to d4/d9/dc/f49 0 2024-09-16T08:57:06.304 INFO:tasks.workunit.client.1.smithi097.stdout:3/153: dwrite d7/f2b [0,4194304] 0 2024-09-16T08:57:06.304 INFO:tasks.workunit.client.1.smithi097.stdout:3/154: write d7/de/f23 [248134,49354] 0 2024-09-16T08:57:06.305 INFO:tasks.workunit.client.1.smithi097.stdout:7/201: symlink d4/d9/l4a 0 2024-09-16T08:57:06.308 INFO:tasks.workunit.client.1.smithi097.stdout:3/155: write d7/fa [2627384,130168] 0 2024-09-16T08:57:06.312 INFO:tasks.workunit.client.1.smithi097.stdout:6/231: dwrite d1/dc/d1e/f3a [0,4194304] 0 2024-09-16T08:57:06.316 INFO:tasks.workunit.client.1.smithi097.stdout:2/176: dwrite d3/d4/d9/f1d [0,4194304] 0 2024-09-16T08:57:06.320 INFO:tasks.workunit.client.1.smithi097.stdout:2/177: truncate d3/d4/dd/f1f 1892623 0 2024-09-16T08:57:06.322 INFO:tasks.workunit.client.1.smithi097.stdout:7/202: dwrite d4/d12/d1f/f3b [0,4194304] 0 2024-09-16T08:57:06.322 INFO:tasks.workunit.client.1.smithi097.stdout:6/232: write d1/dc/d1e/f3a [1452376,43146] 0 2024-09-16T08:57:06.322 INFO:tasks.workunit.client.1.smithi097.stdout:2/178: dread d3/d4/d9/f1e [0,4194304] 0 2024-09-16T08:57:06.326 INFO:tasks.workunit.client.1.smithi097.stdout:6/233: unlink d1/d3/l8 0 2024-09-16T08:57:06.326 INFO:tasks.workunit.client.1.smithi097.stdout:6/234: chown d1/c9 174435885 1 2024-09-16T08:57:06.326 INFO:tasks.workunit.client.1.smithi097.stdout:6/235: creat d1/d3/df/f47 x:0 0 0 2024-09-16T08:57:06.327 INFO:tasks.workunit.client.1.smithi097.stdout:2/179: unlink d3/d1b/f26 0 2024-09-16T08:57:06.327 INFO:tasks.workunit.client.1.smithi097.stdout:2/180: write d3/d4/dd/f18 [2907264,116427] 0 2024-09-16T08:57:06.330 INFO:tasks.workunit.client.1.smithi097.stdout:2/181: truncate d3/d4/d8/f25 430063 0 2024-09-16T08:57:06.330 INFO:tasks.workunit.client.1.smithi097.stdout:2/182: stat d3/d4/d8 0 2024-09-16T08:57:06.330 INFO:tasks.workunit.client.1.smithi097.stdout:2/183: write d3/f27 [897143,45575] 0 2024-09-16T08:57:06.336 INFO:tasks.workunit.client.1.smithi097.stdout:2/184: fsync d3/d4/dd/f1f 0 2024-09-16T08:57:06.336 INFO:tasks.workunit.client.1.smithi097.stdout:9/220: sync 2024-09-16T08:57:06.338 INFO:tasks.workunit.client.1.smithi097.stdout:2/185: symlink d3/d4/dd/l28 0 2024-09-16T08:57:06.338 INFO:tasks.workunit.client.1.smithi097.stdout:2/186: stat d3/d4/d8 0 2024-09-16T08:57:06.338 INFO:tasks.workunit.client.1.smithi097.stdout:2/187: creat d3/f29 x:0 0 0 2024-09-16T08:57:06.338 INFO:tasks.workunit.client.1.smithi097.stdout:2/188: creat d3/d4/d8/f2a x:0 0 0 2024-09-16T08:57:06.340 INFO:tasks.workunit.client.1.smithi097.stdout:6/236: dwrite d1/d3/f2c [0,4194304] 0 2024-09-16T08:57:06.343 INFO:tasks.workunit.client.1.smithi097.stdout:6/237: creat d1/d3/d3e/f48 x:0 0 0 2024-09-16T08:57:06.343 INFO:tasks.workunit.client.1.smithi097.stdout:6/238: dread - d1/dc/d3c/f44 zero size 2024-09-16T08:57:06.343 INFO:tasks.workunit.client.1.smithi097.stdout:6/239: truncate d1/d3/df/f47 988159 0 2024-09-16T08:57:06.344 INFO:tasks.workunit.client.1.smithi097.stdout:6/240: symlink d1/d3/d16/l49 0 2024-09-16T08:57:06.344 INFO:tasks.workunit.client.1.smithi097.stdout:6/241: creat d1/d3/df/f4a x:0 0 0 2024-09-16T08:57:06.344 INFO:tasks.workunit.client.1.smithi097.stdout:6/242: dread - d1/d3/df/f36 zero size 2024-09-16T08:57:06.344 INFO:tasks.workunit.client.1.smithi097.stdout:6/243: write d1/d3/f1c [654697,69550] 0 2024-09-16T08:57:06.347 INFO:tasks.workunit.client.1.smithi097.stdout:6/244: getdents d1/d3/d3e 0 2024-09-16T08:57:06.347 INFO:tasks.workunit.client.1.smithi097.stdout:6/245: creat d1/d3/f4b x:0 0 0 2024-09-16T08:57:06.348 INFO:tasks.workunit.client.1.smithi097.stdout:6/246: mknod d1/dc/d3c/c4c 0 2024-09-16T08:57:06.348 INFO:tasks.workunit.client.1.smithi097.stdout:6/247: fsync d1/d3/d16/f2e 0 2024-09-16T08:57:06.348 INFO:tasks.workunit.client.1.smithi097.stdout:6/248: stat d1/dc/f38 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:6/249: creat d1/d3/d16/f4d x:0 0 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:6/250: read d1/d3/d16/f21 [1829416,61323] 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:6/251: creat d1/f4e x:0 0 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:2/189: dread d3/d4/f21 [0,4194304] 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:2/190: write d3/d4/d8/f16 [3079961,109414] 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:2/191: write d3/d4/dd/f18 [2736942,110947] 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:2/192: creat d3/d4/d8/f2b x:0 0 0 2024-09-16T08:57:06.350 INFO:tasks.workunit.client.1.smithi097.stdout:2/193: readlink d3/d4/d9/l1a 0 2024-09-16T08:57:06.351 INFO:tasks.workunit.client.1.smithi097.stdout:6/252: mkdir d1/d3/d16/d4f 0 2024-09-16T08:57:06.371 INFO:tasks.workunit.client.1.smithi097.stdout:6/253: dwrite d1/d3/f3d [0,4194304] 0 2024-09-16T08:57:06.371 INFO:tasks.workunit.client.1.smithi097.stdout:2/194: dwrite d3/d4/d9/ff [0,4194304] 0 2024-09-16T08:57:06.372 INFO:tasks.workunit.client.1.smithi097.stdout:6/254: read - d1/d3/df/f36 zero size 2024-09-16T08:57:06.375 INFO:tasks.workunit.client.1.smithi097.stdout:2/195: unlink d3/d4/d8/f2a 0 2024-09-16T08:57:06.375 INFO:tasks.workunit.client.1.smithi097.stdout:2/196: readlink d3/d4/d9/l1a 0 2024-09-16T08:57:06.381 INFO:tasks.workunit.client.1.smithi097.stdout:2/197: rmdir d3/d4/d8 39 2024-09-16T08:57:06.382 INFO:tasks.workunit.client.1.smithi097.stdout:8/178: sync 2024-09-16T08:57:06.382 INFO:tasks.workunit.client.1.smithi097.stdout:1/215: sync 2024-09-16T08:57:06.384 INFO:tasks.workunit.client.1.smithi097.stdout:8/179: truncate d8/dd/d15/f28 2929828 0 2024-09-16T08:57:06.384 INFO:tasks.workunit.client.1.smithi097.stdout:1/216: symlink d5/l3d 0 2024-09-16T08:57:06.384 INFO:tasks.workunit.client.1.smithi097.stdout:1/217: readlink d5/db/dc/l1e 0 2024-09-16T08:57:06.385 INFO:tasks.workunit.client.1.smithi097.stdout:8/180: link d8/fc d8/dd/d15/d16/f3a 0 2024-09-16T08:57:06.386 INFO:tasks.workunit.client.1.smithi097.stdout:1/218: symlink d5/l3e 0 2024-09-16T08:57:06.389 INFO:tasks.workunit.client.1.smithi097.stdout:1/219: link d5/d17/f24 d5/db/dc/d19/f3f 0 2024-09-16T08:57:06.390 INFO:tasks.workunit.client.1.smithi097.stdout:6/255: dwrite d1/d3/d3e/f48 [0,4194304] 0 2024-09-16T08:57:06.390 INFO:tasks.workunit.client.1.smithi097.stdout:6/256: chown d1/d3/f2f 72691937 1 2024-09-16T08:57:06.392 INFO:tasks.workunit.client.1.smithi097.stdout:8/181: rmdir d8/dd 39 2024-09-16T08:57:06.392 INFO:tasks.workunit.client.1.smithi097.stdout:8/182: stat d8/fa 0 2024-09-16T08:57:06.394 INFO:tasks.workunit.client.1.smithi097.stdout:8/183: symlink d8/dd/d15/d2c/l3b 0 2024-09-16T08:57:06.395 INFO:tasks.workunit.client.1.smithi097.stdout:8/184: creat d8/dd/d15/d16/f3c x:0 0 0 2024-09-16T08:57:06.396 INFO:tasks.workunit.client.1.smithi097.stdout:8/185: mkdir d8/dd/d15/d16/d2d/d3d 0 2024-09-16T08:57:06.396 INFO:tasks.workunit.client.1.smithi097.stdout:8/186: fdatasync d8/f31 0 2024-09-16T08:57:06.397 INFO:tasks.workunit.client.1.smithi097.stdout:2/198: dwrite d3/d4/dd/f1f [0,4194304] 0 2024-09-16T08:57:06.398 INFO:tasks.workunit.client.1.smithi097.stdout:8/187: symlink d8/dd/d15/d16/d2d/l3e 0 2024-09-16T08:57:06.400 INFO:tasks.workunit.client.1.smithi097.stdout:2/199: mknod d3/d1b/c2c 0 2024-09-16T08:57:06.402 INFO:tasks.workunit.client.1.smithi097.stdout:8/188: truncate d8/dd/d15/f21 1823490 0 2024-09-16T08:57:06.402 INFO:tasks.workunit.client.1.smithi097.stdout:8/189: chown d8/dd/d15/d16/f36 292285 1 2024-09-16T08:57:06.402 INFO:tasks.workunit.client.1.smithi097.stdout:8/190: chown d8/dd/c27 153 1 2024-09-16T08:57:06.403 INFO:tasks.workunit.client.1.smithi097.stdout:8/191: mkdir d8/dd/d15/d16/d2d/d3f 0 2024-09-16T08:57:06.404 INFO:tasks.workunit.client.1.smithi097.stdout:8/192: symlink d8/dd/d15/d16/d2d/d3f/l40 0 2024-09-16T08:57:06.405 INFO:tasks.workunit.client.1.smithi097.stdout:6/257: dread d1/f22 [0,4194304] 0 2024-09-16T08:57:06.408 INFO:tasks.workunit.client.1.smithi097.stdout:6/258: rename d1/d3/d16/l39 to d1/d3/df/l50 0 2024-09-16T08:57:06.408 INFO:tasks.workunit.client.1.smithi097.stdout:6/259: stat d1/dc/d3c 0 2024-09-16T08:57:06.408 INFO:tasks.workunit.client.1.smithi097.stdout:8/193: dread d8/dd/d15/d16/f3a [0,4194304] 0 2024-09-16T08:57:06.410 INFO:tasks.workunit.client.1.smithi097.stdout:6/260: dread d1/d3/d16/f2e [0,4194304] 0 2024-09-16T08:57:06.411 INFO:tasks.workunit.client.1.smithi097.stdout:6/261: getdents d1/dc 0 2024-09-16T08:57:06.424 INFO:tasks.workunit.client.1.smithi097.stdout:1/220: dwrite d5/db/f34 [4194304,4194304] 0 2024-09-16T08:57:06.424 INFO:tasks.workunit.client.1.smithi097.stdout:2/200: dwrite d3/d4/d8/f16 [4194304,4194304] 0 2024-09-16T08:57:06.427 INFO:tasks.workunit.client.1.smithi097.stdout:2/201: getdents d3/d4/d9 0 2024-09-16T08:57:06.427 INFO:tasks.workunit.client.1.smithi097.stdout:1/221: dread f4 [0,4194304] 0 2024-09-16T08:57:06.427 INFO:tasks.workunit.client.1.smithi097.stdout:1/222: truncate d5/db/dc/f33 1413931 0 2024-09-16T08:57:06.427 INFO:tasks.workunit.client.1.smithi097.stdout:2/202: truncate d3/d4/d8/f25 254789 0 2024-09-16T08:57:06.428 INFO:tasks.workunit.client.1.smithi097.stdout:6/262: getdents d1/d3/df 0 2024-09-16T08:57:06.428 INFO:tasks.workunit.client.1.smithi097.stdout:6/263: write d1/d3/f2f [4525952,10332] 0 2024-09-16T08:57:06.429 INFO:tasks.workunit.client.1.smithi097.stdout:5/227: sync 2024-09-16T08:57:06.429 INFO:tasks.workunit.client.1.smithi097.stdout:4/221: sync 2024-09-16T08:57:06.429 INFO:tasks.workunit.client.1.smithi097.stdout:0/189: sync 2024-09-16T08:57:06.429 INFO:tasks.workunit.client.1.smithi097.stdout:0/190: stat da/d1a/d26 0 2024-09-16T08:57:06.430 INFO:tasks.workunit.client.1.smithi097.stdout:0/191: truncate da/df/d37/f39 94953 0 2024-09-16T08:57:06.430 INFO:tasks.workunit.client.1.smithi097.stdout:3/156: sync 2024-09-16T08:57:06.430 INFO:tasks.workunit.client.1.smithi097.stdout:5/228: rename db/d18/d20/c35 to db/d18/d20/c4d 0 2024-09-16T08:57:06.430 INFO:tasks.workunit.client.1.smithi097.stdout:5/229: chown db/d1a 39 1 2024-09-16T08:57:06.431 INFO:tasks.workunit.client.1.smithi097.stdout:0/192: getdents da/d1a/d26 0 2024-09-16T08:57:06.431 INFO:tasks.workunit.client.1.smithi097.stdout:0/193: creat da/f3e x:0 0 0 2024-09-16T08:57:06.432 INFO:tasks.workunit.client.1.smithi097.stdout:3/157: mkdir d7/de/d29/d33 0 2024-09-16T08:57:06.433 INFO:tasks.workunit.client.1.smithi097.stdout:5/230: mknod db/d18/d1c/d42/c4e 0 2024-09-16T08:57:06.437 INFO:tasks.workunit.client.1.smithi097.stdout:3/158: symlink d7/d15/l34 0 2024-09-16T08:57:06.437 INFO:tasks.workunit.client.1.smithi097.stdout:0/194: creat da/df/f3f x:0 0 0 2024-09-16T08:57:06.438 INFO:tasks.workunit.client.1.smithi097.stdout:2/203: write d3/d4/d8/f16 [7798967,19533] 0 2024-09-16T08:57:06.438 INFO:tasks.workunit.client.1.smithi097.stdout:2/204: creat d3/d4/f2d x:0 0 0 2024-09-16T08:57:06.438 INFO:tasks.workunit.client.1.smithi097.stdout:2/205: stat d3/f23 0 2024-09-16T08:57:06.438 INFO:tasks.workunit.client.1.smithi097.stdout:1/223: write d5/d17/f24 [1906295,75899] 0 2024-09-16T08:57:06.439 INFO:tasks.workunit.client.1.smithi097.stdout:0/195: symlink da/d1a/d26/l40 0 2024-09-16T08:57:06.442 INFO:tasks.workunit.client.1.smithi097.stdout:2/206: mknod d3/d4/d8/c2e 0 2024-09-16T08:57:06.442 INFO:tasks.workunit.client.1.smithi097.stdout:2/207: truncate d3/d4/f2d 955488 0 2024-09-16T08:57:06.443 INFO:tasks.workunit.client.1.smithi097.stdout:1/224: mknod d5/c40 0 2024-09-16T08:57:06.444 INFO:tasks.workunit.client.1.smithi097.stdout:0/196: read f8 [2398151,121119] 0 2024-09-16T08:57:06.445 INFO:tasks.workunit.client.1.smithi097.stdout:2/208: symlink d3/d4/d8/l2f 0 2024-09-16T08:57:06.445 INFO:tasks.workunit.client.1.smithi097.stdout:1/225: creat d5/db/dc/f41 x:0 0 0 2024-09-16T08:57:06.445 INFO:tasks.workunit.client.1.smithi097.stdout:1/226: write d5/db/f3c [16095,111131] 0 2024-09-16T08:57:06.445 INFO:tasks.workunit.client.1.smithi097.stdout:1/227: dread - d5/db/dc/f41 zero size 2024-09-16T08:57:06.446 INFO:tasks.workunit.client.1.smithi097.stdout:1/228: write d5/f21 [1642270,57068] 0 2024-09-16T08:57:06.446 INFO:tasks.workunit.client.1.smithi097.stdout:1/229: write d5/db/dc/f41 [931033,66336] 0 2024-09-16T08:57:06.446 INFO:tasks.workunit.client.1.smithi097.stdout:1/230: write d5/f21 [1668696,109586] 0 2024-09-16T08:57:06.446 INFO:tasks.workunit.client.1.smithi097.stdout:0/197: mkdir da/d1a/d41 0 2024-09-16T08:57:06.446 INFO:tasks.workunit.client.1.smithi097.stdout:0/198: chown da/df/d1f/d3b 28 1 2024-09-16T08:57:06.446 INFO:tasks.workunit.client.1.smithi097.stdout:1/231: symlink d5/db/d25/l42 0 2024-09-16T08:57:06.446 INFO:tasks.workunit.client.1.smithi097.stdout:1/232: read - d5/f37 zero size 2024-09-16T08:57:06.453 INFO:tasks.workunit.client.1.smithi097.stdout:3/159: getdents d7/de/d29 0 2024-09-16T08:57:06.458 INFO:tasks.workunit.client.1.smithi097.stdout:3/160: rename d7/d15/d20 to d7/de/d29/d35 0 2024-09-16T08:57:06.458 INFO:tasks.workunit.client.1.smithi097.stdout:3/161: getdents d7/de/d29/d33 0 2024-09-16T08:57:06.458 INFO:tasks.workunit.client.1.smithi097.stdout:3/162: fsync d7/f21 0 2024-09-16T08:57:06.462 INFO:tasks.workunit.client.1.smithi097.stdout:0/199: dread da/df/d13/f2f [0,4194304] 0 2024-09-16T08:57:06.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/264: dwrite d1/f4e [0,4194304] 0 2024-09-16T08:57:06.462 INFO:tasks.workunit.client.1.smithi097.stdout:0/200: stat da/d1a/d26/c38 0 2024-09-16T08:57:06.463 INFO:tasks.workunit.client.1.smithi097.stdout:0/201: chown da/c36 2136 1 2024-09-16T08:57:06.463 INFO:tasks.workunit.client.1.smithi097.stdout:0/202: write da/d24/f33 [311901,21637] 0 2024-09-16T08:57:06.463 INFO:tasks.workunit.client.1.smithi097.stdout:0/203: chown da/d1a/d26 127622 1 2024-09-16T08:57:06.465 INFO:tasks.workunit.client.1.smithi097.stdout:4/222: dwrite d0/f3e [0,4194304] 0 2024-09-16T08:57:06.465 INFO:tasks.workunit.client.1.smithi097.stdout:4/223: chown d0/d7/df/f37 0 1 2024-09-16T08:57:06.465 INFO:tasks.workunit.client.1.smithi097.stdout:4/224: truncate d0/d7/f32 680926 0 2024-09-16T08:57:06.465 INFO:tasks.workunit.client.1.smithi097.stdout:4/225: read d0/f2a [4294,83347] 0 2024-09-16T08:57:06.467 INFO:tasks.workunit.client.1.smithi097.stdout:2/209: dwrite d3/f27 [0,4194304] 0 2024-09-16T08:57:06.467 INFO:tasks.workunit.client.1.smithi097.stdout:0/204: write da/df/d13/f2f [3774360,52307] 0 2024-09-16T08:57:06.468 INFO:tasks.workunit.client.1.smithi097.stdout:5/231: dwrite db/d18/d1c/d42/f4c [0,4194304] 0 2024-09-16T08:57:06.469 INFO:tasks.workunit.client.1.smithi097.stdout:1/233: truncate d5/db/f20 302402 0 2024-09-16T08:57:06.470 INFO:tasks.workunit.client.1.smithi097.stdout:2/210: rename d3/d4/d9/l1a to d3/d4/dd/l30 0 2024-09-16T08:57:06.472 INFO:tasks.workunit.client.1.smithi097.stdout:0/205: mkdir da/df/d1f/d42 0 2024-09-16T08:57:06.474 INFO:tasks.workunit.client.1.smithi097.stdout:0/206: dread da/df/f16 [0,4194304] 0 2024-09-16T08:57:06.479 INFO:tasks.workunit.client.1.smithi097.stdout:5/232: symlink db/l4f 0 2024-09-16T08:57:06.479 INFO:tasks.workunit.client.1.smithi097.stdout:1/234: mknod d5/c43 0 2024-09-16T08:57:06.480 INFO:tasks.workunit.client.1.smithi097.stdout:2/211: creat d3/d4/d9/f31 x:0 0 0 2024-09-16T08:57:06.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/207: symlink da/d1a/d41/l43 0 2024-09-16T08:57:06.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/208: write f6 [3344727,14126] 0 2024-09-16T08:57:06.482 INFO:tasks.workunit.client.1.smithi097.stdout:1/235: rename d5/db/dc/d19/c2f to d5/d14/c44 0 2024-09-16T08:57:06.482 INFO:tasks.workunit.client.1.smithi097.stdout:5/233: dread db/d18/d1c/f39 [0,4194304] 0 2024-09-16T08:57:06.482 INFO:tasks.workunit.client.1.smithi097.stdout:5/234: dread - db/d18/f3d zero size 2024-09-16T08:57:06.482 INFO:tasks.workunit.client.1.smithi097.stdout:1/236: write f4 [526302,50582] 0 2024-09-16T08:57:06.483 INFO:tasks.workunit.client.1.smithi097.stdout:0/209: creat da/df/d3d/f44 x:0 0 0 2024-09-16T08:57:06.484 INFO:tasks.workunit.client.1.smithi097.stdout:2/212: rename d3/d4/d9/f31 to d3/f32 0 2024-09-16T08:57:06.485 INFO:tasks.workunit.client.1.smithi097.stdout:1/237: creat d5/db/d25/f45 x:0 0 0 2024-09-16T08:57:06.486 INFO:tasks.workunit.client.1.smithi097.stdout:5/235: mkdir db/d1a/d50 0 2024-09-16T08:57:06.486 INFO:tasks.workunit.client.1.smithi097.stdout:5/236: stat db/l11 0 2024-09-16T08:57:06.486 INFO:tasks.workunit.client.1.smithi097.stdout:5/237: write db/f2e [808846,11618] 0 2024-09-16T08:57:06.486 INFO:tasks.workunit.client.1.smithi097.stdout:1/238: creat d5/db/dc/d19/f46 x:0 0 0 2024-09-16T08:57:06.487 INFO:tasks.workunit.client.1.smithi097.stdout:5/238: truncate db/f14 3917353 0 2024-09-16T08:57:06.491 INFO:tasks.workunit.client.1.smithi097.stdout:2/213: read d3/d4/d9/f1d [3893275,60896] 0 2024-09-16T08:57:06.493 INFO:tasks.workunit.client.1.smithi097.stdout:2/214: link d3/d4/f2d d3/d4/d8/f33 0 2024-09-16T08:57:06.495 INFO:tasks.workunit.client.1.smithi097.stdout:1/239: write d5/db/dc/d19/f29 [3606303,14709] 0 2024-09-16T08:57:06.497 INFO:tasks.workunit.client.1.smithi097.stdout:1/240: truncate d5/f9 4687596 0 2024-09-16T08:57:06.499 INFO:tasks.workunit.client.1.smithi097.stdout:0/210: dread da/d1a/f22 [0,4194304] 0 2024-09-16T08:57:06.500 INFO:tasks.workunit.client.1.smithi097.stdout:0/211: mknod da/c45 0 2024-09-16T08:57:06.501 INFO:tasks.workunit.client.1.smithi097.stdout:0/212: truncate da/f1b 292727 0 2024-09-16T08:57:06.503 INFO:tasks.workunit.client.1.smithi097.stdout:0/213: rename da/d24/f20 to da/d24/f46 0 2024-09-16T08:57:06.503 INFO:tasks.workunit.client.1.smithi097.stdout:0/214: write f6 [2561064,76390] 0 2024-09-16T08:57:06.504 INFO:tasks.workunit.client.1.smithi097.stdout:2/215: read d3/d4/d9/ff [3745430,77138] 0 2024-09-16T08:57:06.507 INFO:tasks.workunit.client.1.smithi097.stdout:0/215: unlink da/df/d13/cd 0 2024-09-16T08:57:06.507 INFO:tasks.workunit.client.1.smithi097.stdout:5/239: dwrite db/d1a/f1f [0,4194304] 0 2024-09-16T08:57:06.507 INFO:tasks.workunit.client.1.smithi097.stdout:5/240: stat db/fe 0 2024-09-16T08:57:06.510 INFO:tasks.workunit.client.1.smithi097.stdout:2/216: mkdir d3/d4/d8/d34 0 2024-09-16T08:57:06.510 INFO:tasks.workunit.client.1.smithi097.stdout:2/217: write d3/d4/d9/f1e [636426,7627] 0 2024-09-16T08:57:06.510 INFO:tasks.workunit.client.1.smithi097.stdout:2/218: stat d3/d4/d8/f2b 0 2024-09-16T08:57:06.516 INFO:tasks.workunit.client.1.smithi097.stdout:5/241: rmdir db/d18/d20/d2b 39 2024-09-16T08:57:06.518 INFO:tasks.workunit.client.1.smithi097.stdout:2/219: rename d3/d4/d9/f1d to d3/d4/dd/f35 0 2024-09-16T08:57:06.530 INFO:tasks.workunit.client.1.smithi097.stdout:2/220: rmdir d3/d4/dd 39 2024-09-16T08:57:06.532 INFO:tasks.workunit.client.1.smithi097.stdout:2/221: rename d3/f29 to d3/d4/d9/f36 0 2024-09-16T08:57:06.532 INFO:tasks.workunit.client.1.smithi097.stdout:2/222: chown d3/f27 3 1 2024-09-16T08:57:06.532 INFO:tasks.workunit.client.1.smithi097.stdout:7/203: sync 2024-09-16T08:57:06.532 INFO:tasks.workunit.client.1.smithi097.stdout:9/221: sync 2024-09-16T08:57:06.532 INFO:tasks.workunit.client.1.smithi097.stdout:8/194: sync 2024-09-16T08:57:06.532 INFO:tasks.workunit.client.1.smithi097.stdout:2/223: rmdir d3/d4/d8 39 2024-09-16T08:57:06.532 INFO:tasks.workunit.client.1.smithi097.stdout:2/224: truncate d3/d4/dd/f1f 4274083 0 2024-09-16T08:57:06.533 INFO:tasks.workunit.client.1.smithi097.stdout:2/225: creat d3/f37 x:0 0 0 2024-09-16T08:57:06.533 INFO:tasks.workunit.client.1.smithi097.stdout:7/204: truncate d4/d9/dc/f49 21861 0 2024-09-16T08:57:06.535 INFO:tasks.workunit.client.1.smithi097.stdout:8/195: creat d8/dd/d15/d2c/f41 x:0 0 0 2024-09-16T08:57:06.535 INFO:tasks.workunit.client.1.smithi097.stdout:9/222: truncate d2/f13 929562 0 2024-09-16T08:57:06.535 INFO:tasks.workunit.client.1.smithi097.stdout:5/242: dwrite db/f2e [0,4194304] 0 2024-09-16T08:57:06.537 INFO:tasks.workunit.client.1.smithi097.stdout:8/196: mkdir d8/dd/d42 0 2024-09-16T08:57:06.537 INFO:tasks.workunit.client.1.smithi097.stdout:0/216: dwrite da/f3e [0,4194304] 0 2024-09-16T08:57:06.537 INFO:tasks.workunit.client.1.smithi097.stdout:0/217: stat da/d1a/d26/c38 0 2024-09-16T08:57:06.537 INFO:tasks.workunit.client.1.smithi097.stdout:1/241: dwrite d5/db/dc/d19/f29 [4194304,4194304] 0 2024-09-16T08:57:06.537 INFO:tasks.workunit.client.1.smithi097.stdout:0/218: dread - da/d1a/f3a zero size 2024-09-16T08:57:06.539 INFO:tasks.workunit.client.1.smithi097.stdout:0/219: read da/d24/f46 [126129,126728] 0 2024-09-16T08:57:06.540 INFO:tasks.workunit.client.1.smithi097.stdout:8/197: unlink d8/dd/d15/d16/f36 0 2024-09-16T08:57:06.541 INFO:tasks.workunit.client.1.smithi097.stdout:1/242: mknod d5/d2c/d3b/c47 0 2024-09-16T08:57:06.541 INFO:tasks.workunit.client.1.smithi097.stdout:1/243: truncate d5/f9 4890353 0 2024-09-16T08:57:06.541 INFO:tasks.workunit.client.1.smithi097.stdout:1/244: creat d5/db/dc/f48 x:0 0 0 2024-09-16T08:57:06.542 INFO:tasks.workunit.client.1.smithi097.stdout:0/220: unlink da/df/d13/f2f 0 2024-09-16T08:57:06.543 INFO:tasks.workunit.client.1.smithi097.stdout:7/205: write d4/d9/dc/f49 [1022506,109139] 0 2024-09-16T08:57:06.543 INFO:tasks.workunit.client.1.smithi097.stdout:7/206: creat d4/d1b/f4b x:0 0 0 2024-09-16T08:57:06.546 INFO:tasks.workunit.client.1.smithi097.stdout:1/245: getdents d5 0 2024-09-16T08:57:06.546 INFO:tasks.workunit.client.1.smithi097.stdout:1/246: fdatasync d5/f37 0 2024-09-16T08:57:06.546 INFO:tasks.workunit.client.1.smithi097.stdout:1/247: fdatasync f4 0 2024-09-16T08:57:06.547 INFO:tasks.workunit.client.1.smithi097.stdout:9/223: dread d2/d5/f11 [0,4194304] 0 2024-09-16T08:57:06.547 INFO:tasks.workunit.client.1.smithi097.stdout:8/198: write d8/f1e [4003899,67701] 0 2024-09-16T08:57:06.547 INFO:tasks.workunit.client.1.smithi097.stdout:8/199: read d8/f24 [1743038,38453] 0 2024-09-16T08:57:06.547 INFO:tasks.workunit.client.1.smithi097.stdout:8/200: stat d8/dd/d15/d16/d2d/f35 0 2024-09-16T08:57:06.548 INFO:tasks.workunit.client.1.smithi097.stdout:0/221: mkdir da/d1a/d26/d47 0 2024-09-16T08:57:06.548 INFO:tasks.workunit.client.1.smithi097.stdout:7/207: truncate f2 1667348 0 2024-09-16T08:57:06.548 INFO:tasks.workunit.client.1.smithi097.stdout:7/208: write d4/d1b/d36/f3a [5188850,41870] 0 2024-09-16T08:57:06.550 INFO:tasks.workunit.client.1.smithi097.stdout:1/248: symlink d5/db/d25/l49 0 2024-09-16T08:57:06.550 INFO:tasks.workunit.client.1.smithi097.stdout:0/222: symlink da/df/d1f/l48 0 2024-09-16T08:57:06.550 INFO:tasks.workunit.client.1.smithi097.stdout:1/249: stat d5/db/f3c 0 2024-09-16T08:57:06.550 INFO:tasks.workunit.client.1.smithi097.stdout:0/223: creat da/d1a/f49 x:0 0 0 2024-09-16T08:57:06.550 INFO:tasks.workunit.client.1.smithi097.stdout:2/226: dwrite d3/d4/d8/f16 [0,4194304] 0 2024-09-16T08:57:06.552 INFO:tasks.workunit.client.1.smithi097.stdout:0/224: symlink da/df/d13/l4a 0 2024-09-16T08:57:06.553 INFO:tasks.workunit.client.1.smithi097.stdout:1/250: dread d5/f21 [0,4194304] 0 2024-09-16T08:57:06.555 INFO:tasks.workunit.client.1.smithi097.stdout:1/251: dread d5/db/f20 [0,4194304] 0 2024-09-16T08:57:06.556 INFO:tasks.workunit.client.1.smithi097.stdout:2/227: mknod d3/d4/d8/d34/c38 0 2024-09-16T08:57:06.556 INFO:tasks.workunit.client.1.smithi097.stdout:1/252: read d5/f21 [357877,70861] 0 2024-09-16T08:57:06.556 INFO:tasks.workunit.client.1.smithi097.stdout:1/253: symlink d5/d14/l4a 0 2024-09-16T08:57:06.557 INFO:tasks.workunit.client.1.smithi097.stdout:1/254: write d5/db/d25/f45 [614196,112375] 0 2024-09-16T08:57:06.557 INFO:tasks.workunit.client.1.smithi097.stdout:1/255: readlink d5/db/dc/d19/l1f 0 2024-09-16T08:57:06.564 INFO:tasks.workunit.client.1.smithi097.stdout:8/201: write d8/fa [4094926,66716] 0 2024-09-16T08:57:06.564 INFO:tasks.workunit.client.1.smithi097.stdout:8/202: stat d8/dd/d15/d16/l2e 0 2024-09-16T08:57:06.569 INFO:tasks.workunit.client.1.smithi097.stdout:2/228: read d3/f27 [2176902,11950] 0 2024-09-16T08:57:06.572 INFO:tasks.workunit.client.1.smithi097.stdout:2/229: creat d3/d4/d8/f39 x:0 0 0 2024-09-16T08:57:06.572 INFO:tasks.workunit.client.1.smithi097.stdout:2/230: creat d3/f3a x:0 0 0 2024-09-16T08:57:06.575 INFO:tasks.workunit.client.1.smithi097.stdout:8/203: read d8/dd/f1f [1189837,51051] 0 2024-09-16T08:57:06.577 INFO:tasks.workunit.client.1.smithi097.stdout:8/204: rmdir d8 39 2024-09-16T08:57:06.579 INFO:tasks.workunit.client.1.smithi097.stdout:8/205: creat d8/dd/d15/d16/d2d/d3d/f43 x:0 0 0 2024-09-16T08:57:06.579 INFO:tasks.workunit.client.1.smithi097.stdout:8/206: write d8/fc [973369,81424] 0 2024-09-16T08:57:06.579 INFO:tasks.workunit.client.1.smithi097.stdout:8/207: write d8/fc [70461,73926] 0 2024-09-16T08:57:06.580 INFO:tasks.workunit.client.1.smithi097.stdout:8/208: symlink d8/dd/d15/d16/d2d/d3f/l44 0 2024-09-16T08:57:06.581 INFO:tasks.workunit.client.1.smithi097.stdout:8/209: link d8/dd/d15/d16/d2d/l3e d8/dd/d15/d2c/l45 0 2024-09-16T08:57:06.581 INFO:tasks.workunit.client.1.smithi097.stdout:8/210: getdents d8/dd/d15/d16/d2d 0 2024-09-16T08:57:06.583 INFO:tasks.workunit.client.1.smithi097.stdout:8/211: unlink d8/dd/d15/d16/l2e 0 2024-09-16T08:57:06.583 INFO:tasks.workunit.client.1.smithi097.stdout:8/212: stat d8/dd/d15/d16/d2d/d3d/f43 0 2024-09-16T08:57:06.583 INFO:tasks.workunit.client.1.smithi097.stdout:8/213: write d8/dd/f1f [3570113,91624] 0 2024-09-16T08:57:06.583 INFO:tasks.workunit.client.1.smithi097.stdout:7/209: dwrite d4/d9/d1c/f2c [0,4194304] 0 2024-09-16T08:57:06.584 INFO:tasks.workunit.client.1.smithi097.stdout:8/214: mknod d8/dd/d42/c46 0 2024-09-16T08:57:06.584 INFO:tasks.workunit.client.1.smithi097.stdout:8/215: write d8/dd/d15/d16/d2d/d3d/f43 [213666,80193] 0 2024-09-16T08:57:06.586 INFO:tasks.workunit.client.1.smithi097.stdout:7/210: creat d4/d27/f4c x:0 0 0 2024-09-16T08:57:06.586 INFO:tasks.workunit.client.1.smithi097.stdout:7/211: chown d4/d1b/d36/d43/f44 1243462 1 2024-09-16T08:57:06.586 INFO:tasks.workunit.client.1.smithi097.stdout:7/212: creat d4/d9/d2b/f4d x:0 0 0 2024-09-16T08:57:06.591 INFO:tasks.workunit.client.1.smithi097.stdout:7/213: dread d4/d1b/f1e [0,4194304] 0 2024-09-16T08:57:06.591 INFO:tasks.workunit.client.1.smithi097.stdout:7/214: getdents d4/d12/d1f/d22 0 2024-09-16T08:57:06.592 INFO:tasks.workunit.client.1.smithi097.stdout:7/215: chown d4/d9/d1c/f2c 2156 1 2024-09-16T08:57:06.592 INFO:tasks.workunit.client.1.smithi097.stdout:0/225: dwrite f6 [0,4194304] 0 2024-09-16T08:57:06.592 INFO:tasks.workunit.client.1.smithi097.stdout:1/256: dwrite d5/db/f34 [0,4194304] 0 2024-09-16T08:57:06.593 INFO:tasks.workunit.client.1.smithi097.stdout:2/231: dwrite d3/d4/dd/f35 [0,4194304] 0 2024-09-16T08:57:06.595 INFO:tasks.workunit.client.1.smithi097.stdout:0/226: mkdir da/d24/d4b 0 2024-09-16T08:57:06.596 INFO:tasks.workunit.client.1.smithi097.stdout:1/257: creat d5/d2c/f4b x:0 0 0 2024-09-16T08:57:06.596 INFO:tasks.workunit.client.1.smithi097.stdout:1/258: write d5/d17/f26 [1203181,20064] 0 2024-09-16T08:57:06.597 INFO:tasks.workunit.client.1.smithi097.stdout:1/259: creat d5/d2c/f4c x:0 0 0 2024-09-16T08:57:06.597 INFO:tasks.workunit.client.1.smithi097.stdout:8/216: dread d8/dd/f1f [0,4194304] 0 2024-09-16T08:57:06.597 INFO:tasks.workunit.client.1.smithi097.stdout:8/217: truncate d8/dd/f1f 4562174 0 2024-09-16T08:57:06.600 INFO:tasks.workunit.client.1.smithi097.stdout:2/232: rename d3/d4/d9/l17 to d3/d4/d9/l3b 0 2024-09-16T08:57:06.600 INFO:tasks.workunit.client.1.smithi097.stdout:2/233: fsync d3/d4/d9/ff 0 2024-09-16T08:57:06.600 INFO:tasks.workunit.client.1.smithi097.stdout:2/234: readlink d3/d4/la 0 2024-09-16T08:57:06.600 INFO:tasks.workunit.client.1.smithi097.stdout:4/226: sync 2024-09-16T08:57:06.600 INFO:tasks.workunit.client.1.smithi097.stdout:3/163: sync 2024-09-16T08:57:06.601 INFO:tasks.workunit.client.1.smithi097.stdout:0/227: unlink da/d1a/d26/l40 0 2024-09-16T08:57:06.601 INFO:tasks.workunit.client.1.smithi097.stdout:6/265: sync 2024-09-16T08:57:06.601 INFO:tasks.workunit.client.1.smithi097.stdout:7/216: write d4/d7/f11 [2075502,24039] 0 2024-09-16T08:57:06.604 INFO:tasks.workunit.client.1.smithi097.stdout:8/218: mkdir d8/dd/d15/d16/d2d/d3d/d47 0 2024-09-16T08:57:06.604 INFO:tasks.workunit.client.1.smithi097.stdout:8/219: truncate d8/dd/f26 356840 0 2024-09-16T08:57:06.606 INFO:tasks.workunit.client.1.smithi097.stdout:3/164: rename d7/de/d29/d33 to d7/d28/d36 0 2024-09-16T08:57:06.607 INFO:tasks.workunit.client.1.smithi097.stdout:0/228: mkdir da/d24/d4c 0 2024-09-16T08:57:06.607 INFO:tasks.workunit.client.1.smithi097.stdout:1/260: dread d5/d2c/f2e [0,4194304] 0 2024-09-16T08:57:06.607 INFO:tasks.workunit.client.1.smithi097.stdout:6/266: symlink d1/d3/df/l51 0 2024-09-16T08:57:06.608 INFO:tasks.workunit.client.1.smithi097.stdout:7/217: mknod d4/d9/d2b/c4e 0 2024-09-16T08:57:06.608 INFO:tasks.workunit.client.1.smithi097.stdout:4/227: dread d0/d7/df/d15/f11 [0,4194304] 0 2024-09-16T08:57:06.609 INFO:tasks.workunit.client.1.smithi097.stdout:8/220: mkdir d8/dd/d15/d2c/d48 0 2024-09-16T08:57:06.610 INFO:tasks.workunit.client.1.smithi097.stdout:8/221: read d8/dd/f26 [265348,107762] 0 2024-09-16T08:57:06.610 INFO:tasks.workunit.client.1.smithi097.stdout:8/222: truncate d8/dd/d15/f28 3184864 0 2024-09-16T08:57:06.610 INFO:tasks.workunit.client.1.smithi097.stdout:8/223: chown d8/dd/f26 5 1 2024-09-16T08:57:06.610 INFO:tasks.workunit.client.1.smithi097.stdout:3/165: read - d7/de/d29/f2d zero size 2024-09-16T08:57:06.613 INFO:tasks.workunit.client.1.smithi097.stdout:3/166: dread d7/de/f23 [0,4194304] 0 2024-09-16T08:57:06.613 INFO:tasks.workunit.client.1.smithi097.stdout:3/167: chown d7/l13 32124 1 2024-09-16T08:57:06.613 INFO:tasks.workunit.client.1.smithi097.stdout:3/168: readlink l5 0 2024-09-16T08:57:06.613 INFO:tasks.workunit.client.1.smithi097.stdout:1/261: unlink d5/db/dc/l1e 0 2024-09-16T08:57:06.613 INFO:tasks.workunit.client.1.smithi097.stdout:1/262: creat d5/db/dc/d19/f4d x:0 0 0 2024-09-16T08:57:06.613 INFO:tasks.workunit.client.1.smithi097.stdout:1/263: readlink d5/l1b 0 2024-09-16T08:57:06.613 INFO:tasks.workunit.client.1.smithi097.stdout:1/264: creat d5/d2c/f4e x:0 0 0 2024-09-16T08:57:06.615 INFO:tasks.workunit.client.1.smithi097.stdout:6/267: truncate d1/d3/df/f47 269234 0 2024-09-16T08:57:06.616 INFO:tasks.workunit.client.1.smithi097.stdout:7/218: creat d4/d12/f4f x:0 0 0 2024-09-16T08:57:06.619 INFO:tasks.workunit.client.1.smithi097.stdout:4/228: creat d0/d7/d52/d26/d3d/f55 x:0 0 0 2024-09-16T08:57:06.619 INFO:tasks.workunit.client.1.smithi097.stdout:4/229: chown d0/d7/df/l4a 7276 1 2024-09-16T08:57:06.619 INFO:tasks.workunit.client.1.smithi097.stdout:8/224: rmdir d8/dd/d15/d16/d2d 39 2024-09-16T08:57:06.623 INFO:tasks.workunit.client.1.smithi097.stdout:1/265: truncate d5/db/dc/f41 329347 0 2024-09-16T08:57:06.624 INFO:tasks.workunit.client.1.smithi097.stdout:6/268: link d1/d3/d16/c42 d1/d3/d16/c52 0 2024-09-16T08:57:06.624 INFO:tasks.workunit.client.1.smithi097.stdout:6/269: fdatasync d1/d3/f1c 0 2024-09-16T08:57:06.625 INFO:tasks.workunit.client.1.smithi097.stdout:2/235: dwrite d3/f27 [0,4194304] 0 2024-09-16T08:57:06.625 INFO:tasks.workunit.client.1.smithi097.stdout:2/236: chown d3/d4/d8 14949244 1 2024-09-16T08:57:06.626 INFO:tasks.workunit.client.1.smithi097.stdout:4/230: rename d0/d7/d52/d26/d3d/f55 to d0/d36/f56 0 2024-09-16T08:57:06.626 INFO:tasks.workunit.client.1.smithi097.stdout:4/231: readlink d0/d7/d52/d26/d3d/l53 0 2024-09-16T08:57:06.626 INFO:tasks.workunit.client.1.smithi097.stdout:4/232: truncate d0/d7/f24 4387906 0 2024-09-16T08:57:06.626 INFO:tasks.workunit.client.1.smithi097.stdout:4/233: truncate d0/d7/df/f1e 1980128 0 2024-09-16T08:57:06.626 INFO:tasks.workunit.client.1.smithi097.stdout:2/237: dread d3/d4/d8/f33 [0,4194304] 0 2024-09-16T08:57:06.627 INFO:tasks.workunit.client.1.smithi097.stdout:8/225: link d8/dd/f2f d8/dd/d15/d16/d2d/f49 0 2024-09-16T08:57:06.629 INFO:tasks.workunit.client.1.smithi097.stdout:0/229: dwrite da/df/d3d/f44 [0,4194304] 0 2024-09-16T08:57:06.629 INFO:tasks.workunit.client.1.smithi097.stdout:0/230: chown f6 109 1 2024-09-16T08:57:06.630 INFO:tasks.workunit.client.1.smithi097.stdout:6/270: truncate d1/f22 1112024 0 2024-09-16T08:57:06.634 INFO:tasks.workunit.client.1.smithi097.stdout:3/169: rmdir d7/de 39 2024-09-16T08:57:06.636 INFO:tasks.workunit.client.1.smithi097.stdout:4/234: mknod d0/d39/c57 0 2024-09-16T08:57:06.636 INFO:tasks.workunit.client.1.smithi097.stdout:4/235: stat d0/d36/f47 0 2024-09-16T08:57:06.636 INFO:tasks.workunit.client.1.smithi097.stdout:4/236: write d0/d36/f56 [11311,95696] 0 2024-09-16T08:57:06.636 INFO:tasks.workunit.client.1.smithi097.stdout:4/237: chown d0/d7/d52/d26 1923183 1 2024-09-16T08:57:06.636 INFO:tasks.workunit.client.1.smithi097.stdout:2/238: creat d3/d4/d8/d34/f3c x:0 0 0 2024-09-16T08:57:06.639 INFO:tasks.workunit.client.1.smithi097.stdout:0/231: mknod da/df/d13/c4d 0 2024-09-16T08:57:06.640 INFO:tasks.workunit.client.1.smithi097.stdout:1/266: getdents d5/db/dc 0 2024-09-16T08:57:06.641 INFO:tasks.workunit.client.1.smithi097.stdout:1/267: chown d5/d17/f24 1032 1 2024-09-16T08:57:06.646 INFO:tasks.workunit.client.1.smithi097.stdout:6/271: symlink d1/d3/d3e/l53 0 2024-09-16T08:57:06.647 INFO:tasks.workunit.client.1.smithi097.stdout:3/170: creat d7/d28/d36/f37 x:0 0 0 2024-09-16T08:57:06.649 INFO:tasks.workunit.client.1.smithi097.stdout:4/238: mknod d0/d23/c58 0 2024-09-16T08:57:06.651 INFO:tasks.workunit.client.1.smithi097.stdout:0/232: creat da/df/d1f/d3b/f4e x:0 0 0 2024-09-16T08:57:06.653 INFO:tasks.workunit.client.1.smithi097.stdout:1/268: unlink d5/f37 0 2024-09-16T08:57:06.653 INFO:tasks.workunit.client.1.smithi097.stdout:1/269: fsync d5/d2c/f2e 0 2024-09-16T08:57:06.657 INFO:tasks.workunit.client.1.smithi097.stdout:8/226: dwrite d8/fa [0,4194304] 0 2024-09-16T08:57:06.657 INFO:tasks.workunit.client.1.smithi097.stdout:2/239: dwrite d3/d4/d8/f39 [0,4194304] 0 2024-09-16T08:57:06.657 INFO:tasks.workunit.client.1.smithi097.stdout:6/272: rename d1/d3/d3e/l53 to d1/l54 0 2024-09-16T08:57:06.658 INFO:tasks.workunit.client.1.smithi097.stdout:6/273: readlink d1/d3/df/l50 0 2024-09-16T08:57:06.658 INFO:tasks.workunit.client.1.smithi097.stdout:3/171: mknod d7/de/c38 0 2024-09-16T08:57:06.659 INFO:tasks.workunit.client.1.smithi097.stdout:4/239: creat d0/d7/d52/f59 x:0 0 0 2024-09-16T08:57:06.659 INFO:tasks.workunit.client.1.smithi097.stdout:4/240: stat d0/d7/d52/f50 0 2024-09-16T08:57:06.659 INFO:tasks.workunit.client.1.smithi097.stdout:0/233: symlink da/l4f 0 2024-09-16T08:57:06.659 INFO:tasks.workunit.client.1.smithi097.stdout:9/224: sync 2024-09-16T08:57:06.659 INFO:tasks.workunit.client.1.smithi097.stdout:5/243: sync 2024-09-16T08:57:06.659 INFO:tasks.workunit.client.1.smithi097.stdout:9/225: readlink d2/d5/d3a/d3c/l44 0 2024-09-16T08:57:06.660 INFO:tasks.workunit.client.1.smithi097.stdout:0/234: dread da/d1a/f22 [0,4194304] 0 2024-09-16T08:57:06.661 INFO:tasks.workunit.client.1.smithi097.stdout:8/227: mkdir d8/dd/d15/d2c/d4a 0 2024-09-16T08:57:06.662 INFO:tasks.workunit.client.1.smithi097.stdout:8/228: creat d8/dd/d15/d2c/f4b x:0 0 0 2024-09-16T08:57:06.662 INFO:tasks.workunit.client.1.smithi097.stdout:8/229: rename d8/dd/d15 to d8/dd/d15/d2c/d48/d4c 22 2024-09-16T08:57:06.663 INFO:tasks.workunit.client.1.smithi097.stdout:4/241: symlink d0/d7/df/d15/d40/l5a 0 2024-09-16T08:57:06.664 INFO:tasks.workunit.client.1.smithi097.stdout:3/172: symlink d7/d28/d36/l39 0 2024-09-16T08:57:06.665 INFO:tasks.workunit.client.1.smithi097.stdout:5/244: creat db/d18/d1c/d42/d48/f51 x:0 0 0 2024-09-16T08:57:06.667 INFO:tasks.workunit.client.1.smithi097.stdout:9/226: symlink d2/d30/l48 0 2024-09-16T08:57:06.668 INFO:tasks.workunit.client.1.smithi097.stdout:1/270: symlink d5/db/dc/d19/l4f 0 2024-09-16T08:57:06.668 INFO:tasks.workunit.client.1.smithi097.stdout:1/271: dread - d5/db/dc/f48 zero size 2024-09-16T08:57:06.668 INFO:tasks.workunit.client.1.smithi097.stdout:1/272: truncate d5/d17/f26 4740735 0 2024-09-16T08:57:06.669 INFO:tasks.workunit.client.1.smithi097.stdout:0/235: creat da/d1a/d41/f50 x:0 0 0 2024-09-16T08:57:06.673 INFO:tasks.workunit.client.1.smithi097.stdout:4/242: rename d0/d7/f32 to d0/d7/f5b 0 2024-09-16T08:57:06.674 INFO:tasks.workunit.client.1.smithi097.stdout:9/227: link d2/l15 d2/d30/l49 0 2024-09-16T08:57:06.675 INFO:tasks.workunit.client.1.smithi097.stdout:0/236: mknod da/c51 0 2024-09-16T08:57:06.677 INFO:tasks.workunit.client.1.smithi097.stdout:3/173: dread d7/f2b [0,4194304] 0 2024-09-16T08:57:06.677 INFO:tasks.workunit.client.1.smithi097.stdout:3/174: readlink d7/d15/l34 0 2024-09-16T08:57:06.678 INFO:tasks.workunit.client.1.smithi097.stdout:3/175: truncate d7/f2b 1920714 0 2024-09-16T08:57:06.678 INFO:tasks.workunit.client.1.smithi097.stdout:3/176: fsync d7/de/f23 0 2024-09-16T08:57:06.678 INFO:tasks.workunit.client.1.smithi097.stdout:5/245: dread db/d18/d1c/d42/f4c [0,4194304] 0 2024-09-16T08:57:06.679 INFO:tasks.workunit.client.1.smithi097.stdout:3/177: read - d7/de/d29/f32 zero size 2024-09-16T08:57:06.679 INFO:tasks.workunit.client.1.smithi097.stdout:8/230: write d8/dd/f2f [643548,10573] 0 2024-09-16T08:57:06.681 INFO:tasks.workunit.client.1.smithi097.stdout:5/246: unlink l5 0 2024-09-16T08:57:06.683 INFO:tasks.workunit.client.1.smithi097.stdout:8/231: dread d8/f31 [0,4194304] 0 2024-09-16T08:57:06.684 INFO:tasks.workunit.client.1.smithi097.stdout:3/178: mknod d7/de/d29/c3a 0 2024-09-16T08:57:06.685 INFO:tasks.workunit.client.1.smithi097.stdout:5/247: symlink db/d18/l52 0 2024-09-16T08:57:06.685 INFO:tasks.workunit.client.1.smithi097.stdout:5/248: creat db/d18/d1c/d42/d48/f53 x:0 0 0 2024-09-16T08:57:06.685 INFO:tasks.workunit.client.1.smithi097.stdout:1/273: write d5/db/dc/d19/f29 [5487807,89870] 0 2024-09-16T08:57:06.687 INFO:tasks.workunit.client.1.smithi097.stdout:6/274: dwrite d1/d3/f2c [0,4194304] 0 2024-09-16T08:57:06.689 INFO:tasks.workunit.client.1.smithi097.stdout:4/243: dread d0/fe [0,4194304] 0 2024-09-16T08:57:06.690 INFO:tasks.workunit.client.1.smithi097.stdout:4/244: chown d0/d36/d4c 2 1 2024-09-16T08:57:06.690 INFO:tasks.workunit.client.1.smithi097.stdout:4/245: write d0/d7/d52/f4b [642919,79657] 0 2024-09-16T08:57:06.692 INFO:tasks.workunit.client.1.smithi097.stdout:2/240: dwrite d3/f27 [0,4194304] 0 2024-09-16T08:57:06.697 INFO:tasks.workunit.client.1.smithi097.stdout:5/249: unlink db/d18/d20/d2b/f2d 0 2024-09-16T08:57:06.702 INFO:tasks.workunit.client.1.smithi097.stdout:6/275: creat d1/d3/d16/d4f/f55 x:0 0 0 2024-09-16T08:57:06.702 INFO:tasks.workunit.client.1.smithi097.stdout:6/276: write d1/d3/d16/f2e [918583,44564] 0 2024-09-16T08:57:06.706 INFO:tasks.workunit.client.1.smithi097.stdout:5/250: unlink c7 0 2024-09-16T08:57:06.709 INFO:tasks.workunit.client.1.smithi097.stdout:6/277: rename d1/d3/df/f2d to d1/d3/df/f56 0 2024-09-16T08:57:06.711 INFO:tasks.workunit.client.1.smithi097.stdout:1/274: rename d5/db/d25/l49 to d5/db/l50 0 2024-09-16T08:57:06.714 INFO:tasks.workunit.client.1.smithi097.stdout:1/275: dread d5/db/d25/f45 [0,4194304] 0 2024-09-16T08:57:06.716 INFO:tasks.workunit.client.1.smithi097.stdout:4/246: fsync d0/d7/f5b 0 2024-09-16T08:57:06.716 INFO:tasks.workunit.client.1.smithi097.stdout:4/247: creat d0/d35/f5c x:0 0 0 2024-09-16T08:57:06.717 INFO:tasks.workunit.client.1.smithi097.stdout:1/276: symlink d5/l51 0 2024-09-16T08:57:06.717 INFO:tasks.workunit.client.1.smithi097.stdout:1/277: write d5/db/dc/f41 [654680,12803] 0 2024-09-16T08:57:06.719 INFO:tasks.workunit.client.1.smithi097.stdout:6/278: write d1/d3/f2c [364018,92441] 0 2024-09-16T08:57:06.719 INFO:tasks.workunit.client.1.smithi097.stdout:6/279: chown d1/c28 240 1 2024-09-16T08:57:06.720 INFO:tasks.workunit.client.1.smithi097.stdout:4/248: mknod d0/d23/c5d 0 2024-09-16T08:57:06.720 INFO:tasks.workunit.client.1.smithi097.stdout:4/249: creat d0/d7/f5e x:0 0 0 2024-09-16T08:57:06.723 INFO:tasks.workunit.client.1.smithi097.stdout:8/232: dwrite d8/dd/f2f [4194304,4194304] 0 2024-09-16T08:57:06.723 INFO:tasks.workunit.client.1.smithi097.stdout:2/241: dwrite d3/d4/d8/f16 [4194304,4194304] 0 2024-09-16T08:57:06.724 INFO:tasks.workunit.client.1.smithi097.stdout:3/179: dwrite d7/de/d29/d35/f22 [0,4194304] 0 2024-09-16T08:57:06.726 INFO:tasks.workunit.client.1.smithi097.stdout:9/228: dwrite d2/f28 [0,4194304] 0 2024-09-16T08:57:06.726 INFO:tasks.workunit.client.1.smithi097.stdout:0/237: dwrite da/f3e [0,4194304] 0 2024-09-16T08:57:06.726 INFO:tasks.workunit.client.1.smithi097.stdout:6/280: write d1/d3/f2f [2271328,55469] 0 2024-09-16T08:57:06.734 INFO:tasks.workunit.client.1.smithi097.stdout:3/180: rename d7/l25 to d7/de/l3b 0 2024-09-16T08:57:06.734 INFO:tasks.workunit.client.1.smithi097.stdout:3/181: chown d7/f21 77042 1 2024-09-16T08:57:06.734 INFO:tasks.workunit.client.1.smithi097.stdout:2/242: dread d3/d4/dd/f1f [0,4194304] 0 2024-09-16T08:57:06.746 INFO:tasks.workunit.client.1.smithi097.stdout:0/238: truncate da/d1a/f22 825557 0 2024-09-16T08:57:06.748 INFO:tasks.workunit.client.1.smithi097.stdout:0/239: dread da/d24/f46 [0,4194304] 0 2024-09-16T08:57:06.748 INFO:tasks.workunit.client.1.smithi097.stdout:6/281: mknod d1/d3/d16/c57 0 2024-09-16T08:57:06.749 INFO:tasks.workunit.client.1.smithi097.stdout:9/229: mknod d2/d30/c4a 0 2024-09-16T08:57:06.749 INFO:tasks.workunit.client.1.smithi097.stdout:9/230: creat d2/d3f/f4b x:0 0 0 2024-09-16T08:57:06.750 INFO:tasks.workunit.client.1.smithi097.stdout:4/250: dwrite d0/d39/f3c [0,4194304] 0 2024-09-16T08:57:06.751 INFO:tasks.workunit.client.1.smithi097.stdout:8/233: getdents d8/dd/d15/d16 0 2024-09-16T08:57:06.751 INFO:tasks.workunit.client.1.smithi097.stdout:8/234: truncate d8/fc 2562549 0 2024-09-16T08:57:06.751 INFO:tasks.workunit.client.1.smithi097.stdout:8/235: fdatasync d8/fa 0 2024-09-16T08:57:06.753 INFO:tasks.workunit.client.1.smithi097.stdout:1/278: dwrite d5/db/f3c [0,4194304] 0 2024-09-16T08:57:06.753 INFO:tasks.workunit.client.1.smithi097.stdout:1/279: dread - d5/db/dc/d19/f46 zero size 2024-09-16T08:57:06.753 INFO:tasks.workunit.client.1.smithi097.stdout:2/243: dwrite d3/d4/f21 [0,4194304] 0 2024-09-16T08:57:06.753 INFO:tasks.workunit.client.1.smithi097.stdout:2/244: write d3/d4/d8/f2b [340352,75489] 0 2024-09-16T08:57:06.753 INFO:tasks.workunit.client.1.smithi097.stdout:2/245: read - d3/f37 zero size 2024-09-16T08:57:06.753 INFO:tasks.workunit.client.1.smithi097.stdout:2/246: dread - d3/d4/d9/f36 zero size 2024-09-16T08:57:06.754 INFO:tasks.workunit.client.1.smithi097.stdout:0/240: creat da/d24/d4b/f52 x:0 0 0 2024-09-16T08:57:06.756 INFO:tasks.workunit.client.1.smithi097.stdout:6/282: symlink d1/d3/d16/d4f/l58 0 2024-09-16T08:57:06.763 INFO:tasks.workunit.client.1.smithi097.stdout:4/251: unlink d0/d7/l48 0 2024-09-16T08:57:06.766 INFO:tasks.workunit.client.1.smithi097.stdout:6/283: write d1/d3/f7 [1347896,84580] 0 2024-09-16T08:57:06.766 INFO:tasks.workunit.client.1.smithi097.stdout:6/284: chown d1/ce 234 1 2024-09-16T08:57:06.766 INFO:tasks.workunit.client.1.smithi097.stdout:1/280: dread d5/f12 [0,4194304] 0 2024-09-16T08:57:06.768 INFO:tasks.workunit.client.1.smithi097.stdout:3/182: rmdir d7 39 2024-09-16T08:57:06.768 INFO:tasks.workunit.client.1.smithi097.stdout:3/183: write d7/de/d29/f32 [552298,971] 0 2024-09-16T08:57:06.769 INFO:tasks.workunit.client.1.smithi097.stdout:3/184: dread - d7/de/d29/f2d zero size 2024-09-16T08:57:06.769 INFO:tasks.workunit.client.1.smithi097.stdout:3/185: write d7/f21 [604473,97957] 0 2024-09-16T08:57:06.772 INFO:tasks.workunit.client.1.smithi097.stdout:2/247: symlink d3/d1b/l3d 0 2024-09-16T08:57:06.773 INFO:tasks.workunit.client.1.smithi097.stdout:9/231: dwrite d2/f13 [0,4194304] 0 2024-09-16T08:57:06.774 INFO:tasks.workunit.client.1.smithi097.stdout:7/219: sync 2024-09-16T08:57:06.774 INFO:tasks.workunit.client.1.smithi097.stdout:7/220: truncate d4/d1b/f4b 318249 0 2024-09-16T08:57:06.775 INFO:tasks.workunit.client.1.smithi097.stdout:0/241: link da/c51 da/d1a/d26/c53 0 2024-09-16T08:57:06.775 INFO:tasks.workunit.client.1.smithi097.stdout:0/242: write da/d1a/f2b [4808192,13392] 0 2024-09-16T08:57:06.775 INFO:tasks.workunit.client.1.smithi097.stdout:0/243: chown da/d24/c2c 222788821 1 2024-09-16T08:57:06.777 INFO:tasks.workunit.client.1.smithi097.stdout:8/236: rename d8/dd/d15/d16/d2d to d8/dd/d15/d2c/d4a/d4d 0 2024-09-16T08:57:06.780 INFO:tasks.workunit.client.1.smithi097.stdout:2/248: dread d3/d4/d8/f2b [0,4194304] 0 2024-09-16T08:57:06.781 INFO:tasks.workunit.client.1.smithi097.stdout:6/285: symlink d1/dc/d3c/l59 0 2024-09-16T08:57:06.783 INFO:tasks.workunit.client.1.smithi097.stdout:1/281: creat d5/db/f52 x:0 0 0 2024-09-16T08:57:06.785 INFO:tasks.workunit.client.1.smithi097.stdout:3/186: mkdir d7/de/d3c 0 2024-09-16T08:57:06.788 INFO:tasks.workunit.client.1.smithi097.stdout:9/232: unlink d2/f2a 0 2024-09-16T08:57:06.788 INFO:tasks.workunit.client.1.smithi097.stdout:9/233: creat d2/d5/d3a/f4c x:0 0 0 2024-09-16T08:57:06.788 INFO:tasks.workunit.client.1.smithi097.stdout:9/234: chown d2/d5/f35 1477 1 2024-09-16T08:57:06.790 INFO:tasks.workunit.client.1.smithi097.stdout:7/221: unlink d4/d27/f4c 0 2024-09-16T08:57:06.791 INFO:tasks.workunit.client.1.smithi097.stdout:0/244: symlink da/df/d1f/d42/l54 0 2024-09-16T08:57:06.794 INFO:tasks.workunit.client.1.smithi097.stdout:4/252: rename d0/d7/d52/d29 to d0/d23/d5f 0 2024-09-16T08:57:06.794 INFO:tasks.workunit.client.1.smithi097.stdout:4/253: truncate d0/d7/f24 4808300 0 2024-09-16T08:57:06.794 INFO:tasks.workunit.client.1.smithi097.stdout:4/254: chown d0/d7/c33 0 1 2024-09-16T08:57:06.797 INFO:tasks.workunit.client.1.smithi097.stdout:2/249: mkdir d3/d1b/d3e 0 2024-09-16T08:57:06.797 INFO:tasks.workunit.client.1.smithi097.stdout:2/250: chown d3/d4 3279045 1 2024-09-16T08:57:06.807 INFO:tasks.workunit.client.1.smithi097.stdout:1/282: unlink d5/d2c/d3b/c47 0 2024-09-16T08:57:06.808 INFO:tasks.workunit.client.1.smithi097.stdout:1/283: creat d5/db/d25/f53 x:0 0 0 2024-09-16T08:57:06.809 INFO:tasks.workunit.client.1.smithi097.stdout:3/187: mkdir d7/de/d29/d35/d3d 0 2024-09-16T08:57:06.813 INFO:tasks.workunit.client.1.smithi097.stdout:9/235: rename d2/l14 to d2/l4d 0 2024-09-16T08:57:06.815 INFO:tasks.workunit.client.1.smithi097.stdout:7/222: creat d4/d7/f50 x:0 0 0 2024-09-16T08:57:06.819 INFO:tasks.workunit.client.1.smithi097.stdout:6/286: rename d1/d3/df to d1/d3/d3e/d5a 0 2024-09-16T08:57:06.819 INFO:tasks.workunit.client.1.smithi097.stdout:6/287: fsync d1/d3/f7 0 2024-09-16T08:57:06.820 INFO:tasks.workunit.client.1.smithi097.stdout:4/255: mknod d0/d23/c60 0 2024-09-16T08:57:06.824 INFO:tasks.workunit.client.1.smithi097.stdout:1/284: getdents d5/d17 0 2024-09-16T08:57:06.826 INFO:tasks.workunit.client.1.smithi097.stdout:3/188: symlink d7/de/d3c/l3e 0 2024-09-16T08:57:06.826 INFO:tasks.workunit.client.1.smithi097.stdout:3/189: stat d7/de/d29/d35/f27 0 2024-09-16T08:57:06.828 INFO:tasks.workunit.client.1.smithi097.stdout:9/236: creat d2/d5/f4e x:0 0 0 2024-09-16T08:57:06.829 INFO:tasks.workunit.client.1.smithi097.stdout:7/223: mkdir d4/d9/d51 0 2024-09-16T08:57:06.832 INFO:tasks.workunit.client.1.smithi097.stdout:2/251: rename d3/d4/d8/d34/c38 to d3/d4/d8/c3f 0 2024-09-16T08:57:06.833 INFO:tasks.workunit.client.1.smithi097.stdout:2/252: stat d3/d4/l1c 0 2024-09-16T08:57:06.834 INFO:tasks.workunit.client.1.smithi097.stdout:4/256: link d0/d7/df/l3b d0/d7/l61 0 2024-09-16T08:57:06.836 INFO:tasks.workunit.client.1.smithi097.stdout:1/285: mkdir d5/d17/d54 0 2024-09-16T08:57:06.837 INFO:tasks.workunit.client.1.smithi097.stdout:0/245: truncate da/d1a/f2b 1903645 0 2024-09-16T08:57:06.838 INFO:tasks.workunit.client.1.smithi097.stdout:0/246: write da/d1a/f1c [8371,42544] 0 2024-09-16T08:57:06.842 INFO:tasks.workunit.client.1.smithi097.stdout:2/253: write d3/d4/f21 [2457202,48604] 0 2024-09-16T08:57:06.842 INFO:tasks.workunit.client.1.smithi097.stdout:3/190: mkdir d7/de/d3c/d3f 0 2024-09-16T08:57:06.843 INFO:tasks.workunit.client.1.smithi097.stdout:3/191: readlink d7/l12 0 2024-09-16T08:57:06.843 INFO:tasks.workunit.client.1.smithi097.stdout:5/251: sync 2024-09-16T08:57:06.843 INFO:tasks.workunit.client.1.smithi097.stdout:5/252: write db/f14 [4796005,95400] 0 2024-09-16T08:57:06.843 INFO:tasks.workunit.client.1.smithi097.stdout:9/237: mknod d2/d5/c4f 0 2024-09-16T08:57:06.844 INFO:tasks.workunit.client.1.smithi097.stdout:4/257: write d0/d7/d52/d26/d3d/f44 [2401983,22380] 0 2024-09-16T08:57:06.845 INFO:tasks.workunit.client.1.smithi097.stdout:7/224: link d4/d12/f4f d4/d1b/d41/f52 0 2024-09-16T08:57:06.845 INFO:tasks.workunit.client.1.smithi097.stdout:7/225: fsync d4/d9/d2b/f2e 0 2024-09-16T08:57:06.848 INFO:tasks.workunit.client.1.smithi097.stdout:6/288: rename d1/d3/d3e/d5a/f56 to d1/f5b 0 2024-09-16T08:57:06.849 INFO:tasks.workunit.client.1.smithi097.stdout:2/254: dread d3/f23 [0,4194304] 0 2024-09-16T08:57:06.850 INFO:tasks.workunit.client.1.smithi097.stdout:2/255: creat d3/f40 x:0 0 0 2024-09-16T08:57:06.850 INFO:tasks.workunit.client.1.smithi097.stdout:2/256: creat d3/d4/f41 x:0 0 0 2024-09-16T08:57:06.852 INFO:tasks.workunit.client.1.smithi097.stdout:6/289: dread d1/d3/f1c [0,4194304] 0 2024-09-16T08:57:06.854 INFO:tasks.workunit.client.1.smithi097.stdout:3/192: creat d7/de/d3c/d3f/f40 x:0 0 0 2024-09-16T08:57:06.856 INFO:tasks.workunit.client.1.smithi097.stdout:7/226: dread d4/d9/dc/f16 [0,4194304] 0 2024-09-16T08:57:06.857 INFO:tasks.workunit.client.1.smithi097.stdout:5/253: mknod db/d18/d1c/c54 0 2024-09-16T08:57:06.860 INFO:tasks.workunit.client.1.smithi097.stdout:0/247: dwrite da/df/f3f [0,4194304] 0 2024-09-16T08:57:06.860 INFO:tasks.workunit.client.1.smithi097.stdout:2/257: write d3/d4/d8/f39 [2993912,60352] 0 2024-09-16T08:57:06.860 INFO:tasks.workunit.client.1.smithi097.stdout:2/258: fdatasync d3/f32 0 2024-09-16T08:57:06.860 INFO:tasks.workunit.client.1.smithi097.stdout:2/259: fdatasync d3/d4/d8/f33 0 2024-09-16T08:57:06.861 INFO:tasks.workunit.client.1.smithi097.stdout:9/238: creat d2/f50 x:0 0 0 2024-09-16T08:57:06.861 INFO:tasks.workunit.client.1.smithi097.stdout:9/239: readlink d2/d5/d3a/d3c/l44 0 2024-09-16T08:57:06.861 INFO:tasks.workunit.client.1.smithi097.stdout:9/240: dread - d2/d30/f45 zero size 2024-09-16T08:57:06.865 INFO:tasks.workunit.client.1.smithi097.stdout:4/258: rmdir d0/d7/df/d15/d1f 39 2024-09-16T08:57:06.865 INFO:tasks.workunit.client.1.smithi097.stdout:4/259: creat d0/d7/d52/d26/d3d/f62 x:0 0 0 2024-09-16T08:57:06.865 INFO:tasks.workunit.client.1.smithi097.stdout:4/260: creat d0/d7/df/d15/f63 x:0 0 0 2024-09-16T08:57:06.866 INFO:tasks.workunit.client.1.smithi097.stdout:9/241: dread d2/d30/f3b [0,4194304] 0 2024-09-16T08:57:06.866 INFO:tasks.workunit.client.1.smithi097.stdout:9/242: creat d2/f51 x:0 0 0 2024-09-16T08:57:06.866 INFO:tasks.workunit.client.1.smithi097.stdout:9/243: write d2/f13 [4637191,3543] 0 2024-09-16T08:57:06.867 INFO:tasks.workunit.client.1.smithi097.stdout:6/290: dwrite d1/d3/d16/d4f/f55 [0,4194304] 0 2024-09-16T08:57:06.871 INFO:tasks.workunit.client.1.smithi097.stdout:3/193: symlink d7/de/d29/d35/d3d/l41 0 2024-09-16T08:57:06.871 INFO:tasks.workunit.client.1.smithi097.stdout:3/194: write d7/d15/f17 [562180,114948] 0 2024-09-16T08:57:06.876 INFO:tasks.workunit.client.1.smithi097.stdout:5/254: dwrite db/d18/d20/f29 [8388608,4194304] 0 2024-09-16T08:57:06.879 INFO:tasks.workunit.client.1.smithi097.stdout:8/237: sync 2024-09-16T08:57:06.886 INFO:tasks.workunit.client.1.smithi097.stdout:2/260: unlink d3/d4/d9/l22 0 2024-09-16T08:57:06.890 INFO:tasks.workunit.client.1.smithi097.stdout:4/261: rename d0/d7/d52/f4b to d0/f64 0 2024-09-16T08:57:06.893 INFO:tasks.workunit.client.1.smithi097.stdout:5/255: dwrite db/d18/d20/f29 [0,4194304] 0 2024-09-16T08:57:06.895 INFO:tasks.workunit.client.1.smithi097.stdout:9/244: symlink d2/d30/l52 0 2024-09-16T08:57:06.895 INFO:tasks.workunit.client.1.smithi097.stdout:9/245: dread - d2/d5/f35 zero size 2024-09-16T08:57:06.895 INFO:tasks.workunit.client.1.smithi097.stdout:9/246: stat d2/c2e 0 2024-09-16T08:57:06.895 INFO:tasks.workunit.client.1.smithi097.stdout:9/247: write d2/d3f/f43 [1790555,23956] 0 2024-09-16T08:57:06.897 INFO:tasks.workunit.client.1.smithi097.stdout:6/291: mknod d1/c5c 0 2024-09-16T08:57:06.900 INFO:tasks.workunit.client.1.smithi097.stdout:3/195: rmdir d7/de/d29/d35 39 2024-09-16T08:57:06.900 INFO:tasks.workunit.client.1.smithi097.stdout:3/196: write d7/de/d29/f2d [749583,81381] 0 2024-09-16T08:57:06.900 INFO:tasks.workunit.client.1.smithi097.stdout:3/197: fdatasync d7/de/d29/d35/f22 0 2024-09-16T08:57:06.902 INFO:tasks.workunit.client.1.smithi097.stdout:7/227: truncate d4/d7/ff 5189493 0 2024-09-16T08:57:06.902 INFO:tasks.workunit.client.1.smithi097.stdout:7/228: creat d4/f53 x:0 0 0 2024-09-16T08:57:06.905 INFO:tasks.workunit.client.1.smithi097.stdout:0/248: rmdir da/df/d13 39 2024-09-16T08:57:06.905 INFO:tasks.workunit.client.1.smithi097.stdout:0/249: fdatasync da/df/d1f/f3c 0 2024-09-16T08:57:06.907 INFO:tasks.workunit.client.1.smithi097.stdout:2/261: mknod d3/d1b/c42 0 2024-09-16T08:57:06.907 INFO:tasks.workunit.client.1.smithi097.stdout:2/262: dread - d3/d4/d8/d34/f3c zero size 2024-09-16T08:57:06.908 INFO:tasks.workunit.client.1.smithi097.stdout:2/263: write d3/d4/d9/f1e [533740,86377] 0 2024-09-16T08:57:06.909 INFO:tasks.workunit.client.1.smithi097.stdout:8/238: rename d8/dd/d15/d2c/d4a/d4d/d3d/d47 to d8/dd/d15/d2c/d4a/d4e 0 2024-09-16T08:57:06.910 INFO:tasks.workunit.client.1.smithi097.stdout:4/262: mknod d0/d7/d52/c65 0 2024-09-16T08:57:06.912 INFO:tasks.workunit.client.1.smithi097.stdout:5/256: dwrite db/d1a/f3a [0,4194304] 0 2024-09-16T08:57:06.912 INFO:tasks.workunit.client.1.smithi097.stdout:5/257: write db/f10 [1602104,56275] 0 2024-09-16T08:57:06.912 INFO:tasks.workunit.client.1.smithi097.stdout:5/258: fsync db/d18/d1c/d42/d48/f53 0 2024-09-16T08:57:06.912 INFO:tasks.workunit.client.1.smithi097.stdout:7/229: dread d4/d9/d1c/f2c [0,4194304] 0 2024-09-16T08:57:06.912 INFO:tasks.workunit.client.1.smithi097.stdout:7/230: dread - d4/d9/dc/f48 zero size 2024-09-16T08:57:06.912 INFO:tasks.workunit.client.1.smithi097.stdout:7/231: write d4/d7/f11 [1005488,66260] 0 2024-09-16T08:57:06.913 INFO:tasks.workunit.client.1.smithi097.stdout:9/248: creat d2/d5/d3a/d3c/f53 x:0 0 0 2024-09-16T08:57:06.915 INFO:tasks.workunit.client.1.smithi097.stdout:6/292: mknod d1/dc/d1e/c5d 0 2024-09-16T08:57:06.915 INFO:tasks.workunit.client.1.smithi097.stdout:6/293: chown d1/c5c 1541872940 1 2024-09-16T08:57:06.917 INFO:tasks.workunit.client.1.smithi097.stdout:3/198: mknod d7/de/d3c/d3f/c42 0 2024-09-16T08:57:06.919 INFO:tasks.workunit.client.1.smithi097.stdout:8/239: read d8/dd/d15/f29 [439025,118663] 0 2024-09-16T08:57:06.922 INFO:tasks.workunit.client.1.smithi097.stdout:5/259: write db/d18/d20/f29 [11954876,52497] 0 2024-09-16T08:57:06.923 INFO:tasks.workunit.client.1.smithi097.stdout:0/250: rename da/df/d13/c4d to da/c55 0 2024-09-16T08:57:06.923 INFO:tasks.workunit.client.1.smithi097.stdout:2/264: rename d3/d1b to d3/d1b/d3e/d43 22 2024-09-16T08:57:06.924 INFO:tasks.workunit.client.1.smithi097.stdout:5/260: dread db/d1a/f38 [0,4194304] 0 2024-09-16T08:57:06.924 INFO:tasks.workunit.client.1.smithi097.stdout:4/263: link d0/d36/f54 d0/d39/f66 0 2024-09-16T08:57:06.924 INFO:tasks.workunit.client.1.smithi097.stdout:2/265: read d3/d4/f2d [392899,103129] 0 2024-09-16T08:57:06.938 INFO:tasks.workunit.client.1.smithi097.stdout:7/232: dread d4/d12/d1f/f21 [0,4194304] 0 2024-09-16T08:57:06.940 INFO:tasks.workunit.client.1.smithi097.stdout:9/249: unlink d2/d5/c1e 0 2024-09-16T08:57:06.940 INFO:tasks.workunit.client.1.smithi097.stdout:9/250: read d2/fc [1453923,100195] 0 2024-09-16T08:57:06.940 INFO:tasks.workunit.client.1.smithi097.stdout:9/251: creat d2/d5/d3a/d3c/f54 x:0 0 0 2024-09-16T08:57:06.941 INFO:tasks.workunit.client.1.smithi097.stdout:9/252: fsync d2/d30/f3b 0 2024-09-16T08:57:06.943 INFO:tasks.workunit.client.1.smithi097.stdout:6/294: mknod d1/d3/d3e/c5e 0 2024-09-16T08:57:06.943 INFO:tasks.workunit.client.1.smithi097.stdout:6/295: write d1/f5b [890792,48222] 0 2024-09-16T08:57:06.945 INFO:tasks.workunit.client.1.smithi097.stdout:7/233: dread d4/d7/f11 [0,4194304] 0 2024-09-16T08:57:06.946 INFO:tasks.workunit.client.1.smithi097.stdout:6/296: dread d1/d3/d16/f2e [0,4194304] 0 2024-09-16T08:57:06.946 INFO:tasks.workunit.client.1.smithi097.stdout:6/297: chown d1/d3/c31 730365 1 2024-09-16T08:57:06.947 INFO:tasks.workunit.client.1.smithi097.stdout:0/251: symlink da/df/d3d/l56 0 2024-09-16T08:57:06.955 INFO:tasks.workunit.client.1.smithi097.stdout:9/253: dwrite d2/f6 [4194304,4194304] 0 2024-09-16T08:57:06.955 INFO:tasks.workunit.client.1.smithi097.stdout:9/254: creat d2/d3f/f55 x:0 0 0 2024-09-16T08:57:06.955 INFO:tasks.workunit.client.1.smithi097.stdout:9/255: truncate d2/f51 5126 0 2024-09-16T08:57:06.955 INFO:tasks.workunit.client.1.smithi097.stdout:9/256: stat d2/d30/c4a 0 2024-09-16T08:57:06.955 INFO:tasks.workunit.client.1.smithi097.stdout:9/257: stat d2/l22 0 2024-09-16T08:57:06.956 INFO:tasks.workunit.client.1.smithi097.stdout:7/234: symlink d4/d1b/d36/d43/l54 0 2024-09-16T08:57:06.958 INFO:tasks.workunit.client.1.smithi097.stdout:3/199: rename d7/d28/d36 to d7/d43 0 2024-09-16T08:57:06.959 INFO:tasks.workunit.client.1.smithi097.stdout:4/264: getdents d0/d7/d52/d26/d3d 0 2024-09-16T08:57:06.960 INFO:tasks.workunit.client.1.smithi097.stdout:1/286: sync 2024-09-16T08:57:06.961 INFO:tasks.workunit.client.1.smithi097.stdout:4/265: dread d0/d36/f56 [0,4194304] 0 2024-09-16T08:57:06.961 INFO:tasks.workunit.client.1.smithi097.stdout:4/266: readlink d0/d7/d52/d26/l34 0 2024-09-16T08:57:06.961 INFO:tasks.workunit.client.1.smithi097.stdout:4/267: creat d0/d7/d52/f67 x:0 0 0 2024-09-16T08:57:06.965 INFO:tasks.workunit.client.1.smithi097.stdout:9/258: truncate d2/d5/fa 557983 0 2024-09-16T08:57:06.967 INFO:tasks.workunit.client.1.smithi097.stdout:1/287: dread d5/d2c/f2e [0,4194304] 0 2024-09-16T08:57:06.968 INFO:tasks.workunit.client.1.smithi097.stdout:7/235: mknod d4/d9/d51/c55 0 2024-09-16T08:57:06.972 INFO:tasks.workunit.client.1.smithi097.stdout:5/261: rename db/d1a/c3b to db/d18/d20/d2b/c55 0 2024-09-16T08:57:06.972 INFO:tasks.workunit.client.1.smithi097.stdout:5/262: chown db/c1b 3 1 2024-09-16T08:57:06.975 INFO:tasks.workunit.client.1.smithi097.stdout:4/268: link d0/d7/f5b d0/d36/d4c/f68 0 2024-09-16T08:57:06.975 INFO:tasks.workunit.client.1.smithi097.stdout:4/269: stat d0/f2a 0 2024-09-16T08:57:06.975 INFO:tasks.workunit.client.1.smithi097.stdout:4/270: chown d0/d35/f38 5 1 2024-09-16T08:57:06.976 INFO:tasks.workunit.client.1.smithi097.stdout:9/259: mkdir d2/d5/d56 0 2024-09-16T08:57:06.976 INFO:tasks.workunit.client.1.smithi097.stdout:9/260: stat d2/d5/l20 0 2024-09-16T08:57:06.976 INFO:tasks.workunit.client.1.smithi097.stdout:9/261: dread - d2/d5/d3a/d3c/f54 zero size 2024-09-16T08:57:06.977 INFO:tasks.workunit.client.1.smithi097.stdout:3/200: dwrite d7/de/d29/f2d [0,4194304] 0 2024-09-16T08:57:06.977 INFO:tasks.workunit.client.1.smithi097.stdout:3/201: chown d7/de/d29/d35/d3d/l41 4 1 2024-09-16T08:57:06.977 INFO:tasks.workunit.client.1.smithi097.stdout:3/202: dread - d7/de/d3c/d3f/f40 zero size 2024-09-16T08:57:06.977 INFO:tasks.workunit.client.1.smithi097.stdout:3/203: chown d7/de/d29/d35/f22 3 1 2024-09-16T08:57:06.978 INFO:tasks.workunit.client.1.smithi097.stdout:2/266: rename d3/d1b/c2c to d3/d4/c44 0 2024-09-16T08:57:06.985 INFO:tasks.workunit.client.1.smithi097.stdout:4/271: dread d0/fe [0,4194304] 0 2024-09-16T08:57:06.992 INFO:tasks.workunit.client.1.smithi097.stdout:3/204: creat d7/f44 x:0 0 0 2024-09-16T08:57:06.992 INFO:tasks.workunit.client.1.smithi097.stdout:3/205: chown d7/de/d3c 29338893 1 2024-09-16T08:57:06.998 INFO:tasks.workunit.client.1.smithi097.stdout:6/298: rename d1/c9 to d1/dc/d1e/c5f 0 2024-09-16T08:57:07.002 INFO:tasks.workunit.client.1.smithi097.stdout:1/288: dwrite d5/db/dc/f33 [0,4194304] 0 2024-09-16T08:57:07.003 INFO:tasks.workunit.client.1.smithi097.stdout:1/289: dread - d5/db/dc/d19/f4d zero size 2024-09-16T08:57:07.003 INFO:tasks.workunit.client.1.smithi097.stdout:3/206: creat d7/de/d3c/f45 x:0 0 0 2024-09-16T08:57:07.004 INFO:tasks.workunit.client.1.smithi097.stdout:6/299: truncate d1/d3/f3d 3455948 0 2024-09-16T08:57:07.004 INFO:tasks.workunit.client.1.smithi097.stdout:6/300: truncate d1/d3/d3e/d5a/f4a 806533 0 2024-09-16T08:57:07.009 INFO:tasks.workunit.client.1.smithi097.stdout:3/207: unlink d7/de/d3c/d3f/f40 0 2024-09-16T08:57:07.009 INFO:tasks.workunit.client.1.smithi097.stdout:3/208: fsync d7/de/d29/f32 0 2024-09-16T08:57:07.009 INFO:tasks.workunit.client.1.smithi097.stdout:3/209: truncate d7/f21 2610710 0 2024-09-16T08:57:07.011 INFO:tasks.workunit.client.1.smithi097.stdout:3/210: symlink d7/de/l46 0 2024-09-16T08:57:07.016 INFO:tasks.workunit.client.1.smithi097.stdout:1/290: write d5/db/dc/f33 [2589781,19240] 0 2024-09-16T08:57:07.018 INFO:tasks.workunit.client.1.smithi097.stdout:1/291: symlink d5/db/dc/d19/l55 0 2024-09-16T08:57:07.023 INFO:tasks.workunit.client.1.smithi097.stdout:1/292: dread d5/db/dc/f41 [0,4194304] 0 2024-09-16T08:57:07.025 INFO:tasks.workunit.client.1.smithi097.stdout:7/236: dwrite d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:07.025 INFO:tasks.workunit.client.1.smithi097.stdout:5/263: dwrite db/d18/f34 [0,4194304] 0 2024-09-16T08:57:07.025 INFO:tasks.workunit.client.1.smithi097.stdout:7/237: read - d4/d9/d2b/f4d zero size 2024-09-16T08:57:07.025 INFO:tasks.workunit.client.1.smithi097.stdout:5/264: creat db/f56 x:0 0 0 2024-09-16T08:57:07.025 INFO:tasks.workunit.client.1.smithi097.stdout:9/262: dwrite d2/fc [0,4194304] 0 2024-09-16T08:57:07.025 INFO:tasks.workunit.client.1.smithi097.stdout:9/263: chown d2/d5/d3a/d3c/f54 19212 1 2024-09-16T08:57:07.026 INFO:tasks.workunit.client.1.smithi097.stdout:1/293: symlink d5/d17/d54/l56 0 2024-09-16T08:57:07.026 INFO:tasks.workunit.client.1.smithi097.stdout:2/267: dwrite d3/d4/dd/f35 [4194304,4194304] 0 2024-09-16T08:57:07.026 INFO:tasks.workunit.client.1.smithi097.stdout:1/294: write d5/db/d25/f53 [862050,46024] 0 2024-09-16T08:57:07.026 INFO:tasks.workunit.client.1.smithi097.stdout:1/295: fdatasync d5/d2c/f2e 0 2024-09-16T08:57:07.026 INFO:tasks.workunit.client.1.smithi097.stdout:1/296: chown d5/l27 2 1 2024-09-16T08:57:07.026 INFO:tasks.workunit.client.1.smithi097.stdout:2/268: truncate d3/d4/d9/f1e 1143957 0 2024-09-16T08:57:07.032 INFO:tasks.workunit.client.1.smithi097.stdout:6/301: dwrite d1/d3/f2f [0,4194304] 0 2024-09-16T08:57:07.036 INFO:tasks.workunit.client.1.smithi097.stdout:3/211: dwrite d7/f44 [0,4194304] 0 2024-09-16T08:57:07.036 INFO:tasks.workunit.client.1.smithi097.stdout:4/272: dwrite d0/d7/df/d15/f11 [0,4194304] 0 2024-09-16T08:57:07.036 INFO:tasks.workunit.client.1.smithi097.stdout:4/273: read - d0/d7/d52/f67 zero size 2024-09-16T08:57:07.036 INFO:tasks.workunit.client.1.smithi097.stdout:4/274: chown d0/d7/df/f27 1370 1 2024-09-16T08:57:07.045 INFO:tasks.workunit.client.1.smithi097.stdout:1/297: mknod d5/db/d25/c57 0 2024-09-16T08:57:07.045 INFO:tasks.workunit.client.1.smithi097.stdout:1/298: chown d5/f9 1193 1 2024-09-16T08:57:07.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/299: stat d5 0 2024-09-16T08:57:07.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/300: dread - d5/db/dc/f48 zero size 2024-09-16T08:57:07.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/301: fsync d5/f35 0 2024-09-16T08:57:07.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/302: creat d5/db/dc/d19/f58 x:0 0 0 2024-09-16T08:57:07.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/303: chown d5/db/dc/f48 931925 1 2024-09-16T08:57:07.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/304: write d5/db/dc/d19/f46 [612299,55473] 0 2024-09-16T08:57:07.048 INFO:tasks.workunit.client.1.smithi097.stdout:9/264: write d2/f38 [2083447,81115] 0 2024-09-16T08:57:07.049 INFO:tasks.workunit.client.1.smithi097.stdout:2/269: symlink d3/l45 0 2024-09-16T08:57:07.049 INFO:tasks.workunit.client.1.smithi097.stdout:2/270: write d3/f32 [861760,7508] 0 2024-09-16T08:57:07.054 INFO:tasks.workunit.client.1.smithi097.stdout:7/238: dwrite d4/d1b/d36/f46 [0,4194304] 0 2024-09-16T08:57:07.054 INFO:tasks.workunit.client.1.smithi097.stdout:7/239: write d4/d1b/d36/d43/f44 [914921,90719] 0 2024-09-16T08:57:07.054 INFO:tasks.workunit.client.1.smithi097.stdout:7/240: readlink d4/l47 0 2024-09-16T08:57:07.056 INFO:tasks.workunit.client.1.smithi097.stdout:3/212: unlink d7/de/l46 0 2024-09-16T08:57:07.058 INFO:tasks.workunit.client.1.smithi097.stdout:4/275: getdents d0/d7/df/d15/d40 0 2024-09-16T08:57:07.064 INFO:tasks.workunit.client.1.smithi097.stdout:1/305: dwrite d5/db/dc/d19/f58 [0,4194304] 0 2024-09-16T08:57:07.064 INFO:tasks.workunit.client.1.smithi097.stdout:5/265: rename db/f19 to db/d1a/f57 0 2024-09-16T08:57:07.079 INFO:tasks.workunit.client.1.smithi097.stdout:1/306: dwrite d5/db/f3c [0,4194304] 0 2024-09-16T08:57:07.082 INFO:tasks.workunit.client.1.smithi097.stdout:1/307: dread d5/db/d25/f53 [0,4194304] 0 2024-09-16T08:57:07.083 INFO:tasks.workunit.client.1.smithi097.stdout:3/213: creat d7/d28/f47 x:0 0 0 2024-09-16T08:57:07.083 INFO:tasks.workunit.client.1.smithi097.stdout:5/266: dread db/d1a/f38 [0,4194304] 0 2024-09-16T08:57:07.084 INFO:tasks.workunit.client.1.smithi097.stdout:4/276: link d0/d7/df/d15/d1f/c21 d0/d23/c69 0 2024-09-16T08:57:07.101 INFO:tasks.workunit.client.1.smithi097.stdout:4/277: link d0/d7/fb d0/d39/f6a 0 2024-09-16T08:57:07.105 INFO:tasks.workunit.client.1.smithi097.stdout:4/278: creat d0/d7/d52/d26/f6b x:0 0 0 2024-09-16T08:57:07.109 INFO:tasks.workunit.client.1.smithi097.stdout:4/279: getdents d0/d35 0 2024-09-16T08:57:07.113 INFO:tasks.workunit.client.1.smithi097.stdout:7/241: dwrite d4/f23 [0,4194304] 0 2024-09-16T08:57:07.114 INFO:tasks.workunit.client.1.smithi097.stdout:7/242: readlink d4/d9/dc/l10 0 2024-09-16T08:57:07.114 INFO:tasks.workunit.client.1.smithi097.stdout:7/243: truncate d4/d9/d1c/f2c 4887954 0 2024-09-16T08:57:07.114 INFO:tasks.workunit.client.1.smithi097.stdout:5/267: dwrite fa [8388608,4194304] 0 2024-09-16T08:57:07.115 INFO:tasks.workunit.client.1.smithi097.stdout:7/244: dread d4/d12/d1f/f21 [0,4194304] 0 2024-09-16T08:57:07.120 INFO:tasks.workunit.client.1.smithi097.stdout:1/308: getdents d5/db/d25 0 2024-09-16T08:57:07.120 INFO:tasks.workunit.client.1.smithi097.stdout:1/309: fsync d5/db/dc/d19/f38 0 2024-09-16T08:57:07.124 INFO:tasks.workunit.client.1.smithi097.stdout:3/214: write d7/f44 [5055892,115722] 0 2024-09-16T08:57:07.124 INFO:tasks.workunit.client.1.smithi097.stdout:3/215: chown d7/de/d3c/l3e 5337366 1 2024-09-16T08:57:07.124 INFO:tasks.workunit.client.1.smithi097.stdout:3/216: truncate d7/f21 3140622 0 2024-09-16T08:57:07.127 INFO:tasks.workunit.client.1.smithi097.stdout:5/268: mkdir db/d22/d58 0 2024-09-16T08:57:07.129 INFO:tasks.workunit.client.1.smithi097.stdout:7/245: creat d4/d12/d1f/d22/f56 x:0 0 0 2024-09-16T08:57:07.129 INFO:tasks.workunit.client.1.smithi097.stdout:7/246: creat d4/d1b/f57 x:0 0 0 2024-09-16T08:57:07.132 INFO:tasks.workunit.client.1.smithi097.stdout:1/310: creat d5/db/f59 x:0 0 0 2024-09-16T08:57:07.135 INFO:tasks.workunit.client.1.smithi097.stdout:4/280: dwrite d0/d7/df/d15/f11 [0,4194304] 0 2024-09-16T08:57:07.137 INFO:tasks.workunit.client.1.smithi097.stdout:5/269: creat db/d1a/d50/f59 x:0 0 0 2024-09-16T08:57:07.137 INFO:tasks.workunit.client.1.smithi097.stdout:5/270: chown db/d18/f34 370477 1 2024-09-16T08:57:07.137 INFO:tasks.workunit.client.1.smithi097.stdout:5/271: fdatasync db/f14 0 2024-09-16T08:57:07.141 INFO:tasks.workunit.client.1.smithi097.stdout:7/247: mknod d4/d9/dc/c58 0 2024-09-16T08:57:07.146 INFO:tasks.workunit.client.1.smithi097.stdout:1/311: link d5/d2c/f4e d5/d2c/d3b/f5a 0 2024-09-16T08:57:07.146 INFO:tasks.workunit.client.1.smithi097.stdout:1/312: chown d5/db/dc/f33 7804 1 2024-09-16T08:57:07.148 INFO:tasks.workunit.client.1.smithi097.stdout:3/217: mknod d7/de/d29/c48 0 2024-09-16T08:57:07.161 INFO:tasks.workunit.client.1.smithi097.stdout:3/218: rename d7/de/d29/f2c to d7/de/d29/f49 0 2024-09-16T08:57:07.164 INFO:tasks.workunit.client.1.smithi097.stdout:1/313: read d5/db/f23 [7738184,33243] 0 2024-09-16T08:57:07.164 INFO:tasks.workunit.client.1.smithi097.stdout:1/314: readlink d5/db/dc/d19/l55 0 2024-09-16T08:57:07.168 INFO:tasks.workunit.client.1.smithi097.stdout:1/315: creat d5/d17/f5b x:0 0 0 2024-09-16T08:57:07.171 INFO:tasks.workunit.client.1.smithi097.stdout:1/316: symlink d5/d2c/d3b/l5c 0 2024-09-16T08:57:07.172 INFO:tasks.workunit.client.1.smithi097.stdout:4/281: dwrite d0/f4 [0,4194304] 0 2024-09-16T08:57:07.172 INFO:tasks.workunit.client.1.smithi097.stdout:7/248: dwrite d4/d9/d1c/f34 [0,4194304] 0 2024-09-16T08:57:07.172 INFO:tasks.workunit.client.1.smithi097.stdout:5/272: dwrite db/fe [0,4194304] 0 2024-09-16T08:57:07.174 INFO:tasks.workunit.client.1.smithi097.stdout:1/317: creat d5/db/f5d x:0 0 0 2024-09-16T08:57:07.174 INFO:tasks.workunit.client.1.smithi097.stdout:1/318: creat d5/d2c/f5e x:0 0 0 2024-09-16T08:57:07.180 INFO:tasks.workunit.client.1.smithi097.stdout:3/219: dwrite d7/de/d29/f2d [0,4194304] 0 2024-09-16T08:57:07.181 INFO:tasks.workunit.client.1.smithi097.stdout:4/282: rmdir d0/d39 39 2024-09-16T08:57:07.181 INFO:tasks.workunit.client.1.smithi097.stdout:4/283: readlink d0/d7/l4d 0 2024-09-16T08:57:07.181 INFO:tasks.workunit.client.1.smithi097.stdout:4/284: truncate d0/d36/f47 835858 0 2024-09-16T08:57:07.190 INFO:tasks.workunit.client.1.smithi097.stdout:1/319: dwrite d5/f9 [0,4194304] 0 2024-09-16T08:57:07.194 INFO:tasks.workunit.client.1.smithi097.stdout:5/273: mknod db/d18/d20/d2b/c5a 0 2024-09-16T08:57:07.194 INFO:tasks.workunit.client.1.smithi097.stdout:5/274: dread - db/d18/f3d zero size 2024-09-16T08:57:07.194 INFO:tasks.workunit.client.1.smithi097.stdout:5/275: write db/d1a/f38 [1396833,96969] 0 2024-09-16T08:57:07.200 INFO:tasks.workunit.client.1.smithi097.stdout:3/220: unlink d7/de/d29/f32 0 2024-09-16T08:57:07.202 INFO:tasks.workunit.client.1.smithi097.stdout:4/285: mkdir d0/d35/d6c 0 2024-09-16T08:57:07.209 INFO:tasks.workunit.client.1.smithi097.stdout:1/320: symlink d5/d2c/l5f 0 2024-09-16T08:57:07.209 INFO:tasks.workunit.client.1.smithi097.stdout:1/321: write d5/d2c/f2e [5150310,39738] 0 2024-09-16T08:57:07.209 INFO:tasks.workunit.client.1.smithi097.stdout:1/322: truncate d5/d2c/f4c 253706 0 2024-09-16T08:57:07.209 INFO:tasks.workunit.client.1.smithi097.stdout:1/323: stat d5/db/dc/d19/l4f 0 2024-09-16T08:57:07.209 INFO:tasks.workunit.client.1.smithi097.stdout:7/249: rmdir d4/d12 39 2024-09-16T08:57:07.211 INFO:tasks.workunit.client.1.smithi097.stdout:8/240: sync 2024-09-16T08:57:07.211 INFO:tasks.workunit.client.1.smithi097.stdout:5/276: creat db/d1a/f5b x:0 0 0 2024-09-16T08:57:07.214 INFO:tasks.workunit.client.1.smithi097.stdout:3/221: creat d7/d43/f4a x:0 0 0 2024-09-16T08:57:07.217 INFO:tasks.workunit.client.1.smithi097.stdout:4/286: unlink d0/d39/f66 0 2024-09-16T08:57:07.217 INFO:tasks.workunit.client.1.smithi097.stdout:3/222: write d7/f2b [1822306,107695] 0 2024-09-16T08:57:07.217 INFO:tasks.workunit.client.1.smithi097.stdout:3/223: write d7/f44 [5853893,22300] 0 2024-09-16T08:57:07.217 INFO:tasks.workunit.client.1.smithi097.stdout:3/224: readlink d7/l12 0 2024-09-16T08:57:07.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:06 smithi031 ceph-mon[27844]: [16/Sep/2024:08:57:05] ENGINE Bus STARTING 2024-09-16T08:57:07.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:06 smithi031 ceph-mon[27844]: [16/Sep/2024:08:57:05] ENGINE Serving on http://172.21.15.97:8765 2024-09-16T08:57:07.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:06 smithi031 ceph-mon[27844]: [16/Sep/2024:08:57:05] ENGINE Serving on https://172.21.15.97:7150 2024-09-16T08:57:07.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:06 smithi031 ceph-mon[27844]: [16/Sep/2024:08:57:05] ENGINE Bus STARTED 2024-09-16T08:57:07.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:06 smithi031 ceph-mon[27844]: [16/Sep/2024:08:57:05] ENGINE Error in HTTPServer.serve 2024-09-16T08:57:07.221 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Traceback (most recent call last): 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._run(expiration_interval) 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self.do_handshake() 2024-09-16T08:57:07.222 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:57:07.223 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T08:57:07.223 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:57:07.223 INFO:tasks.workunit.client.1.smithi097.stdout:1/324: mknod d5/db/dc/d19/c60 0 2024-09-16T08:57:07.223 INFO:tasks.workunit.client.1.smithi097.stdout:5/277: dread db/d1a/f1f [0,4194304] 0 2024-09-16T08:57:07.235 INFO:tasks.workunit.client.1.smithi097.stdout:7/250: symlink d4/d9/l59 0 2024-09-16T08:57:07.240 INFO:tasks.workunit.client.1.smithi097.stdout:8/241: mknod d8/dd/d15/d2c/d4a/d4e/c4f 0 2024-09-16T08:57:07.240 INFO:tasks.workunit.client.1.smithi097.stdout:8/242: chown d8/f24 143528641 1 2024-09-16T08:57:07.240 INFO:tasks.workunit.client.1.smithi097.stdout:8/243: chown d8/f24 1073729988 1 2024-09-16T08:57:07.240 INFO:tasks.workunit.client.1.smithi097.stdout:8/244: write d8/f31 [324008,110199] 0 2024-09-16T08:57:07.243 INFO:tasks.workunit.client.1.smithi097.stdout:4/287: symlink d0/d7/df/d15/d40/l6d 0 2024-09-16T08:57:07.243 INFO:tasks.workunit.client.1.smithi097.stdout:4/288: dread - d0/d7/df/f2c zero size 2024-09-16T08:57:07.243 INFO:tasks.workunit.client.1.smithi097.stdout:4/289: chown d0/d7/df/d15/l17 20526 1 2024-09-16T08:57:07.244 INFO:tasks.workunit.client.1.smithi097.stdout:4/290: dread - d0/f30 zero size 2024-09-16T08:57:07.244 INFO:tasks.workunit.client.1.smithi097.stdout:7/251: write d4/d7/ff [163910,112550] 0 2024-09-16T08:57:07.247 INFO:tasks.workunit.client.1.smithi097.stdout:3/225: mknod d7/de/d29/d35/d3d/c4b 0 2024-09-16T08:57:07.247 INFO:tasks.workunit.client.1.smithi097.stdout:3/226: dread - d7/d28/f47 zero size 2024-09-16T08:57:07.247 INFO:tasks.workunit.client.1.smithi097.stdout:3/227: write d7/d28/f2f [382858,41185] 0 2024-09-16T08:57:07.247 INFO:tasks.workunit.client.1.smithi097.stdout:3/228: dread - d7/d43/f4a zero size 2024-09-16T08:57:07.248 INFO:tasks.workunit.client.1.smithi097.stdout:3/229: write d7/f21 [1541680,20112] 0 2024-09-16T08:57:07.248 INFO:tasks.workunit.client.1.smithi097.stdout:3/230: write d7/de/d3c/f45 [740444,83446] 0 2024-09-16T08:57:07.249 INFO:tasks.workunit.client.1.smithi097.stdout:1/325: creat d5/d2c/d3b/f61 x:0 0 0 2024-09-16T08:57:07.249 INFO:tasks.workunit.client.1.smithi097.stdout:1/326: chown d5/db 366 1 2024-09-16T08:57:07.253 INFO:tasks.workunit.client.1.smithi097.stdout:5/278: creat db/d18/d20/d2b/f5c x:0 0 0 2024-09-16T08:57:07.256 INFO:tasks.workunit.client.1.smithi097.stdout:8/245: symlink d8/dd/l50 0 2024-09-16T08:57:07.256 INFO:tasks.workunit.client.1.smithi097.stdout:8/246: chown c0 2155 1 2024-09-16T08:57:07.258 INFO:tasks.workunit.client.1.smithi097.stdout:4/291: mknod d0/d7/d52/d26/d3d/c6e 0 2024-09-16T08:57:07.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:06 smithi097 ceph-mon[35478]: [16/Sep/2024:08:57:05] ENGINE Bus STARTING 2024-09-16T08:57:07.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:06 smithi097 ceph-mon[35478]: [16/Sep/2024:08:57:05] ENGINE Serving on http://172.21.15.97:8765 2024-09-16T08:57:07.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:06 smithi097 ceph-mon[35478]: [16/Sep/2024:08:57:05] ENGINE Serving on https://172.21.15.97:7150 2024-09-16T08:57:07.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:06 smithi097 ceph-mon[35478]: [16/Sep/2024:08:57:05] ENGINE Bus STARTED 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:06 smithi097 ceph-mon[35478]: [16/Sep/2024:08:57:05] ENGINE Error in HTTPServer.serve 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Traceback (most recent call last): 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._run(expiration_interval) 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:57:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s = self.context.wrap_socket( 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: return self.sslsocket_class._create( 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self.do_handshake() 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._sslobj.do_handshake() 2024-09-16T08:57:07.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:57:07.264 INFO:tasks.workunit.client.1.smithi097.stdout:7/252: dwrite d4/d1b/f1e [0,4194304] 0 2024-09-16T08:57:07.264 INFO:tasks.workunit.client.1.smithi097.stdout:7/253: stat d4/d1b/d41/f52 0 2024-09-16T08:57:07.264 INFO:tasks.workunit.client.1.smithi097.stdout:7/254: chown d4/d12/d1f/f3b 1180844 1 2024-09-16T08:57:07.266 INFO:tasks.workunit.client.1.smithi097.stdout:0/252: sync 2024-09-16T08:57:07.266 INFO:tasks.workunit.client.1.smithi097.stdout:0/253: write da/d1a/f49 [735764,11779] 0 2024-09-16T08:57:07.272 INFO:tasks.workunit.client.1.smithi097.stdout:8/247: creat d8/dd/d42/f51 x:0 0 0 2024-09-16T08:57:07.272 INFO:tasks.workunit.client.1.smithi097.stdout:8/248: chown d8/dd/l38 153865 1 2024-09-16T08:57:07.272 INFO:tasks.workunit.client.1.smithi097.stdout:8/249: truncate d8/dd/d15/d2c/d4a/d4d/f35 159681 0 2024-09-16T08:57:07.272 INFO:tasks.workunit.client.1.smithi097.stdout:5/279: dwrite db/f40 [0,4194304] 0 2024-09-16T08:57:07.272 INFO:tasks.workunit.client.1.smithi097.stdout:5/280: chown db/f32 5 1 2024-09-16T08:57:07.274 INFO:tasks.workunit.client.1.smithi097.stdout:4/292: link d0/f64 d0/d39/f6f 0 2024-09-16T08:57:07.276 INFO:tasks.workunit.client.1.smithi097.stdout:1/327: symlink d5/db/l62 0 2024-09-16T08:57:07.276 INFO:tasks.workunit.client.1.smithi097.stdout:1/328: dread - d5/db/f5d zero size 2024-09-16T08:57:07.276 INFO:tasks.workunit.client.1.smithi097.stdout:1/329: write d5/d2c/f4e [798969,102890] 0 2024-09-16T08:57:07.277 INFO:tasks.workunit.client.1.smithi097.stdout:7/255: mkdir d4/d12/d1f/d22/d5a 0 2024-09-16T08:57:07.279 INFO:tasks.workunit.client.1.smithi097.stdout:0/254: rename da/d1a/d26/c38 to da/d1a/d41/c57 0 2024-09-16T08:57:07.280 INFO:tasks.workunit.client.1.smithi097.stdout:0/255: creat da/d1a/f58 x:0 0 0 2024-09-16T08:57:07.281 INFO:tasks.workunit.client.1.smithi097.stdout:8/250: link d8/dd/l1a d8/dd/d15/d2c/d4a/l52 0 2024-09-16T08:57:07.287 INFO:tasks.workunit.client.1.smithi097.stdout:4/293: mknod d0/d36/c70 0 2024-09-16T08:57:07.288 INFO:tasks.workunit.client.1.smithi097.stdout:4/294: stat d0/d7/df/d15/d40/l5a 0 2024-09-16T08:57:07.290 INFO:tasks.workunit.client.1.smithi097.stdout:1/330: mknod d5/c63 0 2024-09-16T08:57:07.293 INFO:tasks.workunit.client.1.smithi097.stdout:1/331: dread d5/db/dc/f41 [0,4194304] 0 2024-09-16T08:57:07.293 INFO:tasks.workunit.client.1.smithi097.stdout:1/332: write d5/db/dc/d19/f46 [1613807,100761] 0 2024-09-16T08:57:07.297 INFO:tasks.workunit.client.1.smithi097.stdout:0/256: creat da/d1a/d26/f59 x:0 0 0 2024-09-16T08:57:07.297 INFO:tasks.workunit.client.1.smithi097.stdout:9/265: sync 2024-09-16T08:57:07.297 INFO:tasks.workunit.client.1.smithi097.stdout:6/302: sync 2024-09-16T08:57:07.297 INFO:tasks.workunit.client.1.smithi097.stdout:2/271: sync 2024-09-16T08:57:07.298 INFO:tasks.workunit.client.1.smithi097.stdout:6/303: dread - d1/d3/d16/f4d zero size 2024-09-16T08:57:07.298 INFO:tasks.workunit.client.1.smithi097.stdout:6/304: read - d1/dc/d3c/f44 zero size 2024-09-16T08:57:07.298 INFO:tasks.workunit.client.1.smithi097.stdout:6/305: fdatasync d1/f4e 0 2024-09-16T08:57:07.298 INFO:tasks.workunit.client.1.smithi097.stdout:6/306: write d1/d3/f4b [461657,6662] 0 2024-09-16T08:57:07.301 INFO:tasks.workunit.client.1.smithi097.stdout:8/251: dwrite d8/fc [0,4194304] 0 2024-09-16T08:57:07.313 INFO:tasks.workunit.client.1.smithi097.stdout:5/281: rename db/d18/d20/d2b/l36 to db/d1a/l5d 0 2024-09-16T08:57:07.315 INFO:tasks.workunit.client.1.smithi097.stdout:1/333: mknod d5/db/dc/d19/c64 0 2024-09-16T08:57:07.316 INFO:tasks.workunit.client.1.smithi097.stdout:2/272: unlink d3/d4/f41 0 2024-09-16T08:57:07.318 INFO:tasks.workunit.client.1.smithi097.stdout:4/295: dwrite d0/d7/d52/d26/d3d/f44 [0,4194304] 0 2024-09-16T08:57:07.318 INFO:tasks.workunit.client.1.smithi097.stdout:4/296: chown d0/d23/c28 624 1 2024-09-16T08:57:07.318 INFO:tasks.workunit.client.1.smithi097.stdout:9/266: creat d2/d5/d56/f57 x:0 0 0 2024-09-16T08:57:07.318 INFO:tasks.workunit.client.1.smithi097.stdout:9/267: chown d2/d5/c3e 2154 1 2024-09-16T08:57:07.318 INFO:tasks.workunit.client.1.smithi097.stdout:9/268: dread - d2/d3f/f55 zero size 2024-09-16T08:57:07.318 INFO:tasks.workunit.client.1.smithi097.stdout:9/269: write d2/f4 [772861,65700] 0 2024-09-16T08:57:07.319 INFO:tasks.workunit.client.1.smithi097.stdout:0/257: dwrite da/df/d37/f39 [0,4194304] 0 2024-09-16T08:57:07.320 INFO:tasks.workunit.client.1.smithi097.stdout:6/307: dwrite d1/f37 [0,4194304] 0 2024-09-16T08:57:07.322 INFO:tasks.workunit.client.1.smithi097.stdout:5/282: link db/d18/d20/c4d db/d18/d1c/d42/d46/c5e 0 2024-09-16T08:57:07.322 INFO:tasks.workunit.client.1.smithi097.stdout:5/283: fsync db/d18/d1c/d42/f4c 0 2024-09-16T08:57:07.322 INFO:tasks.workunit.client.1.smithi097.stdout:5/284: dread - db/d1a/f5b zero size 2024-09-16T08:57:07.322 INFO:tasks.workunit.client.1.smithi097.stdout:5/285: chown db/d1a/f3a 8038702 1 2024-09-16T08:57:07.330 INFO:tasks.workunit.client.1.smithi097.stdout:7/256: rmdir d4 39 2024-09-16T08:57:07.331 INFO:tasks.workunit.client.1.smithi097.stdout:4/297: getdents d0/d35 0 2024-09-16T08:57:07.331 INFO:tasks.workunit.client.1.smithi097.stdout:4/298: write d0/d36/f56 [149012,64774] 0 2024-09-16T08:57:07.333 INFO:tasks.workunit.client.1.smithi097.stdout:9/270: unlink d2/d5/l33 0 2024-09-16T08:57:07.335 INFO:tasks.workunit.client.1.smithi097.stdout:9/271: write d2/d5/d56/f57 [109502,1894] 0 2024-09-16T08:57:07.335 INFO:tasks.workunit.client.1.smithi097.stdout:9/272: write d2/d30/f47 [172315,90208] 0 2024-09-16T08:57:07.335 INFO:tasks.workunit.client.1.smithi097.stdout:9/273: chown d2/d5/c4f 0 1 2024-09-16T08:57:07.336 INFO:tasks.workunit.client.1.smithi097.stdout:8/252: symlink d8/dd/d15/d16/l53 0 2024-09-16T08:57:07.340 INFO:tasks.workunit.client.1.smithi097.stdout:0/258: dwrite f6 [4194304,4194304] 0 2024-09-16T08:57:07.340 INFO:tasks.workunit.client.1.smithi097.stdout:0/259: truncate da/d1a/d41/f50 297848 0 2024-09-16T08:57:07.340 INFO:tasks.workunit.client.1.smithi097.stdout:0/260: creat da/df/d1f/d3b/f5a x:0 0 0 2024-09-16T08:57:07.341 INFO:tasks.workunit.client.1.smithi097.stdout:6/308: symlink d1/d3/d16/l60 0 2024-09-16T08:57:07.345 INFO:tasks.workunit.client.1.smithi097.stdout:1/334: symlink d5/d17/l65 0 2024-09-16T08:57:07.345 INFO:tasks.workunit.client.1.smithi097.stdout:2/273: dwrite d3/d4/d8/d34/f3c [0,4194304] 0 2024-09-16T08:57:07.348 INFO:tasks.workunit.client.1.smithi097.stdout:7/257: getdents d4/d9/d2b 0 2024-09-16T08:57:07.348 INFO:tasks.workunit.client.1.smithi097.stdout:4/299: dwrite d0/d7/d52/f1d [0,4194304] 0 2024-09-16T08:57:07.351 INFO:tasks.workunit.client.1.smithi097.stdout:9/274: dread d2/fc [0,4194304] 0 2024-09-16T08:57:07.359 INFO:tasks.workunit.client.1.smithi097.stdout:6/309: creat d1/d3/d3e/f61 x:0 0 0 2024-09-16T08:57:07.359 INFO:tasks.workunit.client.1.smithi097.stdout:0/261: dwrite da/df/f16 [0,4194304] 0 2024-09-16T08:57:07.361 INFO:tasks.workunit.client.1.smithi097.stdout:5/286: rename db/d18/l52 to db/d1a/l5f 0 2024-09-16T08:57:07.363 INFO:tasks.workunit.client.1.smithi097.stdout:9/275: write d2/f28 [2812777,41863] 0 2024-09-16T08:57:07.365 INFO:tasks.workunit.client.1.smithi097.stdout:7/258: dwrite d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:07.369 INFO:tasks.workunit.client.1.smithi097.stdout:7/259: write d4/d9/d1c/f2c [4119991,118259] 0 2024-09-16T08:57:07.377 INFO:tasks.workunit.client.1.smithi097.stdout:5/287: dwrite db/d1a/f2c [0,4194304] 0 2024-09-16T08:57:07.377 INFO:tasks.workunit.client.1.smithi097.stdout:5/288: chown db/d22/l43 1310380 1 2024-09-16T08:57:07.377 INFO:tasks.workunit.client.1.smithi097.stdout:5/289: fdatasync db/f10 0 2024-09-16T08:57:07.379 INFO:tasks.workunit.client.1.smithi097.stdout:7/260: dread d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:07.398 INFO:tasks.workunit.client.1.smithi097.stdout:8/253: rmdir d8/dd/d42 39 2024-09-16T08:57:07.399 INFO:tasks.workunit.client.1.smithi097.stdout:6/310: rmdir d1/d3/d3e/d5a 39 2024-09-16T08:57:07.402 INFO:tasks.workunit.client.1.smithi097.stdout:3/231: sync 2024-09-16T08:57:07.403 INFO:tasks.workunit.client.1.smithi097.stdout:0/262: mkdir da/df/d1f/d42/d5b 0 2024-09-16T08:57:07.403 INFO:tasks.workunit.client.1.smithi097.stdout:0/263: chown da/d24 78 1 2024-09-16T08:57:07.405 INFO:tasks.workunit.client.1.smithi097.stdout:2/274: rename d3/d4/d9/ff to d3/d4/dd/f46 0 2024-09-16T08:57:07.406 INFO:tasks.workunit.client.1.smithi097.stdout:9/276: mknod d2/c58 0 2024-09-16T08:57:07.413 INFO:tasks.workunit.client.1.smithi097.stdout:5/290: symlink db/d1a/d50/l60 0 2024-09-16T08:57:07.420 INFO:tasks.workunit.client.1.smithi097.stdout:8/254: dwrite d8/f10 [0,4194304] 0 2024-09-16T08:57:07.420 INFO:tasks.workunit.client.1.smithi097.stdout:8/255: readlink d8/dd/d15/d2c/d4a/d4d/d3f/l40 0 2024-09-16T08:57:07.421 INFO:tasks.workunit.client.1.smithi097.stdout:3/232: symlink d7/de/d3c/d3f/l4c 0 2024-09-16T08:57:07.421 INFO:tasks.workunit.client.1.smithi097.stdout:3/233: stat d7/de/d3c/d3f/l4c 0 2024-09-16T08:57:07.422 INFO:tasks.workunit.client.1.smithi097.stdout:0/264: symlink da/df/d1f/d3b/l5c 0 2024-09-16T08:57:07.422 INFO:tasks.workunit.client.1.smithi097.stdout:0/265: chown da/df/f16 18590 1 2024-09-16T08:57:07.422 INFO:tasks.workunit.client.1.smithi097.stdout:0/266: write da/d1a/f2b [37664,44515] 0 2024-09-16T08:57:07.422 INFO:tasks.workunit.client.1.smithi097.stdout:0/267: chown da/c12 6135968 1 2024-09-16T08:57:07.425 INFO:tasks.workunit.client.1.smithi097.stdout:4/300: rename d0/d7/df/d15/d40 to d0/d39/d71 0 2024-09-16T08:57:07.425 INFO:tasks.workunit.client.1.smithi097.stdout:8/256: rename d8/dd/d15/d2c/d4a/d4d to d8/dd/d15/d2c/d4a/d4d/d54 22 2024-09-16T08:57:07.425 INFO:tasks.workunit.client.1.smithi097.stdout:8/257: fsync d8/f10 0 2024-09-16T08:57:07.426 INFO:tasks.workunit.client.1.smithi097.stdout:9/277: chown d2/d5/c26 5209554 1 2024-09-16T08:57:07.429 INFO:tasks.workunit.client.1.smithi097.stdout:3/234: dread d7/fa [0,4194304] 0 2024-09-16T08:57:07.429 INFO:tasks.workunit.client.1.smithi097.stdout:3/235: stat d7/fa 0 2024-09-16T08:57:07.429 INFO:tasks.workunit.client.1.smithi097.stdout:3/236: dread - d7/d28/f47 zero size 2024-09-16T08:57:07.430 INFO:tasks.workunit.client.1.smithi097.stdout:6/311: dwrite d1/d3/d3e/d5a/f36 [0,4194304] 0 2024-09-16T08:57:07.430 INFO:tasks.workunit.client.1.smithi097.stdout:6/312: creat d1/d3/d3e/f62 x:0 0 0 2024-09-16T08:57:07.430 INFO:tasks.workunit.client.1.smithi097.stdout:2/275: dwrite d3/d4/d8/f2b [0,4194304] 0 2024-09-16T08:57:07.431 INFO:tasks.workunit.client.1.smithi097.stdout:6/313: write d1/d3/f4b [420575,37215] 0 2024-09-16T08:57:07.431 INFO:tasks.workunit.client.1.smithi097.stdout:6/314: truncate d1/dc/d3c/f44 986410 0 2024-09-16T08:57:07.431 INFO:tasks.workunit.client.1.smithi097.stdout:0/268: dread da/df/f3f [0,4194304] 0 2024-09-16T08:57:07.431 INFO:tasks.workunit.client.1.smithi097.stdout:0/269: creat da/df/d37/f5d x:0 0 0 2024-09-16T08:57:07.431 INFO:tasks.workunit.client.1.smithi097.stdout:7/261: creat d4/d7/d30/f5b x:0 0 0 2024-09-16T08:57:07.434 INFO:tasks.workunit.client.1.smithi097.stdout:7/262: read d4/d9/d1c/f2d [3694573,21312] 0 2024-09-16T08:57:07.438 INFO:tasks.workunit.client.1.smithi097.stdout:4/301: unlink d0/d7/d52/f59 0 2024-09-16T08:57:07.439 INFO:tasks.workunit.client.1.smithi097.stdout:8/258: mknod d8/dd/c55 0 2024-09-16T08:57:07.445 INFO:tasks.workunit.client.1.smithi097.stdout:5/291: getdents db/d1a 0 2024-09-16T08:57:07.445 INFO:tasks.workunit.client.1.smithi097.stdout:5/292: fdatasync db/d1a/f2c 0 2024-09-16T08:57:07.445 INFO:tasks.workunit.client.1.smithi097.stdout:5/293: write db/d18/f3d [73183,11820] 0 2024-09-16T08:57:07.445 INFO:tasks.workunit.client.1.smithi097.stdout:5/294: chown db/d18/d1c/l2a 1538082 1 2024-09-16T08:57:07.445 INFO:tasks.workunit.client.1.smithi097.stdout:5/295: stat db/d22/c3c 0 2024-09-16T08:57:07.452 INFO:tasks.workunit.client.1.smithi097.stdout:7/263: dwrite d4/d9/d2b/f4d [0,4194304] 0 2024-09-16T08:57:07.452 INFO:tasks.workunit.client.1.smithi097.stdout:7/264: creat d4/d7/d30/f5c x:0 0 0 2024-09-16T08:57:07.452 INFO:tasks.workunit.client.1.smithi097.stdout:7/265: write d4/f5 [1133423,38120] 0 2024-09-16T08:57:07.452 INFO:tasks.workunit.client.1.smithi097.stdout:7/266: chown d4/d1b/d36/d43/f44 578 1 2024-09-16T08:57:07.459 INFO:tasks.workunit.client.1.smithi097.stdout:5/296: dwrite db/d1a/f41 [0,4194304] 0 2024-09-16T08:57:07.462 INFO:tasks.workunit.client.1.smithi097.stdout:2/276: unlink d3/d4/dd/l30 0 2024-09-16T08:57:07.462 INFO:tasks.workunit.client.1.smithi097.stdout:2/277: chown d3/d4/dd/f1f 175 1 2024-09-16T08:57:07.467 INFO:tasks.workunit.client.1.smithi097.stdout:6/315: symlink d1/d3/d3e/d5a/l63 0 2024-09-16T08:57:07.471 INFO:tasks.workunit.client.1.smithi097.stdout:0/270: mkdir da/df/d37/d5e 0 2024-09-16T08:57:07.474 INFO:tasks.workunit.client.1.smithi097.stdout:5/297: dwrite db/f2e [0,4194304] 0 2024-09-16T08:57:07.476 INFO:tasks.workunit.client.1.smithi097.stdout:4/302: rename d0/f4f to d0/d39/d71/f72 0 2024-09-16T08:57:07.479 INFO:tasks.workunit.client.1.smithi097.stdout:3/237: getdents d7/de 0 2024-09-16T08:57:07.479 INFO:tasks.workunit.client.1.smithi097.stdout:3/238: fdatasync d7/fa 0 2024-09-16T08:57:07.479 INFO:tasks.workunit.client.1.smithi097.stdout:3/239: dread - d7/de/f1e zero size 2024-09-16T08:57:07.479 INFO:tasks.workunit.client.1.smithi097.stdout:3/240: fdatasync d7/f44 0 2024-09-16T08:57:07.487 INFO:tasks.workunit.client.1.smithi097.stdout:7/267: symlink d4/d9/d2b/l5d 0 2024-09-16T08:57:07.491 INFO:tasks.workunit.client.1.smithi097.stdout:2/278: mkdir d3/d1b/d3e/d47 0 2024-09-16T08:57:07.491 INFO:tasks.workunit.client.1.smithi097.stdout:2/279: read - d3/f40 zero size 2024-09-16T08:57:07.492 INFO:tasks.workunit.client.1.smithi097.stdout:0/271: creat da/df/d37/d5e/f5f x:0 0 0 2024-09-16T08:57:07.495 INFO:tasks.workunit.client.1.smithi097.stdout:4/303: dwrite d0/d23/f3a [0,4194304] 0 2024-09-16T08:57:07.495 INFO:tasks.workunit.client.1.smithi097.stdout:5/298: dwrite db/d18/d20/d2b/f5c [0,4194304] 0 2024-09-16T08:57:07.500 INFO:tasks.workunit.client.1.smithi097.stdout:9/278: rmdir d2/d5/d3a 39 2024-09-16T08:57:07.502 INFO:tasks.workunit.client.1.smithi097.stdout:3/241: truncate d7/f21 3101592 0 2024-09-16T08:57:07.502 INFO:tasks.workunit.client.1.smithi097.stdout:0/272: write da/f3e [115902,102724] 0 2024-09-16T08:57:07.502 INFO:tasks.workunit.client.1.smithi097.stdout:0/273: chown da/df/d3d 2 1 2024-09-16T08:57:07.505 INFO:tasks.workunit.client.1.smithi097.stdout:7/268: rename d4/d9/l20 to d4/d9/l5e 0 2024-09-16T08:57:07.507 INFO:tasks.workunit.client.1.smithi097.stdout:2/280: mkdir d3/d4/d8/d34/d48 0 2024-09-16T08:57:07.510 INFO:tasks.workunit.client.1.smithi097.stdout:4/304: dwrite d0/d7/df/d15/f41 [0,4194304] 0 2024-09-16T08:57:07.516 INFO:tasks.workunit.client.1.smithi097.stdout:5/299: mkdir db/d22/d61 0 2024-09-16T08:57:07.516 INFO:tasks.workunit.client.1.smithi097.stdout:5/300: chown db/d1a/d50 281038 1 2024-09-16T08:57:07.516 INFO:tasks.workunit.client.1.smithi097.stdout:5/301: chown db/l4a 41549874 1 2024-09-16T08:57:07.517 INFO:tasks.workunit.client.1.smithi097.stdout:9/279: mknod d2/d3f/c59 0 2024-09-16T08:57:07.518 INFO:tasks.workunit.client.1.smithi097.stdout:0/274: stat da/d24/c2c 0 2024-09-16T08:57:07.518 INFO:tasks.workunit.client.1.smithi097.stdout:9/280: read d2/d30/f3b [206172,38729] 0 2024-09-16T08:57:07.520 INFO:tasks.workunit.client.1.smithi097.stdout:7/269: mkdir d4/d9/d1c/d5f 0 2024-09-16T08:57:07.520 INFO:tasks.workunit.client.1.smithi097.stdout:7/270: dread - d4/d12/f4f zero size 2024-09-16T08:57:07.522 INFO:tasks.workunit.client.1.smithi097.stdout:2/281: creat d3/d1b/d3e/d47/f49 x:0 0 0 2024-09-16T08:57:07.522 INFO:tasks.workunit.client.1.smithi097.stdout:2/282: truncate d3/d4/d9/f1e 2130362 0 2024-09-16T08:57:07.528 INFO:tasks.workunit.client.1.smithi097.stdout:0/275: unlink da/df/d13/c1d 0 2024-09-16T08:57:07.528 INFO:tasks.workunit.client.1.smithi097.stdout:7/271: dread d4/d1b/d36/f3a [0,4194304] 0 2024-09-16T08:57:07.529 INFO:tasks.workunit.client.1.smithi097.stdout:7/272: dread - d4/d7/d30/f5b zero size 2024-09-16T08:57:07.529 INFO:tasks.workunit.client.1.smithi097.stdout:9/281: unlink d2/d30/f45 0 2024-09-16T08:57:07.531 INFO:tasks.workunit.client.1.smithi097.stdout:2/283: symlink d3/d4/l4a 0 2024-09-16T08:57:07.531 INFO:tasks.workunit.client.1.smithi097.stdout:2/284: readlink d3/d4/d9/l3b 0 2024-09-16T08:57:07.537 INFO:tasks.workunit.client.1.smithi097.stdout:5/302: rmdir db/d18 39 2024-09-16T08:57:07.539 INFO:tasks.workunit.client.1.smithi097.stdout:0/276: mkdir da/df/d1f/d42/d5b/d60 0 2024-09-16T08:57:07.541 INFO:tasks.workunit.client.1.smithi097.stdout:7/273: creat d4/d9/d1c/f60 x:0 0 0 2024-09-16T08:57:07.543 INFO:tasks.workunit.client.1.smithi097.stdout:9/282: unlink d2/f6 0 2024-09-16T08:57:07.544 INFO:tasks.workunit.client.1.smithi097.stdout:7/274: dread d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:07.544 INFO:tasks.workunit.client.1.smithi097.stdout:7/275: chown d4/d1b/f57 280748 1 2024-09-16T08:57:07.545 INFO:tasks.workunit.client.1.smithi097.stdout:5/303: dread db/f32 [4194304,4194304] 0 2024-09-16T08:57:07.547 INFO:tasks.workunit.client.1.smithi097.stdout:0/277: link da/df/d3d/l56 da/df/d1f/d42/d5b/d60/l61 0 2024-09-16T08:57:07.551 INFO:tasks.workunit.client.1.smithi097.stdout:9/283: readlink d2/d5/l39 0 2024-09-16T08:57:07.552 INFO:tasks.workunit.client.1.smithi097.stdout:7/276: truncate f2 269250 0 2024-09-16T08:57:07.552 INFO:tasks.workunit.client.1.smithi097.stdout:7/277: stat d4 0 2024-09-16T08:57:07.552 INFO:tasks.workunit.client.1.smithi097.stdout:7/278: write d4/f5 [3665037,76707] 0 2024-09-16T08:57:07.553 INFO:tasks.workunit.client.1.smithi097.stdout:5/304: dread db/d1a/f41 [0,4194304] 0 2024-09-16T08:57:07.553 INFO:tasks.workunit.client.1.smithi097.stdout:2/285: getdents d3/d1b 0 2024-09-16T08:57:07.554 INFO:tasks.workunit.client.1.smithi097.stdout:2/286: truncate d3/d1b/d3e/d47/f49 967206 0 2024-09-16T08:57:07.556 INFO:tasks.workunit.client.1.smithi097.stdout:0/278: creat da/df/d1f/f62 x:0 0 0 2024-09-16T08:57:07.557 INFO:tasks.workunit.client.1.smithi097.stdout:5/305: dread db/d1a/f41 [0,4194304] 0 2024-09-16T08:57:07.557 INFO:tasks.workunit.client.1.smithi097.stdout:5/306: stat db/d1a/f38 0 2024-09-16T08:57:07.557 INFO:tasks.workunit.client.1.smithi097.stdout:2/287: dread d3/d4/f21 [0,4194304] 0 2024-09-16T08:57:07.557 INFO:tasks.workunit.client.1.smithi097.stdout:2/288: chown d3/d4/f21 557 1 2024-09-16T08:57:07.561 INFO:tasks.workunit.client.1.smithi097.stdout:7/279: mknod d4/d9/d1c/d5f/c61 0 2024-09-16T08:57:07.561 INFO:tasks.workunit.client.1.smithi097.stdout:7/280: chown f2 236 1 2024-09-16T08:57:07.561 INFO:tasks.workunit.client.1.smithi097.stdout:7/281: chown d4/d12/f4f 677901 1 2024-09-16T08:57:07.566 INFO:tasks.workunit.client.1.smithi097.stdout:5/307: rename db/f32 to db/d18/f62 0 2024-09-16T08:57:07.566 INFO:tasks.workunit.client.1.smithi097.stdout:5/308: write db/d18/f3d [38681,15377] 0 2024-09-16T08:57:07.570 INFO:tasks.workunit.client.1.smithi097.stdout:2/289: rename d3/d4/f21 to d3/d1b/d3e/d47/f4b 0 2024-09-16T08:57:07.571 INFO:tasks.workunit.client.1.smithi097.stdout:2/290: creat d3/d4/d9/f4c x:0 0 0 2024-09-16T08:57:07.572 INFO:tasks.workunit.client.1.smithi097.stdout:7/282: rmdir d4/d27 0 2024-09-16T08:57:07.573 INFO:tasks.workunit.client.1.smithi097.stdout:9/284: dwrite d2/d30/f32 [0,4194304] 0 2024-09-16T08:57:07.576 INFO:tasks.workunit.client.1.smithi097.stdout:2/291: unlink d3/d4/la 0 2024-09-16T08:57:07.576 INFO:tasks.workunit.client.1.smithi097.stdout:2/292: fdatasync d3/f40 0 2024-09-16T08:57:07.577 INFO:tasks.workunit.client.1.smithi097.stdout:2/293: getdents d3/d4/d8/d34/d48 0 2024-09-16T08:57:07.577 INFO:tasks.workunit.client.1.smithi097.stdout:2/294: chown d3/d4/f2d 0 1 2024-09-16T08:57:07.580 INFO:tasks.workunit.client.1.smithi097.stdout:2/295: creat d3/d4/d8/d34/d48/f4d x:0 0 0 2024-09-16T08:57:07.582 INFO:tasks.workunit.client.1.smithi097.stdout:5/309: dwrite db/f2e [0,4194304] 0 2024-09-16T08:57:07.582 INFO:tasks.workunit.client.1.smithi097.stdout:5/310: readlink db/d18/d20/l3f 0 2024-09-16T08:57:07.582 INFO:tasks.workunit.client.1.smithi097.stdout:5/311: readlink db/l11 0 2024-09-16T08:57:07.583 INFO:tasks.workunit.client.1.smithi097.stdout:2/296: mknod d3/d4/c4e 0 2024-09-16T08:57:07.583 INFO:tasks.workunit.client.1.smithi097.stdout:9/285: dread d2/d5/fa [0,4194304] 0 2024-09-16T08:57:07.583 INFO:tasks.workunit.client.1.smithi097.stdout:9/286: chown d2/d30/l49 6482 1 2024-09-16T08:57:07.583 INFO:tasks.workunit.client.1.smithi097.stdout:9/287: chown d2/c58 51 1 2024-09-16T08:57:07.583 INFO:tasks.workunit.client.1.smithi097.stdout:9/288: creat d2/d30/f5a x:0 0 0 2024-09-16T08:57:07.584 INFO:tasks.workunit.client.1.smithi097.stdout:9/289: fsync d2/d30/f32 0 2024-09-16T08:57:07.585 INFO:tasks.workunit.client.1.smithi097.stdout:7/283: dread d4/d9/d1c/f34 [0,4194304] 0 2024-09-16T08:57:07.596 INFO:tasks.workunit.client.1.smithi097.stdout:5/312: rename db/f10 to db/d22/d61/f63 0 2024-09-16T08:57:07.596 INFO:tasks.workunit.client.1.smithi097.stdout:5/313: getdents db/d22/d58 0 2024-09-16T08:57:07.596 INFO:tasks.workunit.client.1.smithi097.stdout:5/314: chown db/d18/d1c/l2a 91 1 2024-09-16T08:57:07.597 INFO:tasks.workunit.client.1.smithi097.stdout:5/315: chown db/d1a/f41 68455 1 2024-09-16T08:57:07.597 INFO:tasks.workunit.client.1.smithi097.stdout:2/297: link d3/d4/d8/f2b d3/d4/d9/f4f 0 2024-09-16T08:57:07.598 INFO:tasks.workunit.client.1.smithi097.stdout:9/290: dwrite d2/d30/f3b [0,4194304] 0 2024-09-16T08:57:07.601 INFO:tasks.workunit.client.1.smithi097.stdout:7/284: symlink d4/d1b/d41/l62 0 2024-09-16T08:57:07.601 INFO:tasks.workunit.client.1.smithi097.stdout:7/285: read - d4/d9/d1c/f60 zero size 2024-09-16T08:57:07.601 INFO:tasks.workunit.client.1.smithi097.stdout:7/286: read d4/d7/d19/f33 [44837,84165] 0 2024-09-16T08:57:07.601 INFO:tasks.workunit.client.1.smithi097.stdout:7/287: write d4/d7/ff [746033,42475] 0 2024-09-16T08:57:07.604 INFO:tasks.workunit.client.1.smithi097.stdout:5/316: dread db/f14 [0,4194304] 0 2024-09-16T08:57:07.608 INFO:tasks.workunit.client.1.smithi097.stdout:9/291: mknod d2/d30/c5b 0 2024-09-16T08:57:07.610 INFO:tasks.workunit.client.1.smithi097.stdout:7/288: mknod d4/d12/d1f/d22/d5a/c63 0 2024-09-16T08:57:07.612 INFO:tasks.workunit.client.1.smithi097.stdout:1/335: sync 2024-09-16T08:57:07.612 INFO:tasks.workunit.client.1.smithi097.stdout:8/259: sync 2024-09-16T08:57:07.612 INFO:tasks.workunit.client.1.smithi097.stdout:8/260: readlink d8/l2a 0 2024-09-16T08:57:07.612 INFO:tasks.workunit.client.1.smithi097.stdout:1/336: chown d5/db/dc 776403 1 2024-09-16T08:57:07.612 INFO:tasks.workunit.client.1.smithi097.stdout:8/261: read - d8/dd/d15/f34 zero size 2024-09-16T08:57:07.612 INFO:tasks.workunit.client.1.smithi097.stdout:8/262: write d8/dd/f2f [9256284,57759] 0 2024-09-16T08:57:07.614 INFO:tasks.workunit.client.1.smithi097.stdout:5/317: rename db/d22/d61 to db/d18/d1c/d42/d48/d64 0 2024-09-16T08:57:07.615 INFO:tasks.workunit.client.1.smithi097.stdout:2/298: fsync d3/d4/d9/f4f 0 2024-09-16T08:57:07.615 INFO:tasks.workunit.client.1.smithi097.stdout:2/299: dread - d3/d4/d9/f4c zero size 2024-09-16T08:57:07.616 INFO:tasks.workunit.client.1.smithi097.stdout:9/292: symlink d2/d3f/l5c 0 2024-09-16T08:57:07.618 INFO:tasks.workunit.client.1.smithi097.stdout:8/263: mkdir d8/dd/d15/d2c/d48/d56 0 2024-09-16T08:57:07.621 INFO:tasks.workunit.client.1.smithi097.stdout:7/289: rename d4/d12/d1f/d22/f56 to d4/f64 0 2024-09-16T08:57:07.624 INFO:tasks.workunit.client.1.smithi097.stdout:7/290: fdatasync d4/d7/ff 0 2024-09-16T08:57:07.624 INFO:tasks.workunit.client.1.smithi097.stdout:7/291: readlink d4/d9/l4a 0 2024-09-16T08:57:07.624 INFO:tasks.workunit.client.1.smithi097.stdout:7/292: fsync d4/d9/dc/f16 0 2024-09-16T08:57:07.625 INFO:tasks.workunit.client.1.smithi097.stdout:2/300: unlink d3/d4/d8/l2f 0 2024-09-16T08:57:07.625 INFO:tasks.workunit.client.1.smithi097.stdout:2/301: write d3/d1b/d3e/d47/f49 [1136642,111444] 0 2024-09-16T08:57:07.626 INFO:tasks.workunit.client.1.smithi097.stdout:9/293: creat d2/d5/d56/f5d x:0 0 0 2024-09-16T08:57:07.626 INFO:tasks.workunit.client.1.smithi097.stdout:9/294: write d2/f51 [101061,117361] 0 2024-09-16T08:57:07.626 INFO:tasks.workunit.client.1.smithi097.stdout:8/264: fsync d8/dd/d15/d16/f3a 0 2024-09-16T08:57:07.626 INFO:tasks.workunit.client.1.smithi097.stdout:8/265: write d8/dd/d15/d2c/d4a/d4d/f35 [875067,46224] 0 2024-09-16T08:57:07.627 INFO:tasks.workunit.client.1.smithi097.stdout:1/337: rename d5/c40 to d5/d2c/c66 0 2024-09-16T08:57:07.628 INFO:tasks.workunit.client.1.smithi097.stdout:5/318: symlink db/d18/l65 0 2024-09-16T08:57:07.628 INFO:tasks.workunit.client.1.smithi097.stdout:5/319: fdatasync db/d18/d1c/d42/f4c 0 2024-09-16T08:57:07.630 INFO:tasks.workunit.client.1.smithi097.stdout:7/293: link d4/d1b/l26 d4/l65 0 2024-09-16T08:57:07.631 INFO:tasks.workunit.client.1.smithi097.stdout:7/294: write d4/d12/d1f/f21 [731362,47635] 0 2024-09-16T08:57:07.632 INFO:tasks.workunit.client.1.smithi097.stdout:2/302: dread d3/d4/dd/f18 [4194304,4194304] 0 2024-09-16T08:57:07.632 INFO:tasks.workunit.client.1.smithi097.stdout:2/303: chown d3/d4/d9/f36 88 1 2024-09-16T08:57:07.636 INFO:tasks.workunit.client.1.smithi097.stdout:2/304: dread d3/d4/d8/f39 [0,4194304] 0 2024-09-16T08:57:07.637 INFO:tasks.workunit.client.1.smithi097.stdout:8/266: creat d8/dd/d15/d2c/d4a/d4d/d3d/f57 x:0 0 0 2024-09-16T08:57:07.637 INFO:tasks.workunit.client.1.smithi097.stdout:8/267: truncate d8/dd/d15/d2c/f41 15929 0 2024-09-16T08:57:07.637 INFO:tasks.workunit.client.1.smithi097.stdout:8/268: write d8/f10 [4729756,123603] 0 2024-09-16T08:57:07.638 INFO:tasks.workunit.client.1.smithi097.stdout:1/338: mknod d5/d14/c67 0 2024-09-16T08:57:07.638 INFO:tasks.workunit.client.1.smithi097.stdout:2/305: dread d3/d1b/d3e/d47/f49 [0,4194304] 0 2024-09-16T08:57:07.649 INFO:tasks.workunit.client.1.smithi097.stdout:7/295: symlink d4/d9/dc/l66 0 2024-09-16T08:57:07.649 INFO:tasks.workunit.client.1.smithi097.stdout:7/296: fdatasync d4/d1b/d36/f40 0 2024-09-16T08:57:07.649 INFO:tasks.workunit.client.1.smithi097.stdout:7/297: chown d4/d9/l4a 5379857 1 2024-09-16T08:57:07.649 INFO:tasks.workunit.client.1.smithi097.stdout:9/295: rmdir d2/d3f 39 2024-09-16T08:57:07.653 INFO:tasks.workunit.client.1.smithi097.stdout:1/339: rename d5/db/l62 to d5/db/dc/d19/l68 0 2024-09-16T08:57:07.656 INFO:tasks.workunit.client.1.smithi097.stdout:2/306: creat d3/d1b/d3e/f50 x:0 0 0 2024-09-16T08:57:07.656 INFO:tasks.workunit.client.1.smithi097.stdout:2/307: chown d3/d4/d9/f4c 9706 1 2024-09-16T08:57:07.657 INFO:tasks.workunit.client.1.smithi097.stdout:1/340: dread d5/db/dc/f33 [0,4194304] 0 2024-09-16T08:57:07.659 INFO:tasks.workunit.client.1.smithi097.stdout:1/341: write d5/db/dc/f41 [397360,65952] 0 2024-09-16T08:57:07.659 INFO:tasks.workunit.client.1.smithi097.stdout:1/342: dread - d5/d2c/f5e zero size 2024-09-16T08:57:07.663 INFO:tasks.workunit.client.1.smithi097.stdout:1/343: unlink d5/c8 0 2024-09-16T08:57:07.674 INFO:tasks.workunit.client.1.smithi097.stdout:6/316: sync 2024-09-16T08:57:07.674 INFO:tasks.workunit.client.1.smithi097.stdout:4/305: sync 2024-09-16T08:57:07.674 INFO:tasks.workunit.client.1.smithi097.stdout:3/242: sync 2024-09-16T08:57:07.674 INFO:tasks.workunit.client.1.smithi097.stdout:4/306: fdatasync d0/f2a 0 2024-09-16T08:57:07.674 INFO:tasks.workunit.client.1.smithi097.stdout:3/243: creat d7/de/d3c/f4d x:0 0 0 2024-09-16T08:57:07.679 INFO:tasks.workunit.client.1.smithi097.stdout:4/307: rename d0/d23/c58 to d0/d7/df/c73 0 2024-09-16T08:57:07.683 INFO:tasks.workunit.client.1.smithi097.stdout:4/308: rename d0/d7/d52/d26/d3d/c6e to d0/c74 0 2024-09-16T08:57:07.683 INFO:tasks.workunit.client.1.smithi097.stdout:4/309: creat d0/d7/f75 x:0 0 0 2024-09-16T08:57:07.683 INFO:tasks.workunit.client.1.smithi097.stdout:4/310: chown d0/d36/d4c 1432152 1 2024-09-16T08:57:07.687 INFO:tasks.workunit.client.1.smithi097.stdout:4/311: creat d0/d7/d52/d26/d3d/f76 x:0 0 0 2024-09-16T08:57:07.687 INFO:tasks.workunit.client.1.smithi097.stdout:4/312: chown d0/c1c 2478539 1 2024-09-16T08:57:07.689 INFO:tasks.workunit.client.1.smithi097.stdout:4/313: creat d0/d39/d71/f77 x:0 0 0 2024-09-16T08:57:07.689 INFO:tasks.workunit.client.1.smithi097.stdout:4/314: read - d0/d35/f38 zero size 2024-09-16T08:57:07.690 INFO:tasks.workunit.client.1.smithi097.stdout:8/269: dwrite d8/dd/d42/f51 [0,4194304] 0 2024-09-16T08:57:07.691 INFO:tasks.workunit.client.1.smithi097.stdout:8/270: chown d8/dd/d15/d2c/d4a/d4d/l3e 31 1 2024-09-16T08:57:07.694 INFO:tasks.workunit.client.1.smithi097.stdout:4/315: link d0/d7/df/f1e d0/d7/d52/f78 0 2024-09-16T08:57:07.694 INFO:tasks.workunit.client.1.smithi097.stdout:4/316: chown d0/d35/f38 7385509 1 2024-09-16T08:57:07.698 INFO:tasks.workunit.client.1.smithi097.stdout:8/271: rename d8/dd/c17 to d8/dd/d15/d2c/c58 0 2024-09-16T08:57:07.722 INFO:tasks.workunit.client.1.smithi097.stdout:7/298: dwrite d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:07.722 INFO:tasks.workunit.client.1.smithi097.stdout:7/299: fdatasync d4/d7/d30/f5c 0 2024-09-16T08:57:07.727 INFO:tasks.workunit.client.1.smithi097.stdout:8/272: dread d8/dd/d15/d2c/d4a/d4d/f49 [4194304,4194304] 0 2024-09-16T08:57:07.727 INFO:tasks.workunit.client.1.smithi097.stdout:8/273: chown d8/dd/c55 7937 1 2024-09-16T08:57:07.734 INFO:tasks.workunit.client.1.smithi097.stdout:9/296: dwrite d2/d5/d3a/d3c/f54 [0,4194304] 0 2024-09-16T08:57:07.734 INFO:tasks.workunit.client.1.smithi097.stdout:6/317: dwrite d1/d3/d3e/f62 [0,4194304] 0 2024-09-16T08:57:07.735 INFO:tasks.workunit.client.1.smithi097.stdout:2/308: dwrite d3/f32 [0,4194304] 0 2024-09-16T08:57:07.735 INFO:tasks.workunit.client.1.smithi097.stdout:7/300: write d4/d9/d2b/f4d [2467086,105447] 0 2024-09-16T08:57:07.735 INFO:tasks.workunit.client.1.smithi097.stdout:4/317: dwrite d0/d7/d52/f1d [4194304,4194304] 0 2024-09-16T08:57:07.736 INFO:tasks.workunit.client.1.smithi097.stdout:4/318: creat d0/d39/d71/f79 x:0 0 0 2024-09-16T08:57:07.736 INFO:tasks.workunit.client.1.smithi097.stdout:4/319: chown d0/d7/d52/f78 562 1 2024-09-16T08:57:07.736 INFO:tasks.workunit.client.1.smithi097.stdout:9/297: write d2/d5/f4e [402491,76400] 0 2024-09-16T08:57:07.739 INFO:tasks.workunit.client.1.smithi097.stdout:8/274: dread d8/dd/d15/f28 [0,4194304] 0 2024-09-16T08:57:07.739 INFO:tasks.workunit.client.1.smithi097.stdout:3/244: dwrite d7/de/d29/d35/f27 [0,4194304] 0 2024-09-16T08:57:07.741 INFO:tasks.workunit.client.1.smithi097.stdout:4/320: dread d0/d39/f6f [0,4194304] 0 2024-09-16T08:57:07.741 INFO:tasks.workunit.client.1.smithi097.stdout:4/321: chown d0/d23/d5f 17234 1 2024-09-16T08:57:07.744 INFO:tasks.workunit.client.1.smithi097.stdout:8/275: dread d8/dd/d15/f28 [0,4194304] 0 2024-09-16T08:57:07.745 INFO:tasks.workunit.client.1.smithi097.stdout:8/276: chown d8/dd/d15/d2c/d4a/d4d/d3d/f57 41322664 1 2024-09-16T08:57:07.749 INFO:tasks.workunit.client.1.smithi097.stdout:2/309: mkdir d3/d4/d8/d51 0 2024-09-16T08:57:07.750 INFO:tasks.workunit.client.1.smithi097.stdout:7/301: truncate d4/f5 3281398 0 2024-09-16T08:57:07.754 INFO:tasks.workunit.client.1.smithi097.stdout:3/245: rmdir d7/de/d29/d35 39 2024-09-16T08:57:07.755 INFO:tasks.workunit.client.1.smithi097.stdout:3/246: fsync d7/de/d29/f2d 0 2024-09-16T08:57:07.757 INFO:tasks.workunit.client.1.smithi097.stdout:2/310: read d3/d4/dd/f35 [6331781,67982] 0 2024-09-16T08:57:07.758 INFO:tasks.workunit.client.1.smithi097.stdout:6/318: dwrite d1/d3/f3d [0,4194304] 0 2024-09-16T08:57:07.758 INFO:tasks.workunit.client.1.smithi097.stdout:6/319: creat d1/f64 x:0 0 0 2024-09-16T08:57:07.758 INFO:tasks.workunit.client.1.smithi097.stdout:9/298: dwrite d2/d5/d3a/f3d [0,4194304] 0 2024-09-16T08:57:07.765 INFO:tasks.workunit.client.1.smithi097.stdout:8/277: link d8/dd/c27 d8/dd/d15/d2c/d4a/d4d/d3d/c59 0 2024-09-16T08:57:07.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/302: rename d4/d12/l39 to d4/d12/l67 0 2024-09-16T08:57:07.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/303: truncate d4/f53 254513 0 2024-09-16T08:57:07.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/304: truncate d4/d9/d1c/f60 28070 0 2024-09-16T08:57:07.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/305: chown f2 3310775 1 2024-09-16T08:57:07.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/306: creat d4/d7/d30/f68 x:0 0 0 2024-09-16T08:57:07.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/307: readlink d4/d1b/d36/d43/l54 0 2024-09-16T08:57:07.770 INFO:tasks.workunit.client.1.smithi097.stdout:7/308: chown d4/d7 88644 1 2024-09-16T08:57:07.771 INFO:tasks.workunit.client.1.smithi097.stdout:3/247: symlink d7/de/l4e 0 2024-09-16T08:57:07.775 INFO:tasks.workunit.client.1.smithi097.stdout:2/311: symlink d3/d4/d8/d51/l52 0 2024-09-16T08:57:07.779 INFO:tasks.workunit.client.1.smithi097.stdout:6/320: dwrite d1/d3/d3e/d5a/f47 [0,4194304] 0 2024-09-16T08:57:07.786 INFO:tasks.workunit.client.1.smithi097.stdout:4/322: getdents d0/d7/df 0 2024-09-16T08:57:07.787 INFO:tasks.workunit.client.1.smithi097.stdout:8/278: dwrite d8/dd/d15/d2c/d4a/d4d/f35 [0,4194304] 0 2024-09-16T08:57:07.787 INFO:tasks.workunit.client.1.smithi097.stdout:7/309: dread d4/d12/d1f/d22/f3d [0,4194304] 0 2024-09-16T08:57:07.787 INFO:tasks.workunit.client.1.smithi097.stdout:7/310: dread - d4/d7/d30/f5c zero size 2024-09-16T08:57:07.800 INFO:tasks.workunit.client.1.smithi097.stdout:3/248: dwrite d7/d15/f17 [0,4194304] 0 2024-09-16T08:57:07.802 INFO:tasks.workunit.client.1.smithi097.stdout:6/321: mknod d1/d3/d3e/c65 0 2024-09-16T08:57:07.805 INFO:tasks.workunit.client.1.smithi097.stdout:6/322: dread d1/d3/d16/f2e [0,4194304] 0 2024-09-16T08:57:07.805 INFO:tasks.workunit.client.1.smithi097.stdout:9/299: symlink d2/d5/l5e 0 2024-09-16T08:57:07.805 INFO:tasks.workunit.client.1.smithi097.stdout:9/300: chown d2/d3f/c59 0 1 2024-09-16T08:57:07.806 INFO:tasks.workunit.client.1.smithi097.stdout:7/311: dwrite d4/d9/dc/f48 [0,4194304] 0 2024-09-16T08:57:07.807 INFO:tasks.workunit.client.1.smithi097.stdout:4/323: link d0/d39/d71/l6d d0/d7/l7a 0 2024-09-16T08:57:07.807 INFO:tasks.workunit.client.1.smithi097.stdout:4/324: stat d0/d7/d52/d26/d3d/f44 0 2024-09-16T08:57:07.807 INFO:tasks.workunit.client.1.smithi097.stdout:4/325: read - d0/f30 zero size 2024-09-16T08:57:07.807 INFO:tasks.workunit.client.1.smithi097.stdout:4/326: chown d0/d7/d52/f67 13 1 2024-09-16T08:57:07.808 INFO:tasks.workunit.client.1.smithi097.stdout:4/327: creat d0/d23/f7b x:0 0 0 2024-09-16T08:57:07.808 INFO:tasks.workunit.client.1.smithi097.stdout:4/328: truncate d0/d7/df/d15/f1b 527072 0 2024-09-16T08:57:07.813 INFO:tasks.workunit.client.1.smithi097.stdout:8/279: creat d8/dd/d15/d2c/d4a/d4e/f5a x:0 0 0 2024-09-16T08:57:07.820 INFO:tasks.workunit.client.1.smithi097.stdout:8/280: dread d8/dd/d15/f29 [0,4194304] 0 2024-09-16T08:57:07.820 INFO:tasks.workunit.client.1.smithi097.stdout:8/281: creat d8/dd/d15/d2c/d4a/d4d/d3d/f5b x:0 0 0 2024-09-16T08:57:07.820 INFO:tasks.workunit.client.1.smithi097.stdout:8/282: fsync d8/f10 0 2024-09-16T08:57:07.822 INFO:tasks.workunit.client.1.smithi097.stdout:3/249: mkdir d7/de/d3c/d4f 0 2024-09-16T08:57:07.826 INFO:tasks.workunit.client.1.smithi097.stdout:7/312: creat d4/d1b/f69 x:0 0 0 2024-09-16T08:57:07.827 INFO:tasks.workunit.client.1.smithi097.stdout:9/301: dwrite d2/d30/f5a [0,4194304] 0 2024-09-16T08:57:07.829 INFO:tasks.workunit.client.1.smithi097.stdout:6/323: dwrite d1/dc/d1e/f3a [0,4194304] 0 2024-09-16T08:57:07.831 INFO:tasks.workunit.client.1.smithi097.stdout:2/312: rename d3/d1b/d3e to d3/d4/dd/d53 0 2024-09-16T08:57:07.834 INFO:tasks.workunit.client.1.smithi097.stdout:8/283: creat d8/dd/d15/d2c/d4a/d4d/f5c x:0 0 0 2024-09-16T08:57:07.835 INFO:tasks.workunit.client.1.smithi097.stdout:7/313: symlink d4/d9/d1c/d5f/l6a 0 2024-09-16T08:57:07.845 INFO:tasks.workunit.client.1.smithi097.stdout:4/329: rmdir d0/d7/d52 39 2024-09-16T08:57:07.848 INFO:tasks.workunit.client.1.smithi097.stdout:3/250: dwrite d7/de/d29/d35/f27 [4194304,4194304] 0 2024-09-16T08:57:07.851 INFO:tasks.workunit.client.1.smithi097.stdout:7/314: creat d4/d9/dc/f6b x:0 0 0 2024-09-16T08:57:07.851 INFO:tasks.workunit.client.1.smithi097.stdout:7/315: chown d4/d9/d1c/f28 3447928 1 2024-09-16T08:57:07.854 INFO:tasks.workunit.client.1.smithi097.stdout:9/302: link d2/d5/f11 d2/d5/d3a/d3c/f5f 0 2024-09-16T08:57:07.855 INFO:tasks.workunit.client.1.smithi097.stdout:9/303: write d2/d3f/f43 [657828,8369] 0 2024-09-16T08:57:07.857 INFO:tasks.workunit.client.1.smithi097.stdout:7/316: dread d4/d9/dc/f49 [0,4194304] 0 2024-09-16T08:57:07.857 INFO:tasks.workunit.client.1.smithi097.stdout:7/317: chown d4/d9/c13 10 1 2024-09-16T08:57:07.860 INFO:tasks.workunit.client.1.smithi097.stdout:3/251: link d7/f44 d7/de/d3c/d3f/f50 0 2024-09-16T08:57:07.860 INFO:tasks.workunit.client.1.smithi097.stdout:3/252: write d7/de/f1e [652908,18019] 0 2024-09-16T08:57:07.860 INFO:tasks.workunit.client.1.smithi097.stdout:3/253: creat d7/de/d3c/d3f/f51 x:0 0 0 2024-09-16T08:57:07.865 INFO:tasks.workunit.client.1.smithi097.stdout:8/284: dwrite d8/dd/f19 [0,4194304] 0 2024-09-16T08:57:07.865 INFO:tasks.workunit.client.1.smithi097.stdout:9/304: dread d2/fc [0,4194304] 0 2024-09-16T08:57:07.865 INFO:tasks.workunit.client.1.smithi097.stdout:2/313: dwrite d3/d4/d9/f4f [0,4194304] 0 2024-09-16T08:57:07.866 INFO:tasks.workunit.client.1.smithi097.stdout:7/318: creat d4/d9/d1c/d5f/f6c x:0 0 0 2024-09-16T08:57:07.868 INFO:tasks.workunit.client.1.smithi097.stdout:2/314: dread d3/d4/dd/d53/d47/f49 [0,4194304] 0 2024-09-16T08:57:07.871 INFO:tasks.workunit.client.1.smithi097.stdout:3/254: write d7/f44 [1606972,93527] 0 2024-09-16T08:57:07.873 INFO:tasks.workunit.client.1.smithi097.stdout:8/285: mknod d8/dd/d15/d2c/c5d 0 2024-09-16T08:57:07.877 INFO:tasks.workunit.client.1.smithi097.stdout:7/319: dread d4/f23 [0,4194304] 0 2024-09-16T08:57:07.877 INFO:tasks.workunit.client.1.smithi097.stdout:9/305: rename d2/d30/c4a to d2/d3f/c60 0 2024-09-16T08:57:07.879 INFO:tasks.workunit.client.1.smithi097.stdout:2/315: dread d3/d4/dd/f1f [0,4194304] 0 2024-09-16T08:57:07.880 INFO:tasks.workunit.client.1.smithi097.stdout:7/320: truncate d4/d9/d1c/f34 537356 0 2024-09-16T08:57:07.885 INFO:tasks.workunit.client.1.smithi097.stdout:2/316: truncate d3/d4/d8/f33 611557 0 2024-09-16T08:57:07.888 INFO:tasks.workunit.client.1.smithi097.stdout:3/255: dwrite d7/f21 [0,4194304] 0 2024-09-16T08:57:07.888 INFO:tasks.workunit.client.1.smithi097.stdout:7/321: symlink d4/l6d 0 2024-09-16T08:57:07.888 INFO:tasks.workunit.client.1.smithi097.stdout:7/322: stat d4/d9/d2b/c3c 0 2024-09-16T08:57:07.890 INFO:tasks.workunit.client.1.smithi097.stdout:8/286: rename d8/dd/d15/d2c/d4a/d4d/l3e to d8/dd/d15/d16/l5e 0 2024-09-16T08:57:07.893 INFO:tasks.workunit.client.1.smithi097.stdout:3/256: mkdir d7/d15/d52 0 2024-09-16T08:57:07.893 INFO:tasks.workunit.client.1.smithi097.stdout:3/257: chown d7/l11 2064452 1 2024-09-16T08:57:07.896 INFO:tasks.workunit.client.1.smithi097.stdout:2/317: write d3/d4/d9/f4f [1676977,45775] 0 2024-09-16T08:57:07.896 INFO:tasks.workunit.client.1.smithi097.stdout:7/323: rename d4/d12/d1f/d22/d5a/c63 to d4/d12/c6e 0 2024-09-16T08:57:07.896 INFO:tasks.workunit.client.1.smithi097.stdout:2/318: rename d3/d4/dd to d3/d4/dd/d53/d54 22 2024-09-16T08:57:07.897 INFO:tasks.workunit.client.1.smithi097.stdout:7/324: fsync d4/d12/d1f/f21 0 2024-09-16T08:57:07.898 INFO:tasks.workunit.client.1.smithi097.stdout:8/287: truncate d8/f31 19365 0 2024-09-16T08:57:07.898 INFO:tasks.workunit.client.1.smithi097.stdout:8/288: fdatasync d8/dd/d15/f28 0 2024-09-16T08:57:07.899 INFO:tasks.workunit.client.1.smithi097.stdout:3/258: rmdir d7/de/d29/d35/d3d 39 2024-09-16T08:57:07.902 INFO:tasks.workunit.client.1.smithi097.stdout:8/289: write d8/dd/d15/d2c/d4a/d4d/f49 [5854190,3750] 0 2024-09-16T08:57:07.902 INFO:tasks.workunit.client.1.smithi097.stdout:8/290: write d8/dd/d15/f29 [4005465,36420] 0 2024-09-16T08:57:07.902 INFO:tasks.workunit.client.1.smithi097.stdout:2/319: unlink d3/d4/dd/f46 0 2024-09-16T08:57:07.902 INFO:tasks.workunit.client.1.smithi097.stdout:2/320: chown d3/d4/d8/d51/l52 9456083 1 2024-09-16T08:57:07.905 INFO:tasks.workunit.client.1.smithi097.stdout:7/325: rename d4/f53 to d4/f6f 0 2024-09-16T08:57:07.905 INFO:tasks.workunit.client.1.smithi097.stdout:7/326: write d4/d9/d1c/f28 [841102,88198] 0 2024-09-16T08:57:07.906 INFO:tasks.workunit.client.1.smithi097.stdout:3/259: creat d7/d28/d30/f53 x:0 0 0 2024-09-16T08:57:07.907 INFO:tasks.workunit.client.1.smithi097.stdout:4/330: getdents d0/d39/d71 0 2024-09-16T08:57:07.907 INFO:tasks.workunit.client.1.smithi097.stdout:4/331: dread - d0/d7/f75 zero size 2024-09-16T08:57:07.907 INFO:tasks.workunit.client.1.smithi097.stdout:4/332: write d0/d39/f6f [1079822,17819] 0 2024-09-16T08:57:07.907 INFO:tasks.workunit.client.1.smithi097.stdout:4/333: readlink d0/d7/df/l4a 0 2024-09-16T08:57:07.910 INFO:tasks.workunit.client.1.smithi097.stdout:2/321: write d3/f27 [1870593,52414] 0 2024-09-16T08:57:07.910 INFO:tasks.workunit.client.1.smithi097.stdout:2/322: stat d3/d4/d8/d34/d48/f4d 0 2024-09-16T08:57:07.910 INFO:tasks.workunit.client.1.smithi097.stdout:2/323: chown d3/d4/dd/d53/d47/f4b 379492459 1 2024-09-16T08:57:07.912 INFO:tasks.workunit.client.1.smithi097.stdout:8/291: getdents d8/dd/d42 0 2024-09-16T08:57:07.912 INFO:tasks.workunit.client.1.smithi097.stdout:8/292: chown d8/dd/d15/l1d 7 1 2024-09-16T08:57:07.912 INFO:tasks.workunit.client.1.smithi097.stdout:8/293: write d8/dd/d15/d2c/d4a/d4e/f5a [220609,92625] 0 2024-09-16T08:57:07.912 INFO:tasks.workunit.client.1.smithi097.stdout:8/294: truncate d8/dd/d15/f34 312355 0 2024-09-16T08:57:07.912 INFO:tasks.workunit.client.1.smithi097.stdout:8/295: readlink d8/dd/l38 0 2024-09-16T08:57:07.912 INFO:tasks.workunit.client.1.smithi097.stdout:8/296: write d8/dd/d15/d2c/f41 [911582,56381] 0 2024-09-16T08:57:07.914 INFO:tasks.workunit.client.1.smithi097.stdout:7/327: mknod d4/d1b/d36/c70 0 2024-09-16T08:57:07.915 INFO:tasks.workunit.client.1.smithi097.stdout:3/260: getdents d7/de/d29/d35/d3d 0 2024-09-16T08:57:07.917 INFO:tasks.workunit.client.1.smithi097.stdout:0/279: sync 2024-09-16T08:57:07.917 INFO:tasks.workunit.client.1.smithi097.stdout:0/280: creat da/d1a/f63 x:0 0 0 2024-09-16T08:57:07.917 INFO:tasks.workunit.client.1.smithi097.stdout:0/281: write da/d24/f33 [604493,115998] 0 2024-09-16T08:57:07.917 INFO:tasks.workunit.client.1.smithi097.stdout:0/282: truncate da/d1a/d26/f59 693937 0 2024-09-16T08:57:07.930 INFO:tasks.workunit.client.1.smithi097.stdout:4/334: unlink d0/d36/f54 0 2024-09-16T08:57:07.932 INFO:tasks.workunit.client.1.smithi097.stdout:2/324: mknod d3/d1b/c55 0 2024-09-16T08:57:07.935 INFO:tasks.workunit.client.1.smithi097.stdout:8/297: unlink d8/dd/d15/d2c/c58 0 2024-09-16T08:57:07.938 INFO:tasks.workunit.client.1.smithi097.stdout:3/261: symlink d7/de/d3c/d3f/l54 0 2024-09-16T08:57:07.942 INFO:tasks.workunit.client.1.smithi097.stdout:0/283: truncate da/df/d1f/f3c 2581812 0 2024-09-16T08:57:07.944 INFO:tasks.workunit.client.1.smithi097.stdout:4/335: mknod d0/d23/d5f/c7c 0 2024-09-16T08:57:07.946 INFO:tasks.workunit.client.1.smithi097.stdout:2/325: mknod d3/d4/dd/c56 0 2024-09-16T08:57:07.948 INFO:tasks.workunit.client.1.smithi097.stdout:8/298: symlink d8/dd/d42/l5f 0 2024-09-16T08:57:07.954 INFO:tasks.workunit.client.1.smithi097.stdout:0/284: link da/l4f da/d24/d4b/l64 0 2024-09-16T08:57:07.954 INFO:tasks.workunit.client.1.smithi097.stdout:0/285: write da/d1a/f22 [1545367,56572] 0 2024-09-16T08:57:07.957 INFO:tasks.workunit.client.1.smithi097.stdout:4/336: mkdir d0/d23/d5f/d7d 0 2024-09-16T08:57:07.957 INFO:tasks.workunit.client.1.smithi097.stdout:4/337: chown d0/d7/d52/f67 12787834 1 2024-09-16T08:57:07.969 INFO:tasks.workunit.client.1.smithi097.stdout:2/326: dwrite d3/d4/d9/f4f [0,4194304] 0 2024-09-16T08:57:07.969 INFO:tasks.workunit.client.1.smithi097.stdout:2/327: fsync d3/d4/d8/f2b 0 2024-09-16T08:57:07.969 INFO:tasks.workunit.client.1.smithi097.stdout:2/328: creat d3/f57 x:0 0 0 2024-09-16T08:57:07.969 INFO:tasks.workunit.client.1.smithi097.stdout:0/286: creat da/df/d1f/d42/d5b/d60/f65 x:0 0 0 2024-09-16T08:57:07.969 INFO:tasks.workunit.client.1.smithi097.stdout:0/287: write da/d1a/f2b [1938690,85195] 0 2024-09-16T08:57:07.969 INFO:tasks.workunit.client.1.smithi097.stdout:0/288: dread - da/d1a/f63 zero size 2024-09-16T08:57:07.969 INFO:tasks.workunit.client.1.smithi097.stdout:0/289: creat da/df/d3d/f66 x:0 0 0 2024-09-16T08:57:07.981 INFO:tasks.workunit.client.1.smithi097.stdout:2/329: unlink d3/f57 0 2024-09-16T08:57:07.981 INFO:tasks.workunit.client.1.smithi097.stdout:2/330: fsync d3/f23 0 2024-09-16T08:57:07.983 INFO:tasks.workunit.client.1.smithi097.stdout:0/290: dwrite da/df/f16 [4194304,4194304] 0 2024-09-16T08:57:07.991 INFO:tasks.workunit.client.1.smithi097.stdout:0/291: symlink da/df/d3d/l67 0 2024-09-16T08:57:08.004 INFO:tasks.workunit.client.1.smithi097.stdout:3/262: fsync d7/f44 0 2024-09-16T08:57:08.005 INFO:tasks.workunit.client.1.smithi097.stdout:0/292: dwrite da/d1a/f3a [0,4194304] 0 2024-09-16T08:57:08.005 INFO:tasks.workunit.client.1.smithi097.stdout:0/293: write da/d1a/f63 [487074,2354] 0 2024-09-16T08:57:08.011 INFO:tasks.workunit.client.1.smithi097.stdout:3/263: mknod d7/de/d29/c55 0 2024-09-16T08:57:08.011 INFO:tasks.workunit.client.1.smithi097.stdout:3/264: write d7/de/d29/f49 [569390,13706] 0 2024-09-16T08:57:08.011 INFO:tasks.workunit.client.1.smithi097.stdout:3/265: readlink d7/de/l1a 0 2024-09-16T08:57:08.011 INFO:tasks.workunit.client.1.smithi097.stdout:3/266: chown d7/d43 10887 1 2024-09-16T08:57:08.013 INFO:tasks.workunit.client.1.smithi097.stdout:9/306: getdents d2/d3f 0 2024-09-16T08:57:08.017 INFO:tasks.workunit.client.1.smithi097.stdout:0/294: rename da/df/d37 to da/df/d1f/d42/d5b/d60/d68 0 2024-09-16T08:57:08.017 INFO:tasks.workunit.client.1.smithi097.stdout:0/295: fdatasync f6 0 2024-09-16T08:57:08.019 INFO:tasks.workunit.client.1.smithi097.stdout:7/328: read d4/d9/d1c/f34 [297315,26471] 0 2024-09-16T08:57:08.020 INFO:tasks.workunit.client.1.smithi097.stdout:3/267: mknod d7/de/d29/c56 0 2024-09-16T08:57:08.020 INFO:tasks.workunit.client.1.smithi097.stdout:3/268: chown d7/de/d29/c48 99 1 2024-09-16T08:57:08.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/329: dread d4/d9/dc/f49 [0,4194304] 0 2024-09-16T08:57:08.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/296: rename da/df/l35 to da/df/d1f/d3b/l69 0 2024-09-16T08:57:08.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/297: creat da/df/d1f/d3b/f6a x:0 0 0 2024-09-16T08:57:08.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/298: fdatasync da/df/f3f 0 2024-09-16T08:57:08.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/299: dread - da/df/d1f/d3b/f6a zero size 2024-09-16T08:57:08.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/300: creat da/d1a/d26/f6b x:0 0 0 2024-09-16T08:57:08.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/301: chown da/f3e 31285749 1 2024-09-16T08:57:08.024 INFO:tasks.workunit.client.1.smithi097.stdout:3/269: mkdir d7/de/d29/d35/d3d/d57 0 2024-09-16T08:57:08.024 INFO:tasks.workunit.client.1.smithi097.stdout:3/270: read d7/de/d3c/d3f/f50 [4913400,98466] 0 2024-09-16T08:57:08.025 INFO:tasks.workunit.client.1.smithi097.stdout:7/330: link d4/d9/d2b/f4d d4/d7/d19/f71 0 2024-09-16T08:57:08.028 INFO:tasks.workunit.client.1.smithi097.stdout:9/307: dread d2/d5/f29 [0,4194304] 0 2024-09-16T08:57:08.029 INFO:tasks.workunit.client.1.smithi097.stdout:3/271: dread d7/f2b [0,4194304] 0 2024-09-16T08:57:08.029 INFO:tasks.workunit.client.1.smithi097.stdout:3/272: fdatasync d7/de/f1e 0 2024-09-16T08:57:08.029 INFO:tasks.workunit.client.1.smithi097.stdout:3/273: chown d7/d28/d30 476301 1 2024-09-16T08:57:08.029 INFO:tasks.workunit.client.1.smithi097.stdout:3/274: fdatasync d7/fa 0 2024-09-16T08:57:08.029 INFO:tasks.workunit.client.1.smithi097.stdout:0/302: symlink da/df/d1f/l6c 0 2024-09-16T08:57:08.031 INFO:tasks.workunit.client.1.smithi097.stdout:7/331: rename d4/d9/d1c/f60 to d4/d9/dc/f72 0 2024-09-16T08:57:08.031 INFO:tasks.workunit.client.1.smithi097.stdout:7/332: chown d4/d9/d1c/f38 19 1 2024-09-16T08:57:08.032 INFO:tasks.workunit.client.1.smithi097.stdout:7/333: stat d4/d9/dc/f16 0 2024-09-16T08:57:08.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/308: read d2/d5/f11 [3455163,72673] 0 2024-09-16T08:57:08.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/309: write d2/f51 [705876,43962] 0 2024-09-16T08:57:08.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/310: dread - d2/d5/d3a/d3c/f53 zero size 2024-09-16T08:57:08.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/311: write d2/f50 [288325,127368] 0 2024-09-16T08:57:08.033 INFO:tasks.workunit.client.1.smithi097.stdout:3/275: mknod d7/de/d29/d35/d3d/c58 0 2024-09-16T08:57:08.033 INFO:tasks.workunit.client.1.smithi097.stdout:3/276: dread - d7/d28/d30/f53 zero size 2024-09-16T08:57:08.034 INFO:tasks.workunit.client.1.smithi097.stdout:0/303: mkdir da/d24/d4c/d6d 0 2024-09-16T08:57:08.035 INFO:tasks.workunit.client.1.smithi097.stdout:7/334: symlink d4/d9/l73 0 2024-09-16T08:57:08.038 INFO:tasks.workunit.client.1.smithi097.stdout:5/320: sync 2024-09-16T08:57:08.038 INFO:tasks.workunit.client.1.smithi097.stdout:1/344: sync 2024-09-16T08:57:08.038 INFO:tasks.workunit.client.1.smithi097.stdout:3/277: creat d7/de/d29/d35/d3d/f59 x:0 0 0 2024-09-16T08:57:08.039 INFO:tasks.workunit.client.1.smithi097.stdout:1/345: dread - d5/d2c/f4b zero size 2024-09-16T08:57:08.039 INFO:tasks.workunit.client.1.smithi097.stdout:3/278: chown d7/d43/f37 814 1 2024-09-16T08:57:08.039 INFO:tasks.workunit.client.1.smithi097.stdout:1/346: chown d5 0 1 2024-09-16T08:57:08.039 INFO:tasks.workunit.client.1.smithi097.stdout:3/279: creat d7/d43/f5a x:0 0 0 2024-09-16T08:57:08.039 INFO:tasks.workunit.client.1.smithi097.stdout:3/280: creat d7/de/d3c/d3f/f5b x:0 0 0 2024-09-16T08:57:08.039 INFO:tasks.workunit.client.1.smithi097.stdout:3/281: write d7/de/d29/d35/f27 [8591561,31247] 0 2024-09-16T08:57:08.039 INFO:tasks.workunit.client.1.smithi097.stdout:3/282: truncate d7/de/d29/f49 843888 0 2024-09-16T08:57:08.042 INFO:tasks.workunit.client.1.smithi097.stdout:0/304: unlink da/d1a/d41/c57 0 2024-09-16T08:57:08.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/335: symlink d4/l74 0 2024-09-16T08:57:08.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/336: chown d4/d1b/f57 897 1 2024-09-16T08:57:08.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/337: truncate d4/d9/d1c/f34 1363091 0 2024-09-16T08:57:08.044 INFO:tasks.workunit.client.1.smithi097.stdout:6/324: sync 2024-09-16T08:57:08.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/338: truncate d4/d7/d30/f5b 520419 0 2024-09-16T08:57:08.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/339: write d4/d7/d30/f5c [625831,123372] 0 2024-09-16T08:57:08.045 INFO:tasks.workunit.client.1.smithi097.stdout:5/321: rename db/d1a/d50/f59 to db/d18/d1c/d42/d46/f66 0 2024-09-16T08:57:08.045 INFO:tasks.workunit.client.1.smithi097.stdout:5/322: chown db/d18/d20/d2b 4 1 2024-09-16T08:57:08.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/347: mknod d5/c69 0 2024-09-16T08:57:08.053 INFO:tasks.workunit.client.1.smithi097.stdout:6/325: symlink d1/d3/d3e/d5a/l66 0 2024-09-16T08:57:08.055 INFO:tasks.workunit.client.1.smithi097.stdout:7/340: creat d4/d1b/d41/f75 x:0 0 0 2024-09-16T08:57:08.057 INFO:tasks.workunit.client.1.smithi097.stdout:5/323: creat db/d18/d1c/d42/d48/f67 x:0 0 0 2024-09-16T08:57:08.059 INFO:tasks.workunit.client.1.smithi097.stdout:0/305: dwrite da/d24/d4b/f52 [0,4194304] 0 2024-09-16T08:57:08.059 INFO:tasks.workunit.client.1.smithi097.stdout:0/306: write da/df/d1f/f3c [423942,190] 0 2024-09-16T08:57:08.060 INFO:tasks.workunit.client.1.smithi097.stdout:0/307: dread - da/df/d1f/d42/d5b/d60/f65 zero size 2024-09-16T08:57:08.060 INFO:tasks.workunit.client.1.smithi097.stdout:1/348: rename d5/db/dc/d19/f46 to d5/d2c/f6a 0 2024-09-16T08:57:08.060 INFO:tasks.workunit.client.1.smithi097.stdout:1/349: readlink d5/db/dc/d19/l1f 0 2024-09-16T08:57:08.062 INFO:tasks.workunit.client.1.smithi097.stdout:3/283: truncate d7/f21 2576041 0 2024-09-16T08:57:08.062 INFO:tasks.workunit.client.1.smithi097.stdout:3/284: creat d7/d43/f5c x:0 0 0 2024-09-16T08:57:08.062 INFO:tasks.workunit.client.1.smithi097.stdout:3/285: readlink d7/de/l3b 0 2024-09-16T08:57:08.063 INFO:tasks.workunit.client.1.smithi097.stdout:6/326: mknod d1/dc/c67 0 2024-09-16T08:57:08.066 INFO:tasks.workunit.client.1.smithi097.stdout:5/324: dread db/d1a/f2c [0,4194304] 0 2024-09-16T08:57:08.073 INFO:tasks.workunit.client.1.smithi097.stdout:0/308: creat da/d24/d4c/d6d/f6e x:0 0 0 2024-09-16T08:57:08.076 INFO:tasks.workunit.client.1.smithi097.stdout:1/350: getdents d5/d14 0 2024-09-16T08:57:08.080 INFO:tasks.workunit.client.1.smithi097.stdout:3/286: getdents d7/de 0 2024-09-16T08:57:08.082 INFO:tasks.workunit.client.1.smithi097.stdout:5/325: creat db/d18/f68 x:0 0 0 2024-09-16T08:57:08.086 INFO:tasks.workunit.client.1.smithi097.stdout:3/287: rename d7/d15/l34 to d7/de/d3c/d3f/l5d 0 2024-09-16T08:57:08.088 INFO:tasks.workunit.client.1.smithi097.stdout:3/288: dread d7/de/d3c/f45 [0,4194304] 0 2024-09-16T08:57:08.091 INFO:tasks.workunit.client.1.smithi097.stdout:3/289: unlink d7/de/l4e 0 2024-09-16T08:57:08.091 INFO:tasks.workunit.client.1.smithi097.stdout:3/290: creat d7/de/d3c/d3f/f5e x:0 0 0 2024-09-16T08:57:08.095 INFO:tasks.workunit.client.1.smithi097.stdout:3/291: rename d7/f44 to d7/de/f5f 0 2024-09-16T08:57:08.097 INFO:tasks.workunit.client.1.smithi097.stdout:3/292: dread d7/de/d29/f49 [0,4194304] 0 2024-09-16T08:57:08.098 INFO:tasks.workunit.client.1.smithi097.stdout:3/293: read d7/de/d3c/f45 [150835,15142] 0 2024-09-16T08:57:08.100 INFO:tasks.workunit.client.1.smithi097.stdout:5/326: dwrite db/d18/f68 [0,4194304] 0 2024-09-16T08:57:08.105 INFO:tasks.workunit.client.1.smithi097.stdout:5/327: write db/d18/d1c/d42/f4c [717082,76676] 0 2024-09-16T08:57:08.114 INFO:tasks.workunit.client.1.smithi097.stdout:5/328: write db/fe [2999038,4098] 0 2024-09-16T08:57:08.117 INFO:tasks.workunit.client.1.smithi097.stdout:5/329: write db/d1a/f2c [243069,29858] 0 2024-09-16T08:57:08.118 INFO:tasks.workunit.client.1.smithi097.stdout:3/294: dwrite d7/de/d3c/d3f/f51 [0,4194304] 0 2024-09-16T08:57:08.124 INFO:tasks.workunit.client.1.smithi097.stdout:5/330: rmdir db/d22 39 2024-09-16T08:57:08.126 INFO:tasks.workunit.client.1.smithi097.stdout:3/295: getdents d7/d28 0 2024-09-16T08:57:08.126 INFO:tasks.workunit.client.1.smithi097.stdout:0/309: getdents da/df/d1f 0 2024-09-16T08:57:08.127 INFO:tasks.workunit.client.1.smithi097.stdout:5/331: dread db/d18/f3d [0,4194304] 0 2024-09-16T08:57:08.129 INFO:tasks.workunit.client.1.smithi097.stdout:0/310: rename da/d1a/d26/c53 to da/df/d13/c6f 0 2024-09-16T08:57:08.130 INFO:tasks.workunit.client.1.smithi097.stdout:5/332: mknod db/d1a/c69 0 2024-09-16T08:57:08.134 INFO:tasks.workunit.client.1.smithi097.stdout:3/296: dread d7/de/d3c/d3f/f51 [0,4194304] 0 2024-09-16T08:57:08.154 INFO:tasks.workunit.client.1.smithi097.stdout:5/333: dwrite db/f40 [0,4194304] 0 2024-09-16T08:57:08.155 INFO:tasks.workunit.client.1.smithi097.stdout:3/297: dwrite d7/d43/f37 [0,4194304] 0 2024-09-16T08:57:08.157 INFO:tasks.workunit.client.1.smithi097.stdout:5/334: creat db/d22/f6a x:0 0 0 2024-09-16T08:57:08.157 INFO:tasks.workunit.client.1.smithi097.stdout:5/335: readlink db/l11 0 2024-09-16T08:57:08.183 INFO:tasks.workunit.client.1.smithi097.stdout:5/336: dwrite db/d1a/f38 [0,4194304] 0 2024-09-16T08:57:08.184 INFO:tasks.workunit.client.1.smithi097.stdout:5/337: creat db/d22/f6b x:0 0 0 2024-09-16T08:57:08.184 INFO:tasks.workunit.client.1.smithi097.stdout:5/338: write db/d22/f4b [1045023,16673] 0 2024-09-16T08:57:08.184 INFO:tasks.workunit.client.1.smithi097.stdout:3/298: dwrite d7/f21 [0,4194304] 0 2024-09-16T08:57:08.184 INFO:tasks.workunit.client.1.smithi097.stdout:3/299: truncate d7/d15/f17 4340249 0 2024-09-16T08:57:08.197 INFO:tasks.workunit.client.1.smithi097.stdout:5/339: dwrite db/d1a/f38 [0,4194304] 0 2024-09-16T08:57:08.199 INFO:tasks.workunit.client.1.smithi097.stdout:3/300: mkdir d7/de/d3c/d3f/d60 0 2024-09-16T08:57:08.199 INFO:tasks.workunit.client.1.smithi097.stdout:3/301: chown d7/de/d29/c48 48488 1 2024-09-16T08:57:08.202 INFO:tasks.workunit.client.1.smithi097.stdout:5/340: creat db/d18/d1c/d42/d46/f6c x:0 0 0 2024-09-16T08:57:08.202 INFO:tasks.workunit.client.1.smithi097.stdout:5/341: fsync db/d1a/f5b 0 2024-09-16T08:57:08.208 INFO:tasks.workunit.client.1.smithi097.stdout:5/342: rename db/d1a/d50 to db/d1a/d6d 0 2024-09-16T08:57:08.213 INFO:tasks.workunit.client.1.smithi097.stdout:5/343: mkdir db/d6e 0 2024-09-16T08:57:08.213 INFO:tasks.workunit.client.1.smithi097.stdout:5/344: chown db/d18/d1c/d42/d46/f6c 6 1 2024-09-16T08:57:08.213 INFO:tasks.workunit.client.1.smithi097.stdout:5/345: dread - db/d22/f6a zero size 2024-09-16T08:57:08.213 INFO:tasks.workunit.client.1.smithi097.stdout:5/346: creat db/d18/d20/d2b/f6f x:0 0 0 2024-09-16T08:57:08.215 INFO:tasks.workunit.client.1.smithi097.stdout:5/347: mkdir db/d18/d1c/d42/d46/d70 0 2024-09-16T08:57:08.223 INFO:tasks.workunit.client.1.smithi097.stdout:5/348: mknod db/d1a/d6d/c71 0 2024-09-16T08:57:08.229 INFO:tasks.workunit.client.1.smithi097.stdout:0/311: rename da/df/d13/c6f to da/df/d3d/c70 0 2024-09-16T08:57:08.230 INFO:tasks.workunit.client.1.smithi097.stdout:0/312: creat da/d1a/d41/f71 x:0 0 0 2024-09-16T08:57:08.230 INFO:tasks.workunit.client.1.smithi097.stdout:0/313: write da/df/d1f/d42/d5b/d60/d68/d5e/f5f [159248,28634] 0 2024-09-16T08:57:08.232 INFO:tasks.workunit.client.1.smithi097.stdout:3/302: rename d7/c16 to d7/d15/c61 0 2024-09-16T08:57:08.240 INFO:tasks.workunit.client.1.smithi097.stdout:0/314: dread f7 [0,4194304] 0 2024-09-16T08:57:08.240 INFO:tasks.workunit.client.1.smithi097.stdout:0/315: chown da/d1a/f63 1182548 1 2024-09-16T08:57:08.244 INFO:tasks.workunit.client.1.smithi097.stdout:0/316: creat da/df/d1f/d42/d5b/d60/d68/d5e/f72 x:0 0 0 2024-09-16T08:57:08.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:07 smithi031 ceph-mon[27844]: pgmap v5: 65 pgs: 65 active+clean; 355 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:57:08.247 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:07 smithi031 ceph-mon[27844]: mgrmap e24: smithi097.mpashf(active, since 4s) 2024-09-16T08:57:08.247 INFO:tasks.workunit.client.1.smithi097.stdout:0/317: mkdir da/df/d1f/d42/d5b/d60/d73 0 2024-09-16T08:57:08.249 INFO:tasks.workunit.client.1.smithi097.stdout:0/318: write da/d1a/f49 [593595,11366] 0 2024-09-16T08:57:08.249 INFO:tasks.workunit.client.1.smithi097.stdout:0/319: dread - da/df/d1f/d3b/f6a zero size 2024-09-16T08:57:08.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:07 smithi097 ceph-mon[35478]: pgmap v5: 65 pgs: 65 active+clean; 355 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:57:08.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:07 smithi097 ceph-mon[35478]: mgrmap e24: smithi097.mpashf(active, since 4s) 2024-09-16T08:57:08.274 INFO:tasks.workunit.client.1.smithi097.stdout:3/303: dwrite d7/d15/f17 [0,4194304] 0 2024-09-16T08:57:08.274 INFO:tasks.workunit.client.1.smithi097.stdout:3/304: chown d7/de/d3c/d3f/f50 2626940 1 2024-09-16T08:57:08.274 INFO:tasks.workunit.client.1.smithi097.stdout:0/320: dwrite da/d1a/f63 [0,4194304] 0 2024-09-16T08:57:08.276 INFO:tasks.workunit.client.1.smithi097.stdout:3/305: rename d7/de/l1a to d7/d28/l62 0 2024-09-16T08:57:08.282 INFO:tasks.workunit.client.1.smithi097.stdout:0/321: rename da/df/d1f/d42/d5b/d60/d68/d5e/f5f to da/df/d1f/d42/d5b/d60/d68/f74 0 2024-09-16T08:57:08.282 INFO:tasks.workunit.client.1.smithi097.stdout:0/322: chown da/d24 114118 1 2024-09-16T08:57:08.285 INFO:tasks.workunit.client.1.smithi097.stdout:0/323: mkdir da/d1a/d26/d75 0 2024-09-16T08:57:08.285 INFO:tasks.workunit.client.1.smithi097.stdout:0/324: dread - da/df/d1f/d3b/f5a zero size 2024-09-16T08:57:08.287 INFO:tasks.workunit.client.1.smithi097.stdout:0/325: mkdir da/d24/d76 0 2024-09-16T08:57:08.289 INFO:tasks.workunit.client.1.smithi097.stdout:0/326: truncate f6 8347668 0 2024-09-16T08:57:08.293 INFO:tasks.workunit.client.1.smithi097.stdout:0/327: link da/d1a/f63 da/df/d1f/d42/d5b/d60/d73/f77 0 2024-09-16T08:57:08.293 INFO:tasks.workunit.client.1.smithi097.stdout:0/328: truncate da/df/d1f/d42/d5b/d60/d68/f74 956044 0 2024-09-16T08:57:08.295 INFO:tasks.workunit.client.1.smithi097.stdout:0/329: mknod da/d24/d4c/c78 0 2024-09-16T08:57:08.295 INFO:tasks.workunit.client.1.smithi097.stdout:0/330: fdatasync da/df/d1f/d3b/f5a 0 2024-09-16T08:57:08.313 INFO:tasks.workunit.client.1.smithi097.stdout:0/331: dwrite da/d1a/f49 [0,4194304] 0 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:7/341: sync 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:9/312: sync 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:1/351: sync 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:4/338: sync 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:8/299: sync 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:6/327: sync 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:9/313: fdatasync d2/d3f/f43 0 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:6/328: creat d1/dc/d1e/f68 x:0 0 0 2024-09-16T08:57:08.322 INFO:tasks.workunit.client.1.smithi097.stdout:2/331: sync 2024-09-16T08:57:08.323 INFO:tasks.workunit.client.1.smithi097.stdout:9/314: write d2/d5/f4e [412265,109346] 0 2024-09-16T08:57:08.323 INFO:tasks.workunit.client.1.smithi097.stdout:6/329: readlink d1/l54 0 2024-09-16T08:57:08.323 INFO:tasks.workunit.client.1.smithi097.stdout:7/342: creat d4/d7/d30/f76 x:0 0 0 2024-09-16T08:57:08.323 INFO:tasks.workunit.client.1.smithi097.stdout:9/315: readlink d2/l23 0 2024-09-16T08:57:08.323 INFO:tasks.workunit.client.1.smithi097.stdout:7/343: chown d4/l6d 6207 1 2024-09-16T08:57:08.325 INFO:tasks.workunit.client.1.smithi097.stdout:8/300: getdents d8/dd/d15/d16 0 2024-09-16T08:57:08.332 INFO:tasks.workunit.client.1.smithi097.stdout:4/339: mknod d0/d7/df/d15/c7e 0 2024-09-16T08:57:08.332 INFO:tasks.workunit.client.1.smithi097.stdout:4/340: chown d0/d7/df/d15/c7e 654159 1 2024-09-16T08:57:08.332 INFO:tasks.workunit.client.1.smithi097.stdout:4/341: chown d0/d23/f7b 4127 1 2024-09-16T08:57:08.332 INFO:tasks.workunit.client.1.smithi097.stdout:4/342: chown d0/c74 228680 1 2024-09-16T08:57:08.337 INFO:tasks.workunit.client.1.smithi097.stdout:8/301: write d8/dd/d15/d16/f3a [4071825,57072] 0 2024-09-16T08:57:08.337 INFO:tasks.workunit.client.1.smithi097.stdout:8/302: truncate d8/dd/d15/d2c/f4b 1020003 0 2024-09-16T08:57:08.341 INFO:tasks.workunit.client.1.smithi097.stdout:1/352: truncate d5/d17/f26 1838394 0 2024-09-16T08:57:08.344 INFO:tasks.workunit.client.1.smithi097.stdout:4/343: rename d0/d7/df/c73 to d0/d7/c7f 0 2024-09-16T08:57:08.345 INFO:tasks.workunit.client.1.smithi097.stdout:0/332: dwrite da/d1a/f63 [0,4194304] 0 2024-09-16T08:57:08.346 INFO:tasks.workunit.client.1.smithi097.stdout:4/344: write d0/d7/d52/f50 [1343214,27653] 0 2024-09-16T08:57:08.346 INFO:tasks.workunit.client.1.smithi097.stdout:4/345: write d0/f2a [1028983,22238] 0 2024-09-16T08:57:08.350 INFO:tasks.workunit.client.1.smithi097.stdout:6/330: dwrite d1/f12 [0,4194304] 0 2024-09-16T08:57:08.350 INFO:tasks.workunit.client.1.smithi097.stdout:9/316: dwrite d2/d30/f5a [4194304,4194304] 0 2024-09-16T08:57:08.352 INFO:tasks.workunit.client.1.smithi097.stdout:1/353: mkdir d5/d17/d54/d6b 0 2024-09-16T08:57:08.352 INFO:tasks.workunit.client.1.smithi097.stdout:2/332: dwrite d3/d4/d8/f39 [0,4194304] 0 2024-09-16T08:57:08.356 INFO:tasks.workunit.client.1.smithi097.stdout:0/333: link da/d1a/d26/c34 da/d24/d4c/d6d/c79 0 2024-09-16T08:57:08.359 INFO:tasks.workunit.client.1.smithi097.stdout:7/344: rmdir d4/d12/d1f/d22/d5a 0 2024-09-16T08:57:08.365 INFO:tasks.workunit.client.1.smithi097.stdout:6/331: write d1/d3/d3e/f48 [1917531,3301] 0 2024-09-16T08:57:08.366 INFO:tasks.workunit.client.1.smithi097.stdout:0/334: symlink da/df/d1f/d42/d5b/l7a 0 2024-09-16T08:57:08.367 INFO:tasks.workunit.client.1.smithi097.stdout:7/345: getdents d4/d12 0 2024-09-16T08:57:08.369 INFO:tasks.workunit.client.1.smithi097.stdout:9/317: truncate f0 5457375 0 2024-09-16T08:57:08.369 INFO:tasks.workunit.client.1.smithi097.stdout:9/318: fdatasync f0 0 2024-09-16T08:57:08.370 INFO:tasks.workunit.client.1.smithi097.stdout:1/354: getdents d5/d2c 0 2024-09-16T08:57:08.371 INFO:tasks.workunit.client.1.smithi097.stdout:0/335: read da/df/d1f/f2a [8242,88919] 0 2024-09-16T08:57:08.371 INFO:tasks.workunit.client.1.smithi097.stdout:6/332: dread d1/dc/f38 [0,4194304] 0 2024-09-16T08:57:08.371 INFO:tasks.workunit.client.1.smithi097.stdout:6/333: chown d1/f5b 26 1 2024-09-16T08:57:08.372 INFO:tasks.workunit.client.1.smithi097.stdout:9/319: mknod d2/d5/d3a/d3c/c61 0 2024-09-16T08:57:08.375 INFO:tasks.workunit.client.1.smithi097.stdout:8/303: dwrite d8/dd/d15/d16/f3c [0,4194304] 0 2024-09-16T08:57:08.376 INFO:tasks.workunit.client.1.smithi097.stdout:0/336: creat da/df/d1f/d42/d5b/f7b x:0 0 0 2024-09-16T08:57:08.378 INFO:tasks.workunit.client.1.smithi097.stdout:7/346: mkdir d4/d7/d77 0 2024-09-16T08:57:08.378 INFO:tasks.workunit.client.1.smithi097.stdout:7/347: creat d4/d1b/f78 x:0 0 0 2024-09-16T08:57:08.381 INFO:tasks.workunit.client.1.smithi097.stdout:9/320: mkdir d2/d3f/d62 0 2024-09-16T08:57:08.381 INFO:tasks.workunit.client.1.smithi097.stdout:4/346: dwrite d0/f2a [0,4194304] 0 2024-09-16T08:57:08.381 INFO:tasks.workunit.client.1.smithi097.stdout:2/333: dwrite d3/d4/dd/f35 [4194304,4194304] 0 2024-09-16T08:57:08.381 INFO:tasks.workunit.client.1.smithi097.stdout:2/334: chown d3/d4/c4e 269594 1 2024-09-16T08:57:08.384 INFO:tasks.workunit.client.1.smithi097.stdout:1/355: rename d5/db/f23 to d5/d17/f6c 0 2024-09-16T08:57:08.384 INFO:tasks.workunit.client.1.smithi097.stdout:1/356: write d5/d2c/f4e [1313969,3823] 0 2024-09-16T08:57:08.387 INFO:tasks.workunit.client.1.smithi097.stdout:0/337: dread da/d1a/f63 [0,4194304] 0 2024-09-16T08:57:08.411 INFO:tasks.workunit.client.1.smithi097.stdout:8/304: symlink d8/dd/d15/d2c/l60 0 2024-09-16T08:57:08.411 INFO:tasks.workunit.client.1.smithi097.stdout:8/305: fdatasync d8/dd/d42/f51 0 2024-09-16T08:57:08.411 INFO:tasks.workunit.client.1.smithi097.stdout:8/306: chown d8/dd/l50 2566361 1 2024-09-16T08:57:08.413 INFO:tasks.workunit.client.1.smithi097.stdout:8/307: dread d8/dd/d15/d2c/f41 [0,4194304] 0 2024-09-16T08:57:08.417 INFO:tasks.workunit.client.1.smithi097.stdout:2/335: dwrite d3/d4/d9/f4f [0,4194304] 0 2024-09-16T08:57:08.417 INFO:tasks.workunit.client.1.smithi097.stdout:9/321: dwrite d2/d5/f4e [0,4194304] 0 2024-09-16T08:57:08.417 INFO:tasks.workunit.client.1.smithi097.stdout:9/322: chown d2/d30/f47 94 1 2024-09-16T08:57:08.419 INFO:tasks.workunit.client.1.smithi097.stdout:4/347: unlink d0/d7/d52/d26/d3d/f62 0 2024-09-16T08:57:08.419 INFO:tasks.workunit.client.1.smithi097.stdout:4/348: write d0/f30 [607490,9282] 0 2024-09-16T08:57:08.421 INFO:tasks.workunit.client.1.smithi097.stdout:2/336: dread d3/d4/d8/f39 [0,4194304] 0 2024-09-16T08:57:08.421 INFO:tasks.workunit.client.1.smithi097.stdout:2/337: stat d3/d4/d8/f33 0 2024-09-16T08:57:08.424 INFO:tasks.workunit.client.1.smithi097.stdout:6/334: rename d1/l54 to d1/d3/d3e/d5a/l69 0 2024-09-16T08:57:08.424 INFO:tasks.workunit.client.1.smithi097.stdout:6/335: truncate d1/d3/d16/f4d 706979 0 2024-09-16T08:57:08.428 INFO:tasks.workunit.client.1.smithi097.stdout:1/357: symlink d5/db/dc/l6d 0 2024-09-16T08:57:08.431 INFO:tasks.workunit.client.1.smithi097.stdout:0/338: mknod da/df/d1f/d42/d5b/d60/d68/d5e/c7c 0 2024-09-16T08:57:08.431 INFO:tasks.workunit.client.1.smithi097.stdout:0/339: readlink da/df/d3d/l56 0 2024-09-16T08:57:08.433 INFO:tasks.workunit.client.1.smithi097.stdout:7/348: mkdir d4/d7/d79 0 2024-09-16T08:57:08.436 INFO:tasks.workunit.client.1.smithi097.stdout:4/349: rename d0/d36/f56 to d0/d36/d4c/f80 0 2024-09-16T08:57:08.437 INFO:tasks.workunit.client.1.smithi097.stdout:0/340: dread da/d1a/f22 [0,4194304] 0 2024-09-16T08:57:08.437 INFO:tasks.workunit.client.1.smithi097.stdout:0/341: write f7 [4364126,43128] 0 2024-09-16T08:57:08.437 INFO:tasks.workunit.client.1.smithi097.stdout:0/342: dread - da/df/d1f/d42/d5b/d60/d68/f5d zero size 2024-09-16T08:57:08.438 INFO:tasks.workunit.client.1.smithi097.stdout:6/336: mkdir d1/d3/d16/d6a 0 2024-09-16T08:57:08.439 INFO:tasks.workunit.client.1.smithi097.stdout:6/337: write d1/d3/d16/f2e [176458,103292] 0 2024-09-16T08:57:08.441 INFO:tasks.workunit.client.1.smithi097.stdout:4/350: unlink d0/d23/c2d 0 2024-09-16T08:57:08.442 INFO:tasks.workunit.client.1.smithi097.stdout:4/351: readlink d0/d7/d52/d26/d3d/l53 0 2024-09-16T08:57:08.442 INFO:tasks.workunit.client.1.smithi097.stdout:4/352: write d0/d7/d52/f67 [632636,129266] 0 2024-09-16T08:57:08.442 INFO:tasks.workunit.client.1.smithi097.stdout:7/349: write d4/d1b/f1e [505181,47451] 0 2024-09-16T08:57:08.442 INFO:tasks.workunit.client.1.smithi097.stdout:7/350: chown d4/d9/d1c/f34 1532737672 1 2024-09-16T08:57:08.442 INFO:tasks.workunit.client.1.smithi097.stdout:7/351: truncate d4/d1b/f78 185464 0 2024-09-16T08:57:08.443 INFO:tasks.workunit.client.1.smithi097.stdout:7/352: dread - d4/d7/f50 zero size 2024-09-16T08:57:08.444 INFO:tasks.workunit.client.1.smithi097.stdout:0/343: mknod da/df/d13/c7d 0 2024-09-16T08:57:08.449 INFO:tasks.workunit.client.1.smithi097.stdout:4/353: link d0/d7/d52/d26/f6b d0/d7/df/d15/d1f/f81 0 2024-09-16T08:57:08.453 INFO:tasks.workunit.client.1.smithi097.stdout:2/338: dwrite d3/d4/d9/f1e [0,4194304] 0 2024-09-16T08:57:08.453 INFO:tasks.workunit.client.1.smithi097.stdout:8/308: dwrite d8/dd/d42/f51 [0,4194304] 0 2024-09-16T08:57:08.453 INFO:tasks.workunit.client.1.smithi097.stdout:8/309: chown d8/dd/d15/d2c/d4a/d4d/d3d/f57 407005 1 2024-09-16T08:57:08.453 INFO:tasks.workunit.client.1.smithi097.stdout:8/310: write d8/dd/f1f [3438097,29311] 0 2024-09-16T08:57:08.453 INFO:tasks.workunit.client.1.smithi097.stdout:9/323: dwrite d2/d5/fa [0,4194304] 0 2024-09-16T08:57:08.454 INFO:tasks.workunit.client.1.smithi097.stdout:1/358: dwrite d5/d2c/f4b [0,4194304] 0 2024-09-16T08:57:08.454 INFO:tasks.workunit.client.1.smithi097.stdout:7/353: rename d4/l6d to d4/d9/l7a 0 2024-09-16T08:57:08.460 INFO:tasks.workunit.client.1.smithi097.stdout:4/354: dread d0/d7/fb [0,4194304] 0 2024-09-16T08:57:08.463 INFO:tasks.workunit.client.1.smithi097.stdout:2/339: write d3/d4/d8/d34/f3c [4062253,117565] 0 2024-09-16T08:57:08.478 INFO:tasks.workunit.client.1.smithi097.stdout:9/324: mkdir d2/d30/d63 0 2024-09-16T08:57:08.480 INFO:tasks.workunit.client.1.smithi097.stdout:9/325: write d2/f1a [969166,100532] 0 2024-09-16T08:57:08.490 INFO:tasks.workunit.client.1.smithi097.stdout:9/326: symlink d2/l64 0 2024-09-16T08:57:08.490 INFO:tasks.workunit.client.1.smithi097.stdout:9/327: dread - d2/f31 zero size 2024-09-16T08:57:08.491 INFO:tasks.workunit.client.1.smithi097.stdout:9/328: dread d2/d30/f47 [0,4194304] 0 2024-09-16T08:57:08.493 INFO:tasks.workunit.client.1.smithi097.stdout:5/349: sync 2024-09-16T08:57:08.493 INFO:tasks.workunit.client.1.smithi097.stdout:5/350: dread - db/f56 zero size 2024-09-16T08:57:08.493 INFO:tasks.workunit.client.1.smithi097.stdout:5/351: write db/d18/f34 [4567468,118194] 0 2024-09-16T08:57:08.494 INFO:tasks.workunit.client.1.smithi097.stdout:9/329: symlink d2/l65 0 2024-09-16T08:57:08.495 INFO:tasks.workunit.client.1.smithi097.stdout:5/352: mknod db/d18/d1c/c72 0 2024-09-16T08:57:08.495 INFO:tasks.workunit.client.1.smithi097.stdout:5/353: chown fa 16 1 2024-09-16T08:57:08.497 INFO:tasks.workunit.client.1.smithi097.stdout:9/330: link d2/l23 d2/d3f/d62/l66 0 2024-09-16T08:57:08.498 INFO:tasks.workunit.client.1.smithi097.stdout:9/331: symlink d2/d3f/l67 0 2024-09-16T08:57:08.506 INFO:tasks.workunit.client.1.smithi097.stdout:8/311: dwrite d8/f24 [4194304,4194304] 0 2024-09-16T08:57:08.511 INFO:tasks.workunit.client.1.smithi097.stdout:6/338: rmdir d1 39 2024-09-16T08:57:08.511 INFO:tasks.workunit.client.1.smithi097.stdout:6/339: fdatasync d1/f4e 0 2024-09-16T08:57:08.520 INFO:tasks.workunit.client.1.smithi097.stdout:5/354: getdents db/d18/d1c 0 2024-09-16T08:57:08.520 INFO:tasks.workunit.client.1.smithi097.stdout:5/355: read - db/d22/f6b zero size 2024-09-16T08:57:08.522 INFO:tasks.workunit.client.1.smithi097.stdout:5/356: getdents db/d18/d1c/d42/d46 0 2024-09-16T08:57:08.522 INFO:tasks.workunit.client.1.smithi097.stdout:5/357: chown db/d1a/f38 600950 1 2024-09-16T08:57:08.524 INFO:tasks.workunit.client.1.smithi097.stdout:5/358: rename db/d18/d1c/c3e to db/d22/c73 0 2024-09-16T08:57:08.525 INFO:tasks.workunit.client.1.smithi097.stdout:5/359: mkdir db/d18/d20/d74 0 2024-09-16T08:57:08.525 INFO:tasks.workunit.client.1.smithi097.stdout:5/360: creat db/d18/d20/d2b/f75 x:0 0 0 2024-09-16T08:57:08.527 INFO:tasks.workunit.client.1.smithi097.stdout:6/340: dwrite d1/d3/d3e/d5a/f4a [0,4194304] 0 2024-09-16T08:57:08.528 INFO:tasks.workunit.client.1.smithi097.stdout:5/361: dread db/d18/d1c/d42/d48/d64/f63 [0,4194304] 0 2024-09-16T08:57:08.528 INFO:tasks.workunit.client.1.smithi097.stdout:5/362: write db/d1a/f5b [542883,100882] 0 2024-09-16T08:57:08.529 INFO:tasks.workunit.client.1.smithi097.stdout:5/363: fdatasync db/f2e 0 2024-09-16T08:57:08.529 INFO:tasks.workunit.client.1.smithi097.stdout:6/341: truncate d1/d3/d3e/d5a/f47 3048000 0 2024-09-16T08:57:08.529 INFO:tasks.workunit.client.1.smithi097.stdout:8/312: dwrite d8/dd/d15/d2c/d4a/d4e/f5a [0,4194304] 0 2024-09-16T08:57:08.529 INFO:tasks.workunit.client.1.smithi097.stdout:8/313: fdatasync d8/dd/d15/d2c/d4a/d4d/d3d/f5b 0 2024-09-16T08:57:08.529 INFO:tasks.workunit.client.1.smithi097.stdout:5/364: mkdir db/d18/d1c/d42/d48/d76 0 2024-09-16T08:57:08.530 INFO:tasks.workunit.client.1.smithi097.stdout:8/314: rmdir d8 39 2024-09-16T08:57:08.530 INFO:tasks.workunit.client.1.smithi097.stdout:8/315: write d8/f13 [142589,126674] 0 2024-09-16T08:57:08.530 INFO:tasks.workunit.client.1.smithi097.stdout:8/316: getdents d8/dd/d15/d16 0 2024-09-16T08:57:08.530 INFO:tasks.workunit.client.1.smithi097.stdout:8/317: fsync d8/dd/d15/d2c/d4a/d4d/d3d/f5b 0 2024-09-16T08:57:08.530 INFO:tasks.workunit.client.1.smithi097.stdout:8/318: fsync d8/dd/d15/f34 0 2024-09-16T08:57:08.532 INFO:tasks.workunit.client.1.smithi097.stdout:3/306: sync 2024-09-16T08:57:08.551 INFO:tasks.workunit.client.1.smithi097.stdout:5/365: dwrite db/d1a/f2c [0,4194304] 0 2024-09-16T08:57:08.551 INFO:tasks.workunit.client.1.smithi097.stdout:6/342: dwrite d1/d3/f34 [4194304,4194304] 0 2024-09-16T08:57:08.553 INFO:tasks.workunit.client.1.smithi097.stdout:6/343: rename d1/d3/d3e/f48 to d1/d3/d16/d6a/f6b 0 2024-09-16T08:57:08.553 INFO:tasks.workunit.client.1.smithi097.stdout:5/366: symlink db/d18/d20/l77 0 2024-09-16T08:57:08.554 INFO:tasks.workunit.client.1.smithi097.stdout:6/344: dread d1/d3/f2f [4194304,4194304] 0 2024-09-16T08:57:08.555 INFO:tasks.workunit.client.1.smithi097.stdout:5/367: rename db/d18/f3d to db/d18/d1c/d42/d48/f78 0 2024-09-16T08:57:08.576 INFO:tasks.workunit.client.0.smithi031.stdout:3/23: sync 2024-09-16T08:57:08.576 INFO:tasks.workunit.client.0.smithi031.stdout:9/0: sync 2024-09-16T08:57:08.576 INFO:tasks.workunit.client.0.smithi031.stdout:9/1: write - no filename 2024-09-16T08:57:08.576 INFO:tasks.workunit.client.0.smithi031.stdout:5/57: sync 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/6: sync 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/7: truncate - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/8: truncate - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:2/15: sync 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/9: readlink - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/10: truncate - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/11: truncate - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/12: dread - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/13: truncate - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:6/14: fdatasync - no filename 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:4/11: sync 2024-09-16T08:57:08.577 INFO:tasks.workunit.client.0.smithi031.stdout:4/12: write d0/f1 [562998,102558] 0 2024-09-16T08:57:08.578 INFO:tasks.workunit.client.0.smithi031.stdout:0/34: sync 2024-09-16T08:57:08.578 INFO:tasks.workunit.client.0.smithi031.stdout:8/16: sync 2024-09-16T08:57:08.578 INFO:tasks.workunit.client.0.smithi031.stdout:8/17: truncate - no filename 2024-09-16T08:57:08.578 INFO:tasks.workunit.client.0.smithi031.stdout:8/18: dwrite - no filename 2024-09-16T08:57:08.578 INFO:tasks.workunit.client.0.smithi031.stdout:1/54: sync 2024-09-16T08:57:08.578 INFO:tasks.workunit.client.0.smithi031.stdout:1/55: readlink d2/d4/l7 0 2024-09-16T08:57:08.578 INFO:tasks.workunit.client.0.smithi031.stdout:7/81: sync 2024-09-16T08:57:08.579 INFO:tasks.workunit.client.0.smithi031.stdout:9/2: mknod c0 0 2024-09-16T08:57:08.579 INFO:tasks.workunit.client.0.smithi031.stdout:9/3: dread - no filename 2024-09-16T08:57:08.579 INFO:tasks.workunit.client.0.smithi031.stdout:5/58: symlink d2/dc/l14 0 2024-09-16T08:57:08.579 INFO:tasks.workunit.client.0.smithi031.stdout:6/15: creat d0/f1 x:0 0 0 2024-09-16T08:57:08.579 INFO:tasks.workunit.client.0.smithi031.stdout:6/16: fdatasync d0/f1 0 2024-09-16T08:57:08.579 INFO:tasks.workunit.client.0.smithi031.stdout:6/17: dread - d0/f1 zero size 2024-09-16T08:57:08.582 INFO:tasks.workunit.client.0.smithi031.stdout:7/82: link d2/d8/d11/l16 d2/d8/d11/l19 0 2024-09-16T08:57:08.582 INFO:tasks.workunit.client.0.smithi031.stdout:7/83: creat d2/f1a x:0 0 0 2024-09-16T08:57:08.582 INFO:tasks.workunit.client.0.smithi031.stdout:0/35: dread d6/f8 [0,4194304] 0 2024-09-16T08:57:08.583 INFO:tasks.workunit.client.0.smithi031.stdout:3/24: link d3/f8 d3/f9 0 2024-09-16T08:57:08.584 INFO:tasks.workunit.client.0.smithi031.stdout:3/25: stat f1 0 2024-09-16T08:57:08.584 INFO:tasks.workunit.client.1.smithi097.stdout:6/345: dwrite d1/d3/d16/f4d [0,4194304] 0 2024-09-16T08:57:08.584 INFO:tasks.workunit.client.1.smithi097.stdout:5/368: dwrite db/d18/d1c/d42/d48/f51 [0,4194304] 0 2024-09-16T08:57:08.585 INFO:tasks.workunit.client.0.smithi031.stdout:9/4: mknod c1 0 2024-09-16T08:57:08.585 INFO:tasks.workunit.client.0.smithi031.stdout:9/5: write - no filename 2024-09-16T08:57:08.585 INFO:tasks.workunit.client.0.smithi031.stdout:6/18: symlink d0/l2 0 2024-09-16T08:57:08.586 INFO:tasks.workunit.client.0.smithi031.stdout:6/19: dread - d0/f1 zero size 2024-09-16T08:57:08.586 INFO:tasks.workunit.client.0.smithi031.stdout:6/20: chown d0/l2 4427 1 2024-09-16T08:57:08.586 INFO:tasks.workunit.client.0.smithi031.stdout:6/21: creat d0/f3 x:0 0 0 2024-09-16T08:57:08.586 INFO:tasks.workunit.client.0.smithi031.stdout:6/22: readlink d0/l2 0 2024-09-16T08:57:08.587 INFO:tasks.workunit.client.0.smithi031.stdout:4/13: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:08.587 INFO:tasks.workunit.client.0.smithi031.stdout:8/19: mkdir d4 0 2024-09-16T08:57:08.588 INFO:tasks.workunit.client.0.smithi031.stdout:8/20: dwrite - no filename 2024-09-16T08:57:08.588 INFO:tasks.workunit.client.0.smithi031.stdout:8/21: truncate - no filename 2024-09-16T08:57:08.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/84: rename d2/d8/d11/f14 to d2/d8/d11/f1b 0 2024-09-16T08:57:08.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/85: truncate d2/fb 1467364 0 2024-09-16T08:57:08.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/86: stat d2/ff 0 2024-09-16T08:57:08.592 INFO:tasks.workunit.client.0.smithi031.stdout:0/36: write d6/fc [670324,129184] 0 2024-09-16T08:57:08.593 INFO:tasks.workunit.client.1.smithi097.stdout:6/346: mknod d1/d3/d16/c6c 0 2024-09-16T08:57:08.594 INFO:tasks.workunit.client.1.smithi097.stdout:5/369: mkdir db/d6e/d79 0 2024-09-16T08:57:08.595 INFO:tasks.workunit.client.0.smithi031.stdout:9/6: creat f2 x:0 0 0 2024-09-16T08:57:08.598 INFO:tasks.workunit.client.1.smithi097.stdout:5/370: rename db/d18/l65 to db/d18/d1c/d42/d48/d76/l7a 0 2024-09-16T08:57:08.600 INFO:tasks.workunit.client.0.smithi031.stdout:8/22: mknod d4/c5 0 2024-09-16T08:57:08.600 INFO:tasks.workunit.client.0.smithi031.stdout:8/23: dwrite - no filename 2024-09-16T08:57:08.601 INFO:tasks.workunit.client.1.smithi097.stdout:5/371: unlink db/d22/f6b 0 2024-09-16T08:57:08.601 INFO:tasks.workunit.client.1.smithi097.stdout:5/372: write db/d22/f6a [198827,16065] 0 2024-09-16T08:57:08.602 INFO:tasks.workunit.client.0.smithi031.stdout:7/87: stat d2/d8/d11/l19 0 2024-09-16T08:57:08.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/37: fsync d6/f7 0 2024-09-16T08:57:08.602 INFO:tasks.workunit.client.1.smithi097.stdout:5/373: symlink db/d18/d1c/d42/d46/d70/l7b 0 2024-09-16T08:57:08.602 INFO:tasks.workunit.client.1.smithi097.stdout:5/374: creat db/d18/d20/d2b/f7c x:0 0 0 2024-09-16T08:57:08.604 INFO:tasks.workunit.client.0.smithi031.stdout:3/26: rename d3/f9 to d3/fa 0 2024-09-16T08:57:08.605 INFO:tasks.workunit.client.0.smithi031.stdout:9/7: mkdir d3 0 2024-09-16T08:57:08.608 INFO:tasks.workunit.client.0.smithi031.stdout:8/24: unlink c2 0 2024-09-16T08:57:08.610 INFO:tasks.workunit.client.0.smithi031.stdout:7/88: link d2/d8/d11/l16 d2/d8/d11/l1c 0 2024-09-16T08:57:08.610 INFO:tasks.workunit.client.0.smithi031.stdout:7/89: dread - d2/f1a zero size 2024-09-16T08:57:08.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/27: creat d3/fb x:0 0 0 2024-09-16T08:57:08.611 INFO:tasks.workunit.client.1.smithi097.stdout:6/347: dread d1/d3/f11 [0,4194304] 0 2024-09-16T08:57:08.611 INFO:tasks.workunit.client.0.smithi031.stdout:9/8: rename c1 to d3/c4 0 2024-09-16T08:57:08.612 INFO:tasks.workunit.client.0.smithi031.stdout:8/25: creat d4/f6 x:0 0 0 2024-09-16T08:57:08.613 INFO:tasks.workunit.client.1.smithi097.stdout:6/348: rmdir d1/d3/d3e 39 2024-09-16T08:57:08.613 INFO:tasks.workunit.client.1.smithi097.stdout:6/349: fdatasync d1/d3/d3e/d5a/f4a 0 2024-09-16T08:57:08.613 INFO:tasks.workunit.client.0.smithi031.stdout:7/90: fdatasync d2/d8/fd 0 2024-09-16T08:57:08.614 INFO:tasks.workunit.client.0.smithi031.stdout:2/16: dwrite f1 [0,4194304] 0 2024-09-16T08:57:08.614 INFO:tasks.workunit.client.1.smithi097.stdout:6/350: unlink d1/dc/d3c/l59 0 2024-09-16T08:57:08.615 INFO:tasks.workunit.client.1.smithi097.stdout:6/351: fsync d1/dc/d1e/f3a 0 2024-09-16T08:57:08.615 INFO:tasks.workunit.client.0.smithi031.stdout:5/59: dwrite d2/fa [0,4194304] 0 2024-09-16T08:57:08.617 INFO:tasks.workunit.client.1.smithi097.stdout:6/352: creat d1/dc/d3c/f6d x:0 0 0 2024-09-16T08:57:08.617 INFO:tasks.workunit.client.1.smithi097.stdout:6/353: truncate d1/d3/f1c 1326269 0 2024-09-16T08:57:08.617 INFO:tasks.workunit.client.1.smithi097.stdout:6/354: stat d1/d3/d16 0 2024-09-16T08:57:08.620 INFO:tasks.workunit.client.0.smithi031.stdout:8/26: mkdir d4/d7 0 2024-09-16T08:57:08.620 INFO:tasks.workunit.client.0.smithi031.stdout:8/27: creat d4/f8 x:0 0 0 2024-09-16T08:57:08.621 INFO:tasks.workunit.client.0.smithi031.stdout:3/28: mkdir d3/dc 0 2024-09-16T08:57:08.623 INFO:tasks.workunit.client.0.smithi031.stdout:5/60: unlink d2/dc/l14 0 2024-09-16T08:57:08.624 INFO:tasks.workunit.client.0.smithi031.stdout:5/61: stat d2/dc/c10 0 2024-09-16T08:57:08.624 INFO:tasks.workunit.client.0.smithi031.stdout:5/62: fsync d2/f11 0 2024-09-16T08:57:08.624 INFO:tasks.workunit.client.0.smithi031.stdout:5/63: rename d2 to d2/dc/d15 22 2024-09-16T08:57:08.625 INFO:tasks.workunit.client.0.smithi031.stdout:2/17: getdents . 0 2024-09-16T08:57:08.625 INFO:tasks.workunit.client.0.smithi031.stdout:2/18: fdatasync f2 0 2024-09-16T08:57:08.626 INFO:tasks.workunit.client.0.smithi031.stdout:5/64: unlink d2/dc/ff 0 2024-09-16T08:57:08.627 INFO:tasks.workunit.client.0.smithi031.stdout:2/19: mknod c5 0 2024-09-16T08:57:08.628 INFO:tasks.workunit.client.0.smithi031.stdout:5/65: link d2/f13 d2/dc/f16 0 2024-09-16T08:57:08.628 INFO:tasks.workunit.client.0.smithi031.stdout:5/66: fdatasync d2/dc/fd 0 2024-09-16T08:57:08.628 INFO:tasks.workunit.client.0.smithi031.stdout:0/38: dwrite d6/f8 [0,4194304] 0 2024-09-16T08:57:08.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/67: symlink d2/dc/l17 0 2024-09-16T08:57:08.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/68: fdatasync d2/fe 0 2024-09-16T08:57:08.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/69: chown d2/f13 17040 1 2024-09-16T08:57:08.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/70: chown d2/fb 43 1 2024-09-16T08:57:08.631 INFO:tasks.workunit.client.0.smithi031.stdout:2/20: mkdir d6 0 2024-09-16T08:57:08.631 INFO:tasks.workunit.client.0.smithi031.stdout:2/21: write f2 [416634,50137] 0 2024-09-16T08:57:08.634 INFO:tasks.workunit.client.0.smithi031.stdout:2/22: link f4 d6/f7 0 2024-09-16T08:57:08.636 INFO:tasks.workunit.client.0.smithi031.stdout:2/23: creat d6/f8 x:0 0 0 2024-09-16T08:57:08.653 INFO:tasks.workunit.client.1.smithi097.stdout:5/375: dwrite db/fe [4194304,4194304] 0 2024-09-16T08:57:08.654 INFO:tasks.workunit.client.0.smithi031.stdout:1/56: dwrite d2/d4/f12 [0,4194304] 0 2024-09-16T08:57:08.655 INFO:tasks.workunit.client.1.smithi097.stdout:5/376: creat db/d22/d58/f7d x:0 0 0 2024-09-16T08:57:08.659 INFO:tasks.workunit.client.0.smithi031.stdout:0/39: dwrite d6/fc [0,4194304] 0 2024-09-16T08:57:08.661 INFO:tasks.workunit.client.0.smithi031.stdout:0/40: rename d6/cb to d6/ce 0 2024-09-16T08:57:08.663 INFO:tasks.workunit.client.0.smithi031.stdout:0/41: symlink d6/lf 0 2024-09-16T08:57:08.663 INFO:tasks.workunit.client.0.smithi031.stdout:0/42: chown c1 172094 1 2024-09-16T08:57:08.664 INFO:tasks.workunit.client.0.smithi031.stdout:0/43: mknod d6/c10 0 2024-09-16T08:57:08.665 INFO:tasks.workunit.client.0.smithi031.stdout:0/44: creat d6/f11 x:0 0 0 2024-09-16T08:57:08.667 INFO:tasks.workunit.client.0.smithi031.stdout:2/24: dwrite f4 [0,4194304] 0 2024-09-16T08:57:08.667 INFO:tasks.workunit.client.0.smithi031.stdout:5/71: dwrite d2/dc/fd [0,4194304] 0 2024-09-16T08:57:08.667 INFO:tasks.workunit.client.0.smithi031.stdout:5/72: write d2/f11 [324877,5180] 0 2024-09-16T08:57:08.672 INFO:tasks.workunit.client.0.smithi031.stdout:2/25: rename d6/f7 to d6/f9 0 2024-09-16T08:57:08.682 INFO:tasks.workunit.client.0.smithi031.stdout:0/45: dwrite d6/f11 [0,4194304] 0 2024-09-16T08:57:08.682 INFO:tasks.workunit.client.0.smithi031.stdout:0/46: stat d6/f11 0 2024-09-16T08:57:08.686 INFO:tasks.workunit.client.0.smithi031.stdout:2/26: write d6/f9 [3148744,129171] 0 2024-09-16T08:57:08.688 INFO:tasks.workunit.client.0.smithi031.stdout:2/27: mkdir d6/da 0 2024-09-16T08:57:08.688 INFO:tasks.workunit.client.0.smithi031.stdout:2/28: creat d6/fb x:0 0 0 2024-09-16T08:57:08.688 INFO:tasks.workunit.client.0.smithi031.stdout:2/29: write d6/f9 [3309836,14725] 0 2024-09-16T08:57:08.691 INFO:tasks.workunit.client.0.smithi031.stdout:7/91: dwrite d2/fb [0,4194304] 0 2024-09-16T08:57:08.692 INFO:tasks.workunit.client.0.smithi031.stdout:7/92: creat d2/d8/f1d x:0 0 0 2024-09-16T08:57:08.693 INFO:tasks.workunit.client.0.smithi031.stdout:7/93: write d2/d8/d11/f1b [253824,2349] 0 2024-09-16T08:57:08.693 INFO:tasks.workunit.client.0.smithi031.stdout:0/47: read d6/f11 [21799,14201] 0 2024-09-16T08:57:08.695 INFO:tasks.workunit.client.1.smithi097.stdout:6/355: write d1/d3/d3e/d5a/f4a [2721225,27713] 0 2024-09-16T08:57:08.695 INFO:tasks.workunit.client.1.smithi097.stdout:6/356: truncate d1/d3/f4b 1374737 0 2024-09-16T08:57:08.695 INFO:tasks.workunit.client.0.smithi031.stdout:7/94: symlink d2/d8/d11/l1e 0 2024-09-16T08:57:08.697 INFO:tasks.workunit.client.0.smithi031.stdout:5/73: dwrite d2/f11 [0,4194304] 0 2024-09-16T08:57:08.700 INFO:tasks.workunit.client.0.smithi031.stdout:7/95: symlink d2/d8/d11/l1f 0 2024-09-16T08:57:08.703 INFO:tasks.workunit.client.0.smithi031.stdout:9/9: fsync f2 0 2024-09-16T08:57:08.705 INFO:tasks.workunit.client.0.smithi031.stdout:0/48: mkdir d6/d12 0 2024-09-16T08:57:08.706 INFO:tasks.workunit.client.0.smithi031.stdout:1/57: dread d2/d4/f12 [0,4194304] 0 2024-09-16T08:57:08.706 INFO:tasks.workunit.client.0.smithi031.stdout:1/58: write d2/f3 [1243923,23116] 0 2024-09-16T08:57:08.707 INFO:tasks.workunit.client.0.smithi031.stdout:6/23: rmdir d0 39 2024-09-16T08:57:08.709 INFO:tasks.workunit.client.0.smithi031.stdout:5/74: mkdir d2/d18 0 2024-09-16T08:57:08.709 INFO:tasks.workunit.client.0.smithi031.stdout:5/75: write d2/fe [303194,26233] 0 2024-09-16T08:57:08.713 INFO:tasks.workunit.client.0.smithi031.stdout:7/96: symlink d2/d8/d11/l20 0 2024-09-16T08:57:08.714 INFO:tasks.workunit.client.0.smithi031.stdout:1/59: read d2/f3 [505831,21641] 0 2024-09-16T08:57:08.716 INFO:tasks.workunit.client.0.smithi031.stdout:0/49: mkdir d6/d12/d13 0 2024-09-16T08:57:08.717 INFO:tasks.workunit.client.1.smithi097.stdout:6/357: dwrite d1/d3/f4b [0,4194304] 0 2024-09-16T08:57:08.718 INFO:tasks.workunit.client.0.smithi031.stdout:6/24: creat d0/f4 x:0 0 0 2024-09-16T08:57:08.718 INFO:tasks.workunit.client.0.smithi031.stdout:6/25: stat d0/f4 0 2024-09-16T08:57:08.718 INFO:tasks.workunit.client.0.smithi031.stdout:6/26: truncate d0/f3 921828 0 2024-09-16T08:57:08.718 INFO:tasks.workunit.client.0.smithi031.stdout:3/29: dwrite d3/f4 [0,4194304] 0 2024-09-16T08:57:08.718 INFO:tasks.workunit.client.0.smithi031.stdout:3/30: chown f1 3741 1 2024-09-16T08:57:08.718 INFO:tasks.workunit.client.0.smithi031.stdout:3/31: truncate f1 4643317 0 2024-09-16T08:57:08.720 INFO:tasks.workunit.client.0.smithi031.stdout:1/60: creat d2/de/f13 x:0 0 0 2024-09-16T08:57:08.723 INFO:tasks.workunit.client.0.smithi031.stdout:0/50: link c1 d6/d12/d13/c14 0 2024-09-16T08:57:08.727 INFO:tasks.workunit.client.0.smithi031.stdout:7/97: mknod d2/d8/c21 0 2024-09-16T08:57:08.727 INFO:tasks.workunit.client.0.smithi031.stdout:7/98: creat d2/f22 x:0 0 0 2024-09-16T08:57:08.729 INFO:tasks.workunit.client.0.smithi031.stdout:1/61: rename d2/d4/fa to d2/f14 0 2024-09-16T08:57:08.730 INFO:tasks.workunit.client.0.smithi031.stdout:0/51: unlink f4 0 2024-09-16T08:57:08.733 INFO:tasks.workunit.client.0.smithi031.stdout:7/99: symlink d2/l23 0 2024-09-16T08:57:08.733 INFO:tasks.workunit.client.0.smithi031.stdout:3/32: write d3/f4 [454879,2632] 0 2024-09-16T08:57:08.733 INFO:tasks.workunit.client.0.smithi031.stdout:3/33: chown c0 98 1 2024-09-16T08:57:08.733 INFO:tasks.workunit.client.0.smithi031.stdout:8/28: getdents d4 0 2024-09-16T08:57:08.733 INFO:tasks.workunit.client.0.smithi031.stdout:8/29: fdatasync d4/f6 0 2024-09-16T08:57:08.734 INFO:tasks.workunit.client.0.smithi031.stdout:9/10: dwrite f2 [0,4194304] 0 2024-09-16T08:57:08.735 INFO:tasks.workunit.client.0.smithi031.stdout:9/11: stat d3 0 2024-09-16T08:57:08.735 INFO:tasks.workunit.client.0.smithi031.stdout:7/100: rename d2/d8/fe to d2/d8/d11/f24 0 2024-09-16T08:57:08.735 INFO:tasks.workunit.client.0.smithi031.stdout:7/101: write d2/d8/d11/f24 [1009214,22883] 0 2024-09-16T08:57:08.736 INFO:tasks.workunit.client.0.smithi031.stdout:3/34: link d3/c6 d3/dc/cd 0 2024-09-16T08:57:08.736 INFO:tasks.workunit.client.0.smithi031.stdout:3/35: stat c2 0 2024-09-16T08:57:08.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/30: mknod d4/c9 0 2024-09-16T08:57:08.744 INFO:tasks.workunit.client.0.smithi031.stdout:5/76: dwrite d2/f5 [0,4194304] 0 2024-09-16T08:57:08.744 INFO:tasks.workunit.client.0.smithi031.stdout:5/77: chown d2/dc 3 1 2024-09-16T08:57:08.745 INFO:tasks.workunit.client.1.smithi097.stdout:6/358: dwrite d1/f64 [0,4194304] 0 2024-09-16T08:57:08.745 INFO:tasks.workunit.client.0.smithi031.stdout:1/62: dread d2/f14 [0,4194304] 0 2024-09-16T08:57:08.745 INFO:tasks.workunit.client.0.smithi031.stdout:6/27: dwrite d0/f1 [0,4194304] 0 2024-09-16T08:57:08.746 INFO:tasks.workunit.client.0.smithi031.stdout:7/102: rename d2/l17 to d2/d8/l25 0 2024-09-16T08:57:08.746 INFO:tasks.workunit.client.0.smithi031.stdout:7/103: chown d2/d8/d11/l1e 91570728 1 2024-09-16T08:57:08.746 INFO:tasks.workunit.client.0.smithi031.stdout:3/36: symlink d3/le 0 2024-09-16T08:57:08.749 INFO:tasks.workunit.client.0.smithi031.stdout:5/78: dread d2/fe [0,4194304] 0 2024-09-16T08:57:08.749 INFO:tasks.workunit.client.0.smithi031.stdout:5/79: stat d2/dc/c10 0 2024-09-16T08:57:08.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/80: write d2/fb [979132,39169] 0 2024-09-16T08:57:08.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/81: creat d2/dc/f19 x:0 0 0 2024-09-16T08:57:08.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/82: readlink d2/dc/l17 0 2024-09-16T08:57:08.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/83: fsync d2/f11 0 2024-09-16T08:57:08.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/84: creat d2/f1a x:0 0 0 2024-09-16T08:57:08.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/85: chown d2/dc/fd 277 1 2024-09-16T08:57:08.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/30: fsync d6/f9 0 2024-09-16T08:57:08.754 INFO:tasks.workunit.client.0.smithi031.stdout:0/52: truncate d6/f11 2935323 0 2024-09-16T08:57:08.762 INFO:tasks.workunit.client.0.smithi031.stdout:2/31: unlink c5 0 2024-09-16T08:57:08.762 INFO:tasks.workunit.client.0.smithi031.stdout:2/32: fsync f4 0 2024-09-16T08:57:08.762 INFO:tasks.workunit.client.0.smithi031.stdout:2/33: write d6/fb [630873,110973] 0 2024-09-16T08:57:08.762 INFO:tasks.workunit.client.0.smithi031.stdout:2/34: creat d6/fc x:0 0 0 2024-09-16T08:57:08.763 INFO:tasks.workunit.client.0.smithi031.stdout:0/53: rename c1 to d6/d12/d13/c15 0 2024-09-16T08:57:08.767 INFO:tasks.workunit.client.0.smithi031.stdout:3/37: mkdir d3/df 0 2024-09-16T08:57:08.770 INFO:tasks.workunit.client.1.smithi097.stdout:0/344: sync 2024-09-16T08:57:08.770 INFO:tasks.workunit.client.1.smithi097.stdout:0/345: write da/df/d1f/d3b/f5a [473120,38764] 0 2024-09-16T08:57:08.770 INFO:tasks.workunit.client.1.smithi097.stdout:0/346: chown da/df 14 1 2024-09-16T08:57:08.770 INFO:tasks.workunit.client.0.smithi031.stdout:8/31: dwrite d4/f8 [0,4194304] 0 2024-09-16T08:57:08.770 INFO:tasks.workunit.client.0.smithi031.stdout:8/32: dread - d4/f6 zero size 2024-09-16T08:57:08.773 INFO:tasks.workunit.client.0.smithi031.stdout:6/28: dwrite d0/f1 [0,4194304] 0 2024-09-16T08:57:08.773 INFO:tasks.workunit.client.0.smithi031.stdout:6/29: truncate d0/f4 552812 0 2024-09-16T08:57:08.776 INFO:tasks.workunit.client.0.smithi031.stdout:6/30: dread d0/f3 [0,4194304] 0 2024-09-16T08:57:08.784 INFO:tasks.workunit.client.1.smithi097.stdout:6/359: dwrite d1/d3/d3e/f61 [0,4194304] 0 2024-09-16T08:57:08.789 INFO:tasks.workunit.client.0.smithi031.stdout:8/33: write d4/f8 [1264228,15668] 0 2024-09-16T08:57:08.790 INFO:tasks.workunit.client.0.smithi031.stdout:6/31: dwrite d0/f4 [0,4194304] 0 2024-09-16T08:57:08.804 INFO:tasks.workunit.client.1.smithi097.stdout:0/347: mkdir da/d7e 0 2024-09-16T08:57:08.804 INFO:tasks.workunit.client.1.smithi097.stdout:0/348: write da/df/d1f/d3b/f6a [711452,129516] 0 2024-09-16T08:57:08.804 INFO:tasks.workunit.client.1.smithi097.stdout:0/349: chown da/c51 2001 1 2024-09-16T08:57:08.807 INFO:tasks.workunit.client.1.smithi097.stdout:6/360: dwrite d1/d3/d3e/f61 [0,4194304] 0 2024-09-16T08:57:08.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/35: fdatasync f1 0 2024-09-16T08:57:08.808 INFO:tasks.workunit.client.0.smithi031.stdout:8/34: dwrite d4/f6 [0,4194304] 0 2024-09-16T08:57:08.808 INFO:tasks.workunit.client.0.smithi031.stdout:8/35: readlink - no filename 2024-09-16T08:57:08.809 INFO:tasks.workunit.client.0.smithi031.stdout:0/54: stat d6/d12/d13/c15 0 2024-09-16T08:57:08.811 INFO:tasks.workunit.client.0.smithi031.stdout:1/63: getdents d2/d4 0 2024-09-16T08:57:08.811 INFO:tasks.workunit.client.0.smithi031.stdout:1/64: dread - d2/de/f13 zero size 2024-09-16T08:57:08.813 INFO:tasks.workunit.client.0.smithi031.stdout:2/36: dread d6/f9 [0,4194304] 0 2024-09-16T08:57:08.813 INFO:tasks.workunit.client.0.smithi031.stdout:2/37: chown d6 150384 1 2024-09-16T08:57:08.813 INFO:tasks.workunit.client.0.smithi031.stdout:2/38: fsync d6/f8 0 2024-09-16T08:57:08.813 INFO:tasks.workunit.client.0.smithi031.stdout:3/38: rename d3/f8 to d3/df/f10 0 2024-09-16T08:57:08.813 INFO:tasks.workunit.client.0.smithi031.stdout:3/39: dread - d3/df/f10 zero size 2024-09-16T08:57:08.817 INFO:tasks.workunit.client.0.smithi031.stdout:9/12: truncate f2 2346272 0 2024-09-16T08:57:08.817 INFO:tasks.workunit.client.0.smithi031.stdout:9/13: chown d3 7892877 1 2024-09-16T08:57:08.818 INFO:tasks.workunit.client.0.smithi031.stdout:7/104: rmdir d2/d8 39 2024-09-16T08:57:08.818 INFO:tasks.workunit.client.0.smithi031.stdout:7/105: write d2/d8/fd [1532653,83913] 0 2024-09-16T08:57:08.824 INFO:tasks.workunit.client.0.smithi031.stdout:3/40: write f1 [999878,61803] 0 2024-09-16T08:57:08.825 INFO:tasks.workunit.client.0.smithi031.stdout:3/41: write d3/df/f10 [157042,19464] 0 2024-09-16T08:57:08.847 INFO:tasks.workunit.client.1.smithi097.stdout:0/350: dread da/d1a/f63 [0,4194304] 0 2024-09-16T08:57:08.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/361: getdents d1/dc 0 2024-09-16T08:57:08.867 INFO:tasks.workunit.client.0.smithi031.stdout:6/32: mknod d0/c5 0 2024-09-16T08:57:08.867 INFO:tasks.workunit.client.0.smithi031.stdout:6/33: write d0/f3 [319058,91079] 0 2024-09-16T08:57:08.874 INFO:tasks.workunit.client.1.smithi097.stdout:6/362: symlink d1/d3/d16/l6e 0 2024-09-16T08:57:08.879 INFO:tasks.workunit.client.1.smithi097.stdout:0/351: rmdir da/d24 39 2024-09-16T08:57:08.879 INFO:tasks.workunit.client.1.smithi097.stdout:0/352: creat da/df/d1f/d3b/f7f x:0 0 0 2024-09-16T08:57:08.879 INFO:tasks.workunit.client.1.smithi097.stdout:0/353: chown da/c36 197 1 2024-09-16T08:57:08.881 INFO:tasks.workunit.client.1.smithi097.stdout:6/363: rmdir d1/d3/d3e/d5a 39 2024-09-16T08:57:08.885 INFO:tasks.workunit.client.0.smithi031.stdout:0/55: creat d6/d12/f16 x:0 0 0 2024-09-16T08:57:08.888 INFO:tasks.workunit.client.0.smithi031.stdout:1/65: mkdir d2/d15 0 2024-09-16T08:57:08.889 INFO:tasks.workunit.client.1.smithi097.stdout:6/364: dread d1/f64 [0,4194304] 0 2024-09-16T08:57:08.891 INFO:tasks.workunit.client.0.smithi031.stdout:2/39: creat d6/da/fd x:0 0 0 2024-09-16T08:57:08.899 INFO:tasks.workunit.client.0.smithi031.stdout:6/34: unlink d0/f3 0 2024-09-16T08:57:08.900 INFO:tasks.workunit.client.0.smithi031.stdout:0/56: read d6/f11 [497510,102234] 0 2024-09-16T08:57:08.904 INFO:tasks.workunit.client.0.smithi031.stdout:2/40: rename f2 to d6/da/fe 0 2024-09-16T08:57:08.905 INFO:tasks.workunit.client.0.smithi031.stdout:7/106: mknod d2/d8/c26 0 2024-09-16T08:57:08.906 INFO:tasks.workunit.client.1.smithi097.stdout:6/365: dwrite d1/d3/f34 [4194304,4194304] 0 2024-09-16T08:57:08.910 INFO:tasks.workunit.client.1.smithi097.stdout:6/366: rmdir d1/d3/d16 39 2024-09-16T08:57:08.910 INFO:tasks.workunit.client.1.smithi097.stdout:6/367: creat d1/dc/d3c/f6f x:0 0 0 2024-09-16T08:57:08.914 INFO:tasks.workunit.client.0.smithi031.stdout:7/107: getdents d2 0 2024-09-16T08:57:08.914 INFO:tasks.workunit.client.0.smithi031.stdout:7/108: getdents d2 0 2024-09-16T08:57:08.917 INFO:tasks.workunit.client.0.smithi031.stdout:6/35: dwrite d0/f4 [0,4194304] 0 2024-09-16T08:57:08.924 INFO:tasks.workunit.client.1.smithi097.stdout:4/355: sync 2024-09-16T08:57:08.924 INFO:tasks.workunit.client.1.smithi097.stdout:4/356: chown d0/d7/df/c2e 18043 1 2024-09-16T08:57:08.925 INFO:tasks.workunit.client.0.smithi031.stdout:3/42: link d3/df/f10 d3/f11 0 2024-09-16T08:57:08.925 INFO:tasks.workunit.client.1.smithi097.stdout:1/359: sync 2024-09-16T08:57:08.926 INFO:tasks.workunit.client.1.smithi097.stdout:7/354: sync 2024-09-16T08:57:08.926 INFO:tasks.workunit.client.1.smithi097.stdout:7/355: write d4/d9/dc/f6b [656501,37610] 0 2024-09-16T08:57:08.926 INFO:tasks.workunit.client.1.smithi097.stdout:1/360: dread d5/d2c/f4c [0,4194304] 0 2024-09-16T08:57:08.927 INFO:tasks.workunit.client.0.smithi031.stdout:3/43: dread d3/df/f10 [0,4194304] 0 2024-09-16T08:57:08.928 INFO:tasks.workunit.client.0.smithi031.stdout:3/44: write d3/fa [892832,5662] 0 2024-09-16T08:57:08.928 INFO:tasks.workunit.client.0.smithi031.stdout:3/45: write d3/f4 [4649552,30617] 0 2024-09-16T08:57:08.928 INFO:tasks.workunit.client.0.smithi031.stdout:3/46: dread - d3/fb zero size 2024-09-16T08:57:08.930 INFO:tasks.workunit.client.0.smithi031.stdout:6/36: read d0/f1 [1586826,92631] 0 2024-09-16T08:57:08.930 INFO:tasks.workunit.client.0.smithi031.stdout:6/37: stat d0/f4 0 2024-09-16T08:57:08.931 INFO:tasks.workunit.client.0.smithi031.stdout:6/38: chown d0/c5 30451 1 2024-09-16T08:57:08.931 INFO:tasks.workunit.client.0.smithi031.stdout:6/39: stat d0 0 2024-09-16T08:57:08.931 INFO:tasks.workunit.client.0.smithi031.stdout:6/40: write d0/f1 [4155066,67291] 0 2024-09-16T08:57:08.932 INFO:tasks.workunit.client.0.smithi031.stdout:3/47: link c0 d3/dc/c12 0 2024-09-16T08:57:08.932 INFO:tasks.workunit.client.1.smithi097.stdout:4/357: creat d0/d7/d52/d26/d3d/f82 x:0 0 0 2024-09-16T08:57:08.932 INFO:tasks.workunit.client.1.smithi097.stdout:1/361: symlink d5/d17/d54/d6b/l6e 0 2024-09-16T08:57:08.933 INFO:tasks.workunit.client.0.smithi031.stdout:3/48: creat d3/dc/f13 x:0 0 0 2024-09-16T08:57:08.934 INFO:tasks.workunit.client.0.smithi031.stdout:3/49: write d3/dc/f13 [672665,44274] 0 2024-09-16T08:57:08.935 INFO:tasks.workunit.client.0.smithi031.stdout:3/50: mkdir d3/d14 0 2024-09-16T08:57:08.935 INFO:tasks.workunit.client.0.smithi031.stdout:3/51: stat d3 0 2024-09-16T08:57:08.935 INFO:tasks.workunit.client.0.smithi031.stdout:3/52: write d3/dc/f13 [1223203,103258] 0 2024-09-16T08:57:08.938 INFO:tasks.workunit.client.1.smithi097.stdout:4/358: write d0/d39/f6a [2708859,53992] 0 2024-09-16T08:57:08.940 INFO:tasks.workunit.client.1.smithi097.stdout:1/362: dread d5/d2c/f2e [4194304,4194304] 0 2024-09-16T08:57:08.940 INFO:tasks.workunit.client.1.smithi097.stdout:1/363: fsync d5/db/f3c 0 2024-09-16T08:57:08.940 INFO:tasks.workunit.client.1.smithi097.stdout:4/359: mkdir d0/d7/df/d83 0 2024-09-16T08:57:08.943 INFO:tasks.workunit.client.1.smithi097.stdout:4/360: mknod d0/d7/df/c84 0 2024-09-16T08:57:08.943 INFO:tasks.workunit.client.1.smithi097.stdout:4/361: fsync d0/d7/df/d15/f41 0 2024-09-16T08:57:08.943 INFO:tasks.workunit.client.1.smithi097.stdout:4/362: truncate d0/d7/d52/d26/d3d/f82 713559 0 2024-09-16T08:57:08.943 INFO:tasks.workunit.client.1.smithi097.stdout:4/363: truncate d0/d35/f38 552582 0 2024-09-16T08:57:08.943 INFO:tasks.workunit.client.1.smithi097.stdout:4/364: truncate d0/d23/d5f/f2b 798122 0 2024-09-16T08:57:08.945 INFO:tasks.workunit.client.0.smithi031.stdout:6/41: write d0/f4 [2163493,87949] 0 2024-09-16T08:57:08.945 INFO:tasks.workunit.client.0.smithi031.stdout:6/42: creat d0/f6 x:0 0 0 2024-09-16T08:57:08.945 INFO:tasks.workunit.client.1.smithi097.stdout:1/364: creat d5/db/f6f x:0 0 0 2024-09-16T08:57:08.946 INFO:tasks.workunit.client.1.smithi097.stdout:1/365: read f4 [465292,72009] 0 2024-09-16T08:57:08.948 INFO:tasks.workunit.client.0.smithi031.stdout:7/109: dwrite d2/f22 [0,4194304] 0 2024-09-16T08:57:08.949 INFO:tasks.workunit.client.0.smithi031.stdout:8/36: dwrite d4/f8 [4194304,4194304] 0 2024-09-16T08:57:08.953 INFO:tasks.workunit.client.0.smithi031.stdout:9/14: dwrite f2 [0,4194304] 0 2024-09-16T08:57:08.954 INFO:tasks.workunit.client.0.smithi031.stdout:6/43: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:08.954 INFO:tasks.workunit.client.0.smithi031.stdout:7/110: mkdir d2/d8/d27 0 2024-09-16T08:57:08.956 INFO:tasks.workunit.client.0.smithi031.stdout:6/44: rename d0/l2 to d0/l7 0 2024-09-16T08:57:08.956 INFO:tasks.workunit.client.0.smithi031.stdout:6/45: write d0/f4 [3706786,121208] 0 2024-09-16T08:57:08.957 INFO:tasks.workunit.client.0.smithi031.stdout:7/111: rmdir d2/d8/d27 0 2024-09-16T08:57:08.959 INFO:tasks.workunit.client.0.smithi031.stdout:7/112: symlink d2/l28 0 2024-09-16T08:57:08.959 INFO:tasks.workunit.client.0.smithi031.stdout:7/113: chown d2/d8/d11/l19 368614 1 2024-09-16T08:57:08.959 INFO:tasks.workunit.client.0.smithi031.stdout:7/114: chown d2/f18 6031435 1 2024-09-16T08:57:08.962 INFO:tasks.workunit.client.0.smithi031.stdout:7/115: mknod d2/d8/c29 0 2024-09-16T08:57:08.963 INFO:tasks.workunit.client.0.smithi031.stdout:7/116: mknod d2/c2a 0 2024-09-16T08:57:08.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/37: dread d4/f6 [0,4194304] 0 2024-09-16T08:57:08.966 INFO:tasks.workunit.client.0.smithi031.stdout:7/117: link d2/l28 d2/d8/l2b 0 2024-09-16T08:57:08.966 INFO:tasks.workunit.client.0.smithi031.stdout:7/118: readlink d2/d8/l10 0 2024-09-16T08:57:08.967 INFO:tasks.workunit.client.0.smithi031.stdout:3/53: dwrite d3/fa [0,4194304] 0 2024-09-16T08:57:08.967 INFO:tasks.workunit.client.0.smithi031.stdout:9/15: write f2 [1329208,118141] 0 2024-09-16T08:57:08.968 INFO:tasks.workunit.client.0.smithi031.stdout:7/119: creat d2/d8/f2c x:0 0 0 2024-09-16T08:57:08.968 INFO:tasks.workunit.client.0.smithi031.stdout:7/120: fsync d2/ff 0 2024-09-16T08:57:08.971 INFO:tasks.workunit.client.0.smithi031.stdout:3/54: unlink c2 0 2024-09-16T08:57:08.986 INFO:tasks.workunit.client.1.smithi097.stdout:8/319: sync 2024-09-16T08:57:08.986 INFO:tasks.workunit.client.1.smithi097.stdout:5/377: sync 2024-09-16T08:57:08.986 INFO:tasks.workunit.client.1.smithi097.stdout:2/340: sync 2024-09-16T08:57:08.986 INFO:tasks.workunit.client.1.smithi097.stdout:9/332: sync 2024-09-16T08:57:08.986 INFO:tasks.workunit.client.1.smithi097.stdout:3/307: sync 2024-09-16T08:57:08.987 INFO:tasks.workunit.client.1.smithi097.stdout:5/378: truncate db/f2e 1914314 0 2024-09-16T08:57:08.987 INFO:tasks.workunit.client.1.smithi097.stdout:5/379: stat db/d1a/f38 0 2024-09-16T08:57:08.987 INFO:tasks.workunit.client.1.smithi097.stdout:8/320: write d8/dd/d15/d2c/f41 [400066,50088] 0 2024-09-16T08:57:08.987 INFO:tasks.workunit.client.1.smithi097.stdout:3/308: mkdir d7/d28/d63 0 2024-09-16T08:57:08.990 INFO:tasks.workunit.client.1.smithi097.stdout:8/321: getdents d8/dd/d15/d2c/d4a/d4d/d3d 0 2024-09-16T08:57:08.991 INFO:tasks.workunit.client.1.smithi097.stdout:8/322: dread - d8/dd/d15/d2c/d4a/d4d/f5c zero size 2024-09-16T08:57:08.991 INFO:tasks.workunit.client.1.smithi097.stdout:5/380: dread db/d22/f4b [0,4194304] 0 2024-09-16T08:57:08.992 INFO:tasks.workunit.client.0.smithi031.stdout:6/46: dwrite d0/f1 [0,4194304] 0 2024-09-16T08:57:08.992 INFO:tasks.workunit.client.0.smithi031.stdout:6/47: creat d0/f8 x:0 0 0 2024-09-16T08:57:08.992 INFO:tasks.workunit.client.1.smithi097.stdout:8/323: truncate d8/dd/d15/f28 1558527 0 2024-09-16T08:57:08.993 INFO:tasks.workunit.client.1.smithi097.stdout:8/324: readlink d8/dd/d15/d16/l53 0 2024-09-16T08:57:08.993 INFO:tasks.workunit.client.1.smithi097.stdout:9/333: read d2/d3f/f41 [417098,16720] 0 2024-09-16T08:57:08.996 INFO:tasks.workunit.client.1.smithi097.stdout:3/309: dread d7/de/d3c/d3f/f51 [0,4194304] 0 2024-09-16T08:57:08.996 INFO:tasks.workunit.client.1.smithi097.stdout:3/310: chown d7/d28/d30/f53 1 1 2024-09-16T08:57:08.996 INFO:tasks.workunit.client.1.smithi097.stdout:2/341: write d3/d4/dd/f35 [1180236,77576] 0 2024-09-16T08:57:08.996 INFO:tasks.workunit.client.1.smithi097.stdout:8/325: creat d8/dd/f61 x:0 0 0 2024-09-16T08:57:08.996 INFO:tasks.workunit.client.1.smithi097.stdout:8/326: chown d8/f1e 160 1 2024-09-16T08:57:08.996 INFO:tasks.workunit.client.1.smithi097.stdout:8/327: truncate d8/dd/f26 597248 0 2024-09-16T08:57:08.997 INFO:tasks.workunit.client.1.smithi097.stdout:9/334: symlink d2/d3f/d62/l68 0 2024-09-16T08:57:08.999 INFO:tasks.workunit.client.1.smithi097.stdout:8/328: mkdir d8/dd/d15/d2c/d4a/d4d/d62 0 2024-09-16T08:57:08.999 INFO:tasks.workunit.client.1.smithi097.stdout:8/329: fdatasync d8/dd/d15/d2c/d4a/d4d/f35 0 2024-09-16T08:57:08.999 INFO:tasks.workunit.client.1.smithi097.stdout:9/335: creat d2/d5/f69 x:0 0 0 2024-09-16T08:57:09.000 INFO:tasks.workunit.client.1.smithi097.stdout:2/342: getdents d3/d4/d8/d34/d48 0 2024-09-16T08:57:09.002 INFO:tasks.workunit.client.1.smithi097.stdout:0/354: rename da/df/d1f to da/d24/d4c/d80 0 2024-09-16T08:57:09.002 INFO:tasks.workunit.client.1.smithi097.stdout:0/355: fsync da/d1a/d41/f50 0 2024-09-16T08:57:09.003 INFO:tasks.workunit.client.1.smithi097.stdout:0/356: fdatasync da/d1a/f22 0 2024-09-16T08:57:09.004 INFO:tasks.workunit.client.1.smithi097.stdout:6/368: rename d1/dc/d1e/c5f to d1/c70 0 2024-09-16T08:57:09.009 INFO:tasks.workunit.client.1.smithi097.stdout:7/356: rename d4/d1b/d36 to d4/d7/d7b 0 2024-09-16T08:57:09.010 INFO:tasks.workunit.client.0.smithi031.stdout:1/66: getdents d2 0 2024-09-16T08:57:09.010 INFO:tasks.workunit.client.0.smithi031.stdout:1/67: write d2/de/f13 [1020760,110066] 0 2024-09-16T08:57:09.011 INFO:tasks.workunit.client.1.smithi097.stdout:7/357: dread d4/f6f [0,4194304] 0 2024-09-16T08:57:09.011 INFO:tasks.workunit.client.1.smithi097.stdout:7/358: chown d4/d7/d7b 0 1 2024-09-16T08:57:09.017 INFO:tasks.workunit.client.0.smithi031.stdout:9/16: dwrite f2 [0,4194304] 0 2024-09-16T08:57:09.017 INFO:tasks.workunit.client.0.smithi031.stdout:8/38: dwrite d4/f6 [0,4194304] 0 2024-09-16T08:57:09.018 INFO:tasks.workunit.client.0.smithi031.stdout:7/121: dwrite d2/d8/f1d [0,4194304] 0 2024-09-16T08:57:09.018 INFO:tasks.workunit.client.0.smithi031.stdout:7/122: chown d2/d8/l2b 14 1 2024-09-16T08:57:09.018 INFO:tasks.workunit.client.0.smithi031.stdout:7/123: write d2/d8/f2c [192153,100282] 0 2024-09-16T08:57:09.019 INFO:tasks.workunit.client.0.smithi031.stdout:1/68: link d2/d4/l7 d2/d4/l16 0 2024-09-16T08:57:09.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/359: unlink d4/d7/d30/f76 0 2024-09-16T08:57:09.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/360: write d4/d1b/f69 [289120,83865] 0 2024-09-16T08:57:09.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/361: truncate d4/d7/d7b/f40 191393 0 2024-09-16T08:57:09.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/362: stat d4/d1b/f1e 0 2024-09-16T08:57:09.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/363: write d4/d1b/d41/f52 [577492,44511] 0 2024-09-16T08:57:09.022 INFO:tasks.workunit.client.0.smithi031.stdout:0/57: write d6/f7 [4340981,110247] 0 2024-09-16T08:57:09.037 INFO:tasks.workunit.client.0.smithi031.stdout:9/17: mkdir d3/d5 0 2024-09-16T08:57:09.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/39: dread d4/f8 [4194304,4194304] 0 2024-09-16T08:57:09.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/40: fsync d4/f6 0 2024-09-16T08:57:09.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/41: chown c1 529387307 1 2024-09-16T08:57:09.042 INFO:tasks.workunit.client.1.smithi097.stdout:3/311: dwrite d7/d28/f2f [0,4194304] 0 2024-09-16T08:57:09.042 INFO:tasks.workunit.client.1.smithi097.stdout:2/343: dwrite d3/d4/d8/f2b [0,4194304] 0 2024-09-16T08:57:09.042 INFO:tasks.workunit.client.1.smithi097.stdout:2/344: write d3/d4/dd/d53/d47/f4b [2337034,5384] 0 2024-09-16T08:57:09.043 INFO:tasks.workunit.client.0.smithi031.stdout:7/124: mkdir d2/d2d 0 2024-09-16T08:57:09.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/364: unlink d4/d1b/f57 0 2024-09-16T08:57:09.044 INFO:tasks.workunit.client.1.smithi097.stdout:0/357: dwrite da/d1a/f1c [0,4194304] 0 2024-09-16T08:57:09.045 INFO:tasks.workunit.client.1.smithi097.stdout:0/358: read f7 [4286070,70419] 0 2024-09-16T08:57:09.045 INFO:tasks.workunit.client.1.smithi097.stdout:0/359: chown da/d24/d4b/l64 0 1 2024-09-16T08:57:09.046 INFO:tasks.workunit.client.0.smithi031.stdout:1/69: creat d2/f17 x:0 0 0 2024-09-16T08:57:09.046 INFO:tasks.workunit.client.1.smithi097.stdout:8/330: dwrite d8/dd/d15/d2c/d4a/d4d/f49 [8388608,4194304] 0 2024-09-16T08:57:09.046 INFO:tasks.workunit.client.1.smithi097.stdout:6/369: dwrite d1/d3/f3d [0,4194304] 0 2024-09-16T08:57:09.048 INFO:tasks.workunit.client.0.smithi031.stdout:0/58: chown d6/f8 1 1 2024-09-16T08:57:09.050 INFO:tasks.workunit.client.1.smithi097.stdout:3/312: mkdir d7/de/d29/d35/d3d/d57/d64 0 2024-09-16T08:57:09.052 INFO:tasks.workunit.client.0.smithi031.stdout:9/18: creat d3/d5/f6 x:0 0 0 2024-09-16T08:57:09.053 INFO:tasks.workunit.client.1.smithi097.stdout:2/345: link d3/f23 d3/d4/dd/d53/f58 0 2024-09-16T08:57:09.054 INFO:tasks.workunit.client.0.smithi031.stdout:8/42: creat d4/fa x:0 0 0 2024-09-16T08:57:09.056 INFO:tasks.workunit.client.1.smithi097.stdout:7/365: creat d4/d7/d7b/f7c x:0 0 0 2024-09-16T08:57:09.057 INFO:tasks.workunit.client.1.smithi097.stdout:8/331: write d8/dd/d15/d2c/d4a/d4e/f5a [3990003,57542] 0 2024-09-16T08:57:09.058 INFO:tasks.workunit.client.0.smithi031.stdout:7/125: creat d2/d2d/f2e x:0 0 0 2024-09-16T08:57:09.060 INFO:tasks.workunit.client.1.smithi097.stdout:3/313: dread d7/de/d29/f2d [0,4194304] 0 2024-09-16T08:57:09.060 INFO:tasks.workunit.client.0.smithi031.stdout:1/70: creat d2/d15/f18 x:0 0 0 2024-09-16T08:57:09.062 INFO:tasks.workunit.client.0.smithi031.stdout:0/59: stat d6/d12 0 2024-09-16T08:57:09.064 INFO:tasks.workunit.client.0.smithi031.stdout:7/126: unlink d2/d8/d11/l1e 0 2024-09-16T08:57:09.067 INFO:tasks.workunit.client.1.smithi097.stdout:4/365: fsync d0/d7/d52/d26/d3d/f82 0 2024-09-16T08:57:09.067 INFO:tasks.workunit.client.0.smithi031.stdout:1/71: stat d2/d4/l6 0 2024-09-16T08:57:09.067 INFO:tasks.workunit.client.0.smithi031.stdout:1/72: readlink d2/d4/l16 0 2024-09-16T08:57:09.068 INFO:tasks.workunit.client.1.smithi097.stdout:0/360: dwrite da/d1a/f3a [0,4194304] 0 2024-09-16T08:57:09.068 INFO:tasks.workunit.client.1.smithi097.stdout:0/361: creat da/d1a/f81 x:0 0 0 2024-09-16T08:57:09.068 INFO:tasks.workunit.client.1.smithi097.stdout:0/362: dread - da/d1a/f81 zero size 2024-09-16T08:57:09.070 INFO:tasks.workunit.client.1.smithi097.stdout:7/366: symlink d4/d9/d1c/d5f/l7d 0 2024-09-16T08:57:09.071 INFO:tasks.workunit.client.0.smithi031.stdout:0/60: link d6/f8 d6/d12/f17 0 2024-09-16T08:57:09.072 INFO:tasks.workunit.client.0.smithi031.stdout:9/19: dread f2 [0,4194304] 0 2024-09-16T08:57:09.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/332: rename d8/dd/l50 to d8/dd/d15/d2c/d4a/d4d/d3d/l63 0 2024-09-16T08:57:09.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/333: readlink d8/dd/l1a 0 2024-09-16T08:57:09.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/334: dread - d8/dd/d15/d2c/d4a/d4d/d3d/f57 zero size 2024-09-16T08:57:09.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/335: fdatasync d8/dd/d15/d2c/d4a/d4d/f49 0 2024-09-16T08:57:09.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/336: chown d8/dd/d15/d2c/d4a/d4e/c4f 15532 1 2024-09-16T08:57:09.075 INFO:tasks.workunit.client.1.smithi097.stdout:8/337: chown d8/dd/c55 5396351 1 2024-09-16T08:57:09.075 INFO:tasks.workunit.client.1.smithi097.stdout:0/363: dread da/d24/d4c/d80/d3b/f5a [0,4194304] 0 2024-09-16T08:57:09.075 INFO:tasks.workunit.client.1.smithi097.stdout:0/364: chown da/df/d3d/l67 0 1 2024-09-16T08:57:09.075 INFO:tasks.workunit.client.1.smithi097.stdout:0/365: fdatasync da/d24/d4c/d80/d42/d5b/d60/d73/f77 0 2024-09-16T08:57:09.075 INFO:tasks.workunit.client.1.smithi097.stdout:0/366: chown da/d24/d4c/d80/d42/d5b/d60/f65 2544232 1 2024-09-16T08:57:09.075 INFO:tasks.workunit.client.0.smithi031.stdout:0/61: unlink d6/d12/d13/c15 0 2024-09-16T08:57:09.076 INFO:tasks.workunit.client.0.smithi031.stdout:7/127: read d2/d8/f1d [1280201,55420] 0 2024-09-16T08:57:09.076 INFO:tasks.workunit.client.0.smithi031.stdout:7/128: dread - d2/f1a zero size 2024-09-16T08:57:09.077 INFO:tasks.workunit.client.1.smithi097.stdout:3/314: symlink d7/d28/d63/l65 0 2024-09-16T08:57:09.078 INFO:tasks.workunit.client.1.smithi097.stdout:4/366: creat d0/d23/d5f/d7d/f85 x:0 0 0 2024-09-16T08:57:09.078 INFO:tasks.workunit.client.0.smithi031.stdout:0/62: mknod d6/d12/c18 0 2024-09-16T08:57:09.079 INFO:tasks.workunit.client.0.smithi031.stdout:1/73: dread d2/d4/f12 [0,4194304] 0 2024-09-16T08:57:09.080 INFO:tasks.workunit.client.0.smithi031.stdout:7/129: creat d2/d8/f2f x:0 0 0 2024-09-16T08:57:09.080 INFO:tasks.workunit.client.0.smithi031.stdout:7/130: fdatasync d2/f22 0 2024-09-16T08:57:09.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/20: write f2 [84686,30584] 0 2024-09-16T08:57:09.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/21: dread - d3/d5/f6 zero size 2024-09-16T08:57:09.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/22: chown d3/d5/f6 1626 1 2024-09-16T08:57:09.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/23: chown c0 126597 1 2024-09-16T08:57:09.085 INFO:tasks.workunit.client.1.smithi097.stdout:0/367: dread da/d1a/f1c [0,4194304] 0 2024-09-16T08:57:09.087 INFO:tasks.workunit.client.1.smithi097.stdout:8/338: unlink d8/dd/d15/d2c/d4a/d4d/d3d/c59 0 2024-09-16T08:57:09.088 INFO:tasks.workunit.client.1.smithi097.stdout:8/339: readlink d8/dd/d15/d2c/l60 0 2024-09-16T08:57:09.088 INFO:tasks.workunit.client.1.smithi097.stdout:7/367: rename d4/d7/d7b/f3a to d4/d9/d2b/f7e 0 2024-09-16T08:57:09.090 INFO:tasks.workunit.client.1.smithi097.stdout:1/366: mkdir d5/d70 0 2024-09-16T08:57:09.090 INFO:tasks.workunit.client.1.smithi097.stdout:1/367: chown d5/db/f52 18 1 2024-09-16T08:57:09.091 INFO:tasks.workunit.client.1.smithi097.stdout:8/340: dread d8/dd/d15/d2c/f41 [0,4194304] 0 2024-09-16T08:57:09.092 INFO:tasks.workunit.client.1.smithi097.stdout:3/315: unlink d7/d43/f5c 0 2024-09-16T08:57:09.093 INFO:tasks.workunit.client.0.smithi031.stdout:0/63: dread d6/fc [0,4194304] 0 2024-09-16T08:57:09.094 INFO:tasks.workunit.client.1.smithi097.stdout:6/370: dwrite d1/d3/d16/f4d [0,4194304] 0 2024-09-16T08:57:09.094 INFO:tasks.workunit.client.0.smithi031.stdout:8/43: dwrite d4/f6 [0,4194304] 0 2024-09-16T08:57:09.094 INFO:tasks.workunit.client.0.smithi031.stdout:8/44: chown d4/f6 34 1 2024-09-16T08:57:09.094 INFO:tasks.workunit.client.0.smithi031.stdout:8/45: creat d4/fb x:0 0 0 2024-09-16T08:57:09.095 INFO:tasks.workunit.client.0.smithi031.stdout:1/74: dread d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:09.095 INFO:tasks.workunit.client.1.smithi097.stdout:2/346: dwrite d3/d4/dd/d53/d47/f4b [0,4194304] 0 2024-09-16T08:57:09.095 INFO:tasks.workunit.client.1.smithi097.stdout:4/367: truncate d0/d7/f24 1981981 0 2024-09-16T08:57:09.095 INFO:tasks.workunit.client.1.smithi097.stdout:4/368: chown d0/d7/df/d15/d1f/f81 47 1 2024-09-16T08:57:09.096 INFO:tasks.workunit.client.1.smithi097.stdout:4/369: read - d0/d23/d5f/d7d/f85 zero size 2024-09-16T08:57:09.098 INFO:tasks.workunit.client.1.smithi097.stdout:0/368: mknod da/d24/d4c/d80/d3b/c82 0 2024-09-16T08:57:09.099 INFO:tasks.workunit.client.0.smithi031.stdout:0/64: rmdir d6 39 2024-09-16T08:57:09.099 INFO:tasks.workunit.client.0.smithi031.stdout:0/65: fdatasync d6/fc 0 2024-09-16T08:57:09.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/75: read d2/d4/f12 [3547909,82361] 0 2024-09-16T08:57:09.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/76: write d2/d4/f12 [1565610,23246] 0 2024-09-16T08:57:09.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/77: chown d2/d4/cb 9593610 1 2024-09-16T08:57:09.101 INFO:tasks.workunit.client.1.smithi097.stdout:7/368: symlink d4/d7/d7b/l7f 0 2024-09-16T08:57:09.103 INFO:tasks.workunit.client.1.smithi097.stdout:8/341: symlink d8/dd/d15/d2c/d4a/d4d/d62/l64 0 2024-09-16T08:57:09.107 INFO:tasks.workunit.client.1.smithi097.stdout:3/316: rename d7/de/f5f to d7/d43/f66 0 2024-09-16T08:57:09.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/48: rename d0/l7 to d0/l9 0 2024-09-16T08:57:09.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/49: dread - d0/f8 zero size 2024-09-16T08:57:09.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/50: dread - d0/f6 zero size 2024-09-16T08:57:09.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/51: dread - d0/f8 zero size 2024-09-16T08:57:09.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/52: chown d0/f8 0 1 2024-09-16T08:57:09.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/53: chown d0 50246402 1 2024-09-16T08:57:09.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/54: read - d0/f8 zero size 2024-09-16T08:57:09.111 INFO:tasks.workunit.client.0.smithi031.stdout:8/46: rename d4/c5 to d4/cc 0 2024-09-16T08:57:09.112 INFO:tasks.workunit.client.1.smithi097.stdout:6/371: stat d1/d3/d3e/d5a/l69 0 2024-09-16T08:57:09.116 INFO:tasks.workunit.client.1.smithi097.stdout:2/347: creat d3/d4/d8/d51/f59 x:0 0 0 2024-09-16T08:57:09.116 INFO:tasks.workunit.client.1.smithi097.stdout:2/348: write d3/d4/d9/f36 [629159,81097] 0 2024-09-16T08:57:09.117 INFO:tasks.workunit.client.1.smithi097.stdout:2/349: stat d3/d4/dd 0 2024-09-16T08:57:09.125 INFO:tasks.workunit.client.1.smithi097.stdout:7/369: dwrite d4/d9/dc/f16 [4194304,4194304] 0 2024-09-16T08:57:09.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/47: unlink d4/fb 0 2024-09-16T08:57:09.129 INFO:tasks.workunit.client.1.smithi097.stdout:1/368: dwrite d5/d2c/d3b/f5a [0,4194304] 0 2024-09-16T08:57:09.131 INFO:tasks.workunit.client.0.smithi031.stdout:1/78: dwrite d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:09.131 INFO:tasks.workunit.client.1.smithi097.stdout:4/370: symlink d0/d7/d52/d26/l86 0 2024-09-16T08:57:09.134 INFO:tasks.workunit.client.1.smithi097.stdout:0/369: symlink da/d1a/d26/l83 0 2024-09-16T08:57:09.135 INFO:tasks.workunit.client.1.smithi097.stdout:8/342: creat d8/dd/d15/d2c/d4a/d4d/d3f/f65 x:0 0 0 2024-09-16T08:57:09.140 INFO:tasks.workunit.client.1.smithi097.stdout:6/372: rename d1/d3/d16/l6e to d1/d3/d3e/d5a/l71 0 2024-09-16T08:57:09.140 INFO:tasks.workunit.client.1.smithi097.stdout:3/317: creat d7/de/d29/d35/f67 x:0 0 0 2024-09-16T08:57:09.140 INFO:tasks.workunit.client.1.smithi097.stdout:3/318: fsync d7/d43/f4a 0 2024-09-16T08:57:09.142 INFO:tasks.workunit.client.0.smithi031.stdout:0/66: dread d6/f9 [0,4194304] 0 2024-09-16T08:57:09.145 INFO:tasks.workunit.client.1.smithi097.stdout:7/370: symlink d4/d9/d1c/d5f/l80 0 2024-09-16T08:57:09.148 INFO:tasks.workunit.client.1.smithi097.stdout:1/369: mkdir d5/d17/d54/d6b/d71 0 2024-09-16T08:57:09.150 INFO:tasks.workunit.client.0.smithi031.stdout:6/55: dwrite d0/f1 [0,4194304] 0 2024-09-16T08:57:09.151 INFO:tasks.workunit.client.1.smithi097.stdout:4/371: creat d0/d39/d71/f87 x:0 0 0 2024-09-16T08:57:09.151 INFO:tasks.workunit.client.1.smithi097.stdout:4/372: write d0/d39/d71/f77 [935562,43438] 0 2024-09-16T08:57:09.154 INFO:tasks.workunit.client.0.smithi031.stdout:3/55: dwrite d3/fa [4194304,4194304] 0 2024-09-16T08:57:09.154 INFO:tasks.workunit.client.1.smithi097.stdout:3/319: symlink d7/de/d3c/d3f/l68 0 2024-09-16T08:57:09.157 INFO:tasks.workunit.client.1.smithi097.stdout:1/370: creat d5/d70/f72 x:0 0 0 2024-09-16T08:57:09.159 INFO:tasks.workunit.client.1.smithi097.stdout:3/320: mkdir d7/d28/d30/d69 0 2024-09-16T08:57:09.159 INFO:tasks.workunit.client.1.smithi097.stdout:3/321: chown d7/de/d3c/d3f/f51 1713025189 1 2024-09-16T08:57:09.159 INFO:tasks.workunit.client.1.smithi097.stdout:3/322: write d7/d43/f66 [6677052,127771] 0 2024-09-16T08:57:09.160 INFO:tasks.workunit.client.1.smithi097.stdout:1/371: symlink d5/d2c/l73 0 2024-09-16T08:57:09.162 INFO:tasks.workunit.client.1.smithi097.stdout:3/323: write d7/de/d29/f2d [777996,121432] 0 2024-09-16T08:57:09.163 INFO:tasks.workunit.client.1.smithi097.stdout:1/372: rename d5/db/dc/d19/f58 to d5/db/f74 0 2024-09-16T08:57:09.168 INFO:tasks.workunit.client.0.smithi031.stdout:9/24: rmdir d3/d5 39 2024-09-16T08:57:09.193 INFO:tasks.workunit.client.0.smithi031.stdout:0/67: truncate d6/f7 3250551 0 2024-09-16T08:57:09.194 INFO:tasks.workunit.client.0.smithi031.stdout:0/68: chown d6/lf 24 1 2024-09-16T08:57:09.195 INFO:tasks.workunit.client.0.smithi031.stdout:6/56: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:09.197 INFO:tasks.workunit.client.0.smithi031.stdout:3/56: dread d3/df/f10 [0,4194304] 0 2024-09-16T08:57:09.197 INFO:tasks.workunit.client.0.smithi031.stdout:6/57: unlink d0/l9 0 2024-09-16T08:57:09.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/58: symlink d0/la 0 2024-09-16T08:57:09.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/59: chown d0/f8 133922 1 2024-09-16T08:57:09.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/60: creat d0/fb x:0 0 0 2024-09-16T08:57:09.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/61: chown d0/la 178763901 1 2024-09-16T08:57:09.201 INFO:tasks.workunit.client.0.smithi031.stdout:6/62: creat d0/fc x:0 0 0 2024-09-16T08:57:09.201 INFO:tasks.workunit.client.0.smithi031.stdout:3/57: rmdir d3/dc 39 2024-09-16T08:57:09.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/63: unlink d0/f6 0 2024-09-16T08:57:09.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/64: dread - d0/f8 zero size 2024-09-16T08:57:09.206 INFO:tasks.workunit.client.0.smithi031.stdout:6/65: mkdir d0/dd 0 2024-09-16T08:57:09.206 INFO:tasks.workunit.client.0.smithi031.stdout:6/66: dread - d0/fc zero size 2024-09-16T08:57:09.206 INFO:tasks.workunit.client.0.smithi031.stdout:3/58: write d3/df/f10 [2531573,56058] 0 2024-09-16T08:57:09.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/48: dwrite d4/f6 [4194304,4194304] 0 2024-09-16T08:57:09.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/49: chown d4/f6 2151922 1 2024-09-16T08:57:09.207 INFO:tasks.workunit.client.1.smithi097.stdout:0/370: dwrite da/d24/d4c/d80/d42/d5b/d60/d68/f39 [0,4194304] 0 2024-09-16T08:57:09.207 INFO:tasks.workunit.client.1.smithi097.stdout:5/381: dwrite db/f2e [0,4194304] 0 2024-09-16T08:57:09.207 INFO:tasks.workunit.client.1.smithi097.stdout:5/382: write db/d18/d1c/d42/d48/f78 [456409,79256] 0 2024-09-16T08:57:09.207 INFO:tasks.workunit.client.1.smithi097.stdout:5/383: getdents db/d18/d1c/d42/d46 0 2024-09-16T08:57:09.207 INFO:tasks.workunit.client.1.smithi097.stdout:2/350: dwrite d3/d4/d9/f36 [0,4194304] 0 2024-09-16T08:57:09.208 INFO:tasks.workunit.client.0.smithi031.stdout:1/79: dwrite d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:09.208 INFO:tasks.workunit.client.0.smithi031.stdout:6/67: mknod d0/dd/ce 0 2024-09-16T08:57:09.208 INFO:tasks.workunit.client.1.smithi097.stdout:6/373: dwrite d1/d3/d16/d4f/f55 [0,4194304] 0 2024-09-16T08:57:09.208 INFO:tasks.workunit.client.1.smithi097.stdout:3/324: dwrite d7/de/d29/d35/f22 [0,4194304] 0 2024-09-16T08:57:09.208 INFO:tasks.workunit.client.1.smithi097.stdout:1/373: dwrite d5/db/d25/f53 [0,4194304] 0 2024-09-16T08:57:09.209 INFO:tasks.workunit.client.1.smithi097.stdout:8/343: dwrite d8/dd/f19 [0,4194304] 0 2024-09-16T08:57:09.209 INFO:tasks.workunit.client.1.smithi097.stdout:8/344: readlink d8/dd/d15/d2c/d4a/d4d/l32 0 2024-09-16T08:57:09.216 INFO:tasks.workunit.client.1.smithi097.stdout:0/371: mkdir da/d1a/d26/d75/d84 0 2024-09-16T08:57:09.220 INFO:tasks.workunit.client.0.smithi031.stdout:1/80: symlink d2/l19 0 2024-09-16T08:57:09.224 INFO:tasks.workunit.client.0.smithi031.stdout:6/68: rename d0/f1 to d0/ff 0 2024-09-16T08:57:09.224 INFO:tasks.workunit.client.0.smithi031.stdout:6/69: chown d0/dd 4 1 2024-09-16T08:57:09.224 INFO:tasks.workunit.client.0.smithi031.stdout:6/70: dread - d0/fc zero size 2024-09-16T08:57:09.227 INFO:tasks.workunit.client.0.smithi031.stdout:1/81: dread d2/de/f13 [0,4194304] 0 2024-09-16T08:57:09.232 INFO:tasks.workunit.client.0.smithi031.stdout:6/71: mkdir d0/d10 0 2024-09-16T08:57:09.232 INFO:tasks.workunit.client.0.smithi031.stdout:6/72: write d0/f4 [5006472,99359] 0 2024-09-16T08:57:09.233 INFO:tasks.workunit.client.1.smithi097.stdout:3/325: creat d7/d28/d30/d69/f6a x:0 0 0 2024-09-16T08:57:09.233 INFO:tasks.workunit.client.1.smithi097.stdout:3/326: fsync d7/de/d3c/d3f/f5e 0 2024-09-16T08:57:09.236 INFO:tasks.workunit.client.1.smithi097.stdout:6/374: creat d1/dc/f72 x:0 0 0 2024-09-16T08:57:09.237 INFO:tasks.workunit.client.1.smithi097.stdout:6/375: stat d1/d3/f3d 0 2024-09-16T08:57:09.240 INFO:tasks.workunit.client.0.smithi031.stdout:0/69: mknod d6/d12/d13/c19 0 2024-09-16T08:57:09.241 INFO:tasks.workunit.client.1.smithi097.stdout:6/376: dread d1/d3/d3e/d5a/f32 [0,4194304] 0 2024-09-16T08:57:09.241 INFO:tasks.workunit.client.1.smithi097.stdout:6/377: chown d1/d3/f11 3 1 2024-09-16T08:57:09.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/82: dread d2/f14 [4194304,4194304] 0 2024-09-16T08:57:09.246 INFO:tasks.workunit.client.1.smithi097.stdout:3/327: dread d7/de/d29/d35/f27 [0,4194304] 0 2024-09-16T08:57:09.247 INFO:tasks.workunit.client.0.smithi031.stdout:0/70: link d6/f7 d6/f1a 0 2024-09-16T08:57:09.248 INFO:tasks.workunit.client.1.smithi097.stdout:0/372: unlink da/d24/d4c/c78 0 2024-09-16T08:57:09.249 INFO:tasks.workunit.client.1.smithi097.stdout:4/373: dwrite d0/d39/d71/f77 [0,4194304] 0 2024-09-16T08:57:09.249 INFO:tasks.workunit.client.0.smithi031.stdout:1/83: unlink d2/d4/l6 0 2024-09-16T08:57:09.249 INFO:tasks.workunit.client.0.smithi031.stdout:1/84: write d2/d4/f12 [3444119,127559] 0 2024-09-16T08:57:09.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/85: write d2/f3 [1168951,123746] 0 2024-09-16T08:57:09.251 INFO:tasks.workunit.client.1.smithi097.stdout:3/328: mkdir d7/de/d3c/d3f/d60/d6b 0 2024-09-16T08:57:09.251 INFO:tasks.workunit.client.0.smithi031.stdout:1/86: write d2/de/f13 [391408,91378] 0 2024-09-16T08:57:09.252 INFO:tasks.workunit.client.1.smithi097.stdout:4/374: rmdir d0/d7/d52/d26 39 2024-09-16T08:57:09.253 INFO:tasks.workunit.client.1.smithi097.stdout:5/384: rename db/d18/d1c/d42/d48 to db/d18/d1c/d42/d46/d7e 0 2024-09-16T08:57:09.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/87: mknod d2/c1a 0 2024-09-16T08:57:09.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/88: write d2/d4/f12 [4989749,80236] 0 2024-09-16T08:57:09.254 INFO:tasks.workunit.client.0.smithi031.stdout:1/89: readlink d2/d4/l8 0 2024-09-16T08:57:09.255 INFO:tasks.workunit.client.1.smithi097.stdout:4/375: creat d0/d7/d52/f88 x:0 0 0 2024-09-16T08:57:09.255 INFO:tasks.workunit.client.1.smithi097.stdout:4/376: write d0/d23/d5f/f2b [1735696,35664] 0 2024-09-16T08:57:09.256 INFO:tasks.workunit.client.1.smithi097.stdout:5/385: rmdir db/d18/d1c/d42/d46 39 2024-09-16T08:57:09.256 INFO:tasks.workunit.client.1.smithi097.stdout:5/386: fdatasync db/d18/d1c/d42/d46/f6c 0 2024-09-16T08:57:09.256 INFO:tasks.workunit.client.1.smithi097.stdout:5/387: stat db/d1a/l5d 0 2024-09-16T08:57:09.258 INFO:tasks.workunit.client.1.smithi097.stdout:5/388: truncate db/d18/f34 1437384 0 2024-09-16T08:57:09.259 INFO:tasks.workunit.client.0.smithi031.stdout:9/25: dwrite f2 [4194304,4194304] 0 2024-09-16T08:57:09.259 INFO:tasks.workunit.client.0.smithi031.stdout:9/26: stat f2 0 2024-09-16T08:57:09.259 INFO:tasks.workunit.client.0.smithi031.stdout:9/27: dread - d3/d5/f6 zero size 2024-09-16T08:57:09.260 INFO:tasks.workunit.client.1.smithi097.stdout:1/374: dwrite f4 [0,4194304] 0 2024-09-16T08:57:09.261 INFO:tasks.workunit.client.0.smithi031.stdout:6/73: dwrite d0/fb [0,4194304] 0 2024-09-16T08:57:09.261 INFO:tasks.workunit.client.0.smithi031.stdout:8/50: dwrite d4/f8 [8388608,4194304] 0 2024-09-16T08:57:09.262 INFO:tasks.workunit.client.1.smithi097.stdout:2/351: dwrite d3/d4/dd/f35 [0,4194304] 0 2024-09-16T08:57:09.262 INFO:tasks.workunit.client.1.smithi097.stdout:2/352: readlink d3/d4/dd/l28 0 2024-09-16T08:57:09.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:08 smithi097 ceph-mon[35478]: pgmap v6: 65 pgs: 65 active+clean; 355 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:57:09.263 INFO:tasks.workunit.client.1.smithi097.stdout:3/329: rename d7/d43 to d7/de/d6c 0 2024-09-16T08:57:09.263 INFO:tasks.workunit.client.1.smithi097.stdout:3/330: creat d7/de/d3c/d3f/f6d x:0 0 0 2024-09-16T08:57:09.263 INFO:tasks.workunit.client.1.smithi097.stdout:3/331: dread - d7/de/d29/d35/d3d/f59 zero size 2024-09-16T08:57:09.263 INFO:tasks.workunit.client.1.smithi097.stdout:5/389: mkdir db/d18/d1c/d42/d46/d7e/d64/d7f 0 2024-09-16T08:57:09.268 INFO:tasks.workunit.client.0.smithi031.stdout:9/28: mknod d3/c7 0 2024-09-16T08:57:09.269 INFO:tasks.workunit.client.1.smithi097.stdout:1/375: rename d5/db/dc/d19/l55 to d5/d17/d54/d6b/l75 0 2024-09-16T08:57:09.275 INFO:tasks.workunit.client.1.smithi097.stdout:3/332: mknod d7/d15/c6e 0 2024-09-16T08:57:09.278 INFO:tasks.workunit.client.0.smithi031.stdout:6/74: creat d0/d10/f11 x:0 0 0 2024-09-16T08:57:09.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/333: creat d7/de/d6c/f6f x:0 0 0 2024-09-16T08:57:09.281 INFO:tasks.workunit.client.1.smithi097.stdout:3/334: truncate d7/de/d29/f2d 4977930 0 2024-09-16T08:57:09.282 INFO:tasks.workunit.client.0.smithi031.stdout:9/29: rmdir d3/d5 39 2024-09-16T08:57:09.285 INFO:tasks.workunit.client.1.smithi097.stdout:3/335: unlink d7/d28/f47 0 2024-09-16T08:57:09.287 INFO:tasks.workunit.client.1.smithi097.stdout:0/373: dwrite da/d24/d4c/d80/d42/d5b/d60/d73/f77 [0,4194304] 0 2024-09-16T08:57:09.287 INFO:tasks.workunit.client.1.smithi097.stdout:0/374: read da/d1a/d41/f50 [252877,81155] 0 2024-09-16T08:57:09.287 INFO:tasks.workunit.client.1.smithi097.stdout:0/375: dread - da/d24/d4c/d80/f62 zero size 2024-09-16T08:57:09.288 INFO:tasks.workunit.client.1.smithi097.stdout:3/336: mkdir d7/d28/d63/d70 0 2024-09-16T08:57:09.288 INFO:tasks.workunit.client.1.smithi097.stdout:3/337: read d7/de/d6c/f66 [4625625,4041] 0 2024-09-16T08:57:09.288 INFO:tasks.workunit.client.1.smithi097.stdout:3/338: write d7/de/d6c/f4a [859571,126913] 0 2024-09-16T08:57:09.289 INFO:tasks.workunit.client.1.smithi097.stdout:3/339: write d7/d28/d30/f53 [13097,53634] 0 2024-09-16T08:57:09.305 INFO:tasks.workunit.client.0.smithi031.stdout:6/75: rmdir d0 39 2024-09-16T08:57:09.305 INFO:tasks.workunit.client.1.smithi097.stdout:0/376: dread da/d24/f46 [0,4194304] 0 2024-09-16T08:57:09.311 INFO:tasks.workunit.client.0.smithi031.stdout:6/76: dread d0/ff [0,4194304] 0 2024-09-16T08:57:09.311 INFO:tasks.workunit.client.0.smithi031.stdout:6/77: write d0/fb [5210066,116365] 0 2024-09-16T08:57:09.312 INFO:tasks.workunit.client.0.smithi031.stdout:6/78: truncate d0/d10/f11 302878 0 2024-09-16T08:57:09.312 INFO:tasks.workunit.client.0.smithi031.stdout:6/79: rename d0/dd to d0/dd/d12 22 2024-09-16T08:57:09.312 INFO:tasks.workunit.client.0.smithi031.stdout:6/80: readlink d0/la 0 2024-09-16T08:57:09.313 INFO:tasks.workunit.client.1.smithi097.stdout:9/336: sync 2024-09-16T08:57:09.320 INFO:tasks.workunit.client.0.smithi031.stdout:0/71: dwrite d6/fa [0,4194304] 0 2024-09-16T08:57:09.320 INFO:tasks.workunit.client.1.smithi097.stdout:2/353: dwrite d3/f40 [0,4194304] 0 2024-09-16T08:57:09.320 INFO:tasks.workunit.client.1.smithi097.stdout:2/354: fsync d3/f3a 0 2024-09-16T08:57:09.321 INFO:tasks.workunit.client.1.smithi097.stdout:2/355: stat d3/d4/d8 0 2024-09-16T08:57:09.330 INFO:tasks.workunit.client.0.smithi031.stdout:1/90: dread d2/d4/f12 [0,4194304] 0 2024-09-16T08:57:09.330 INFO:tasks.workunit.client.0.smithi031.stdout:1/91: chown d2/d4 106182148 1 2024-09-16T08:57:09.330 INFO:tasks.workunit.client.0.smithi031.stdout:1/92: write d2/de/f13 [347007,15222] 0 2024-09-16T08:57:09.332 INFO:tasks.workunit.client.0.smithi031.stdout:1/93: fdatasync d2/d4/f12 0 2024-09-16T08:57:09.332 INFO:tasks.workunit.client.1.smithi097.stdout:4/377: dwrite d0/d7/d52/f78 [0,4194304] 0 2024-09-16T08:57:09.337 INFO:tasks.workunit.client.1.smithi097.stdout:5/390: dwrite db/d22/d58/f7d [0,4194304] 0 2024-09-16T08:57:09.338 INFO:tasks.workunit.client.1.smithi097.stdout:1/376: dwrite d5/d2c/f4b [0,4194304] 0 2024-09-16T08:57:09.338 INFO:tasks.workunit.client.0.smithi031.stdout:0/72: dwrite d6/d12/f17 [0,4194304] 0 2024-09-16T08:57:09.338 INFO:tasks.workunit.client.0.smithi031.stdout:8/51: dwrite d4/f6 [4194304,4194304] 0 2024-09-16T08:57:09.339 INFO:tasks.workunit.client.1.smithi097.stdout:3/340: dwrite d7/de/d3c/d3f/f5e [0,4194304] 0 2024-09-16T08:57:09.339 INFO:tasks.workunit.client.1.smithi097.stdout:3/341: chown d7/f2b 4 1 2024-09-16T08:57:09.339 INFO:tasks.workunit.client.1.smithi097.stdout:3/342: chown d7/de/d29/c48 3245 1 2024-09-16T08:57:09.339 INFO:tasks.workunit.client.1.smithi097.stdout:3/343: write d7/f2b [852987,100309] 0 2024-09-16T08:57:09.339 INFO:tasks.workunit.client.1.smithi097.stdout:0/377: dwrite da/d24/d4c/d80/d3b/f7f [0,4194304] 0 2024-09-16T08:57:09.339 INFO:tasks.workunit.client.1.smithi097.stdout:3/344: truncate d7/de/d29/f49 1821838 0 2024-09-16T08:57:09.339 INFO:tasks.workunit.client.1.smithi097.stdout:9/337: dwrite d2/f50 [0,4194304] 0 2024-09-16T08:57:09.341 INFO:tasks.workunit.client.1.smithi097.stdout:2/356: write d3/d4/d9/f4f [2578634,24484] 0 2024-09-16T08:57:09.341 INFO:tasks.workunit.client.1.smithi097.stdout:4/378: unlink d0/f2f 0 2024-09-16T08:57:09.341 INFO:tasks.workunit.client.1.smithi097.stdout:4/379: truncate d0/d39/d71/f72 711419 0 2024-09-16T08:57:09.341 INFO:tasks.workunit.client.1.smithi097.stdout:4/380: write d0/d39/d71/f79 [897271,53085] 0 2024-09-16T08:57:09.341 INFO:tasks.workunit.client.1.smithi097.stdout:4/381: chown d0/d39/f6a 3366454 1 2024-09-16T08:57:09.342 INFO:tasks.workunit.client.1.smithi097.stdout:2/357: dread d3/d4/d8/f33 [0,4194304] 0 2024-09-16T08:57:09.342 INFO:tasks.workunit.client.1.smithi097.stdout:2/358: chown d3/d4/d9/f4c 1886319 1 2024-09-16T08:57:09.342 INFO:tasks.workunit.client.1.smithi097.stdout:2/359: creat d3/d4/d9/f5a x:0 0 0 2024-09-16T08:57:09.349 INFO:tasks.workunit.client.1.smithi097.stdout:5/391: read db/d1a/f2c [4168541,125109] 0 2024-09-16T08:57:09.350 INFO:tasks.workunit.client.0.smithi031.stdout:0/73: mknod d6/d12/d13/c1b 0 2024-09-16T08:57:09.355 INFO:tasks.workunit.client.0.smithi031.stdout:8/52: rmdir d4 39 2024-09-16T08:57:09.358 INFO:tasks.workunit.client.1.smithi097.stdout:1/377: creat d5/d17/d54/d6b/d71/f76 x:0 0 0 2024-09-16T08:57:09.365 INFO:tasks.workunit.client.1.smithi097.stdout:0/378: unlink da/c12 0 2024-09-16T08:57:09.366 INFO:tasks.workunit.client.1.smithi097.stdout:3/345: link d7/de/d29/c3a d7/de/d29/d35/d3d/c71 0 2024-09-16T08:57:09.366 INFO:tasks.workunit.client.1.smithi097.stdout:3/346: write d7/de/d3c/d3f/f6d [67814,18727] 0 2024-09-16T08:57:09.367 INFO:tasks.workunit.client.0.smithi031.stdout:8/53: dread d4/f6 [0,4194304] 0 2024-09-16T08:57:09.367 INFO:tasks.workunit.client.1.smithi097.stdout:0/379: dread da/d1a/f22 [0,4194304] 0 2024-09-16T08:57:09.368 INFO:tasks.workunit.client.1.smithi097.stdout:4/382: link d0/f64 d0/d7/f89 0 2024-09-16T08:57:09.370 INFO:tasks.workunit.client.0.smithi031.stdout:8/54: creat d4/d7/fd x:0 0 0 2024-09-16T08:57:09.370 INFO:tasks.workunit.client.1.smithi097.stdout:2/360: symlink d3/d4/d9/l5b 0 2024-09-16T08:57:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:08 smithi031 ceph-mon[27844]: pgmap v6: 65 pgs: 65 active+clean; 355 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:57:09.373 INFO:tasks.workunit.client.0.smithi031.stdout:6/81: dwrite d0/f4 [4194304,4194304] 0 2024-09-16T08:57:09.374 INFO:tasks.workunit.client.0.smithi031.stdout:6/82: readlink d0/la 0 2024-09-16T08:57:09.374 INFO:tasks.workunit.client.0.smithi031.stdout:1/94: dwrite d2/f14 [0,4194304] 0 2024-09-16T08:57:09.374 INFO:tasks.workunit.client.0.smithi031.stdout:1/95: chown d2 27331746 1 2024-09-16T08:57:09.375 INFO:tasks.workunit.client.0.smithi031.stdout:0/74: dwrite d6/d12/f17 [0,4194304] 0 2024-09-16T08:57:09.378 INFO:tasks.workunit.client.1.smithi097.stdout:5/392: getdents db/d18/d1c 0 2024-09-16T08:57:09.378 INFO:tasks.workunit.client.1.smithi097.stdout:1/378: creat d5/f77 x:0 0 0 2024-09-16T08:57:09.378 INFO:tasks.workunit.client.1.smithi097.stdout:1/379: chown d5/d14/c44 6690 1 2024-09-16T08:57:09.379 INFO:tasks.workunit.client.1.smithi097.stdout:1/380: dread - d5/db/f6f zero size 2024-09-16T08:57:09.380 INFO:tasks.workunit.client.0.smithi031.stdout:1/96: write d2/f14 [6090487,99053] 0 2024-09-16T08:57:09.380 INFO:tasks.workunit.client.1.smithi097.stdout:3/347: symlink d7/d28/d30/l72 0 2024-09-16T08:57:09.380 INFO:tasks.workunit.client.1.smithi097.stdout:3/348: creat d7/de/d3c/f73 x:0 0 0 2024-09-16T08:57:09.382 INFO:tasks.workunit.client.0.smithi031.stdout:6/83: dread d0/ff [0,4194304] 0 2024-09-16T08:57:09.382 INFO:tasks.workunit.client.0.smithi031.stdout:6/84: dread - d0/f8 zero size 2024-09-16T08:57:09.382 INFO:tasks.workunit.client.0.smithi031.stdout:6/85: fdatasync d0/fc 0 2024-09-16T08:57:09.383 INFO:tasks.workunit.client.1.smithi097.stdout:0/380: mkdir da/d24/d4c/d80/d42/d5b/d60/d73/d85 0 2024-09-16T08:57:09.385 INFO:tasks.workunit.client.1.smithi097.stdout:2/361: rmdir d3/d4/d8/d34/d48 39 2024-09-16T08:57:09.389 INFO:tasks.workunit.client.0.smithi031.stdout:6/86: creat d0/d10/f13 x:0 0 0 2024-09-16T08:57:09.393 INFO:tasks.workunit.client.1.smithi097.stdout:1/381: link d5/d2c/f4e d5/db/dc/f78 0 2024-09-16T08:57:09.393 INFO:tasks.workunit.client.1.smithi097.stdout:4/383: dwrite d0/d7/f24 [0,4194304] 0 2024-09-16T08:57:09.393 INFO:tasks.workunit.client.0.smithi031.stdout:0/75: write d6/fa [2457483,49294] 0 2024-09-16T08:57:09.393 INFO:tasks.workunit.client.0.smithi031.stdout:8/55: dwrite d4/f8 [12582912,4194304] 0 2024-09-16T08:57:09.394 INFO:tasks.workunit.client.0.smithi031.stdout:8/56: dread - d4/d7/fd zero size 2024-09-16T08:57:09.394 INFO:tasks.workunit.client.0.smithi031.stdout:8/57: read - d4/fa zero size 2024-09-16T08:57:09.394 INFO:tasks.workunit.client.0.smithi031.stdout:8/58: read - d4/fa zero size 2024-09-16T08:57:09.394 INFO:tasks.workunit.client.0.smithi031.stdout:8/59: write d4/d7/fd [347638,95781] 0 2024-09-16T08:57:09.394 INFO:tasks.workunit.client.0.smithi031.stdout:1/97: mkdir d2/d4/d1b 0 2024-09-16T08:57:09.400 INFO:tasks.workunit.client.1.smithi097.stdout:3/349: mkdir d7/de/d29/d35/d3d/d74 0 2024-09-16T08:57:09.400 INFO:tasks.workunit.client.1.smithi097.stdout:3/350: fsync d7/de/d29/d35/f27 0 2024-09-16T08:57:09.401 INFO:tasks.workunit.client.0.smithi031.stdout:8/60: mknod d4/d7/ce 0 2024-09-16T08:57:09.402 INFO:tasks.workunit.client.1.smithi097.stdout:0/381: mkdir da/d24/d4c/d80/d3b/d86 0 2024-09-16T08:57:09.404 INFO:tasks.workunit.client.0.smithi031.stdout:1/98: creat d2/d4/f1c x:0 0 0 2024-09-16T08:57:09.404 INFO:tasks.workunit.client.0.smithi031.stdout:6/87: dread d0/fb [0,4194304] 0 2024-09-16T08:57:09.405 INFO:tasks.workunit.client.1.smithi097.stdout:2/362: rename d3/d4/l4a to d3/d4/dd/d53/l5c 0 2024-09-16T08:57:09.406 INFO:tasks.workunit.client.1.smithi097.stdout:2/363: stat d3/d4/d8/d51 0 2024-09-16T08:57:09.407 INFO:tasks.workunit.client.0.smithi031.stdout:8/61: mknod d4/d7/cf 0 2024-09-16T08:57:09.407 INFO:tasks.workunit.client.0.smithi031.stdout:8/62: readlink - no filename 2024-09-16T08:57:09.407 INFO:tasks.workunit.client.0.smithi031.stdout:8/63: write d4/d7/fd [1050767,93213] 0 2024-09-16T08:57:09.408 INFO:tasks.workunit.client.1.smithi097.stdout:1/382: mknod d5/d17/d54/c79 0 2024-09-16T08:57:09.408 INFO:tasks.workunit.client.1.smithi097.stdout:5/393: dwrite db/d18/d1c/d42/d46/d7e/f51 [0,4194304] 0 2024-09-16T08:57:09.408 INFO:tasks.workunit.client.1.smithi097.stdout:5/394: chown db/d18/d1c/d42/d46/f47 2428141 1 2024-09-16T08:57:09.409 INFO:tasks.workunit.client.0.smithi031.stdout:0/76: dread d6/f8 [0,4194304] 0 2024-09-16T08:57:09.409 INFO:tasks.workunit.client.0.smithi031.stdout:1/99: truncate d2/de/f13 556924 0 2024-09-16T08:57:09.413 INFO:tasks.workunit.client.1.smithi097.stdout:4/384: unlink d0/d7/df/f1e 0 2024-09-16T08:57:09.415 INFO:tasks.workunit.client.0.smithi031.stdout:8/64: rmdir d4 39 2024-09-16T08:57:09.416 INFO:tasks.workunit.client.1.smithi097.stdout:3/351: mkdir d7/de/d29/d35/d3d/d57/d75 0 2024-09-16T08:57:09.418 INFO:tasks.workunit.client.1.smithi097.stdout:0/382: rename da/d24/c2c to da/d24/d4c/d80/d42/d5b/d60/c87 0 2024-09-16T08:57:09.418 INFO:tasks.workunit.client.1.smithi097.stdout:0/383: dread - da/d24/d4c/d6d/f6e zero size 2024-09-16T08:57:09.419 INFO:tasks.workunit.client.0.smithi031.stdout:0/77: symlink d6/d12/l1c 0 2024-09-16T08:57:09.419 INFO:tasks.workunit.client.0.smithi031.stdout:0/78: readlink d6/lf 0 2024-09-16T08:57:09.420 INFO:tasks.workunit.client.1.smithi097.stdout:2/364: unlink d3/d4/c4e 0 2024-09-16T08:57:09.421 INFO:tasks.workunit.client.0.smithi031.stdout:1/100: unlink d2/f3 0 2024-09-16T08:57:09.422 INFO:tasks.workunit.client.1.smithi097.stdout:1/383: symlink d5/d14/l7a 0 2024-09-16T08:57:09.423 INFO:tasks.workunit.client.1.smithi097.stdout:1/384: chown d5/db/d25/f45 712561 1 2024-09-16T08:57:09.423 INFO:tasks.workunit.client.1.smithi097.stdout:1/385: creat d5/d2c/d3b/f7b x:0 0 0 2024-09-16T08:57:09.426 INFO:tasks.workunit.client.0.smithi031.stdout:8/65: creat d4/f10 x:0 0 0 2024-09-16T08:57:09.428 INFO:tasks.workunit.client.0.smithi031.stdout:0/79: mknod d6/d12/c1d 0 2024-09-16T08:57:09.428 INFO:tasks.workunit.client.0.smithi031.stdout:0/80: stat d6/d12/d13/c1b 0 2024-09-16T08:57:09.430 INFO:tasks.workunit.client.0.smithi031.stdout:8/66: rename d4/d7/ce to d4/d7/c11 0 2024-09-16T08:57:09.431 INFO:tasks.workunit.client.1.smithi097.stdout:5/395: unlink db/d1a/f41 0 2024-09-16T08:57:09.433 INFO:tasks.workunit.client.0.smithi031.stdout:0/81: creat d6/f1e x:0 0 0 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:7/371: sync 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:8/345: sync 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:8/346: write d8/dd/d15/d2c/d4a/d4d/f49 [2437845,58032] 0 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:8/347: readlink d8/dd/d15/d16/l53 0 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:8/348: write d8/dd/d15/d16/f3a [1614758,29863] 0 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:8/349: readlink d8/dd/l1a 0 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:4/385: mknod d0/d36/d4c/c8a 0 2024-09-16T08:57:09.434 INFO:tasks.workunit.client.1.smithi097.stdout:4/386: readlink d0/d7/df/l22 0 2024-09-16T08:57:09.435 INFO:tasks.workunit.client.0.smithi031.stdout:8/67: symlink d4/l12 0 2024-09-16T08:57:09.437 INFO:tasks.workunit.client.1.smithi097.stdout:3/352: creat d7/de/d29/f76 x:0 0 0 2024-09-16T08:57:09.437 INFO:tasks.workunit.client.1.smithi097.stdout:3/353: stat d7/de/d3c/f45 0 2024-09-16T08:57:09.438 INFO:tasks.workunit.client.1.smithi097.stdout:3/354: rename d7/de/d29/d35/d3d to d7/de/d29/d35/d3d/d57/d64/d77 22 2024-09-16T08:57:09.439 INFO:tasks.workunit.client.1.smithi097.stdout:0/384: mknod da/d1a/d41/c88 0 2024-09-16T08:57:09.439 INFO:tasks.workunit.client.1.smithi097.stdout:0/385: truncate da/d24/d4c/d6d/f6e 398820 0 2024-09-16T08:57:09.439 INFO:tasks.workunit.client.1.smithi097.stdout:0/386: truncate da/d24/d4c/d80/d42/d5b/d60/f65 127119 0 2024-09-16T08:57:09.439 INFO:tasks.workunit.client.1.smithi097.stdout:8/350: dread d8/dd/d15/d2c/d4a/d4e/f5a [0,4194304] 0 2024-09-16T08:57:09.443 INFO:tasks.workunit.client.1.smithi097.stdout:4/387: write d0/f30 [105742,101129] 0 2024-09-16T08:57:09.443 INFO:tasks.workunit.client.0.smithi031.stdout:8/68: symlink d4/l13 0 2024-09-16T08:57:09.443 INFO:tasks.workunit.client.0.smithi031.stdout:8/69: creat d4/f14 x:0 0 0 2024-09-16T08:57:09.443 INFO:tasks.workunit.client.0.smithi031.stdout:8/70: stat d4/f8 0 2024-09-16T08:57:09.445 INFO:tasks.workunit.client.0.smithi031.stdout:0/82: mknod d6/d12/d13/c1f 0 2024-09-16T08:57:09.447 INFO:tasks.workunit.client.1.smithi097.stdout:2/365: dwrite d3/f37 [0,4194304] 0 2024-09-16T08:57:09.447 INFO:tasks.workunit.client.1.smithi097.stdout:6/378: sync 2024-09-16T08:57:09.447 INFO:tasks.workunit.client.1.smithi097.stdout:9/338: sync 2024-09-16T08:57:09.448 INFO:tasks.workunit.client.0.smithi031.stdout:1/101: dwrite d2/d4/f12 [0,4194304] 0 2024-09-16T08:57:09.453 INFO:tasks.workunit.client.0.smithi031.stdout:8/71: rename d4/d7/fd to d4/d7/f15 0 2024-09-16T08:57:09.456 INFO:tasks.workunit.client.1.smithi097.stdout:3/355: truncate d7/de/d29/d35/f22 758037 0 2024-09-16T08:57:09.457 INFO:tasks.workunit.client.1.smithi097.stdout:0/387: symlink da/d24/d4c/d80/d3b/l89 0 2024-09-16T08:57:09.457 INFO:tasks.workunit.client.1.smithi097.stdout:0/388: stat da/df/d3d/l67 0 2024-09-16T08:57:09.458 INFO:tasks.workunit.client.0.smithi031.stdout:1/102: truncate d2/d4/f5 2589931 0 2024-09-16T08:57:09.463 INFO:tasks.workunit.client.1.smithi097.stdout:8/351: rename d8/dd/d15/d2c/d4a/d4d/d3d/l63 to d8/dd/l66 0 2024-09-16T08:57:09.463 INFO:tasks.workunit.client.1.smithi097.stdout:8/352: write d8/dd/d15/f34 [717936,87730] 0 2024-09-16T08:57:09.464 INFO:tasks.workunit.client.0.smithi031.stdout:1/103: symlink d2/l1d 0 2024-09-16T08:57:09.465 INFO:tasks.workunit.client.1.smithi097.stdout:1/386: getdents d5/d17 0 2024-09-16T08:57:09.466 INFO:tasks.workunit.client.0.smithi031.stdout:8/72: dread d4/f6 [4194304,4194304] 0 2024-09-16T08:57:09.466 INFO:tasks.workunit.client.0.smithi031.stdout:1/104: mkdir d2/d15/d1e 0 2024-09-16T08:57:09.466 INFO:tasks.workunit.client.0.smithi031.stdout:8/73: symlink d4/l16 0 2024-09-16T08:57:09.466 INFO:tasks.workunit.client.0.smithi031.stdout:8/74: fdatasync d4/f8 0 2024-09-16T08:57:09.468 INFO:tasks.workunit.client.1.smithi097.stdout:5/396: dwrite db/fe [8388608,4194304] 0 2024-09-16T08:57:09.468 INFO:tasks.workunit.client.1.smithi097.stdout:2/366: link d3/d4/d9/f4f d3/d4/d8/d51/f5d 0 2024-09-16T08:57:09.469 INFO:tasks.workunit.client.1.smithi097.stdout:7/372: dwrite d4/d9/d1c/f2c [0,4194304] 0 2024-09-16T08:57:09.469 INFO:tasks.workunit.client.1.smithi097.stdout:7/373: chown d4/d1b/f69 1309996750 1 2024-09-16T08:57:09.469 INFO:tasks.workunit.client.1.smithi097.stdout:7/374: truncate d4/d7/d7b/f7c 943155 0 2024-09-16T08:57:09.469 INFO:tasks.workunit.client.1.smithi097.stdout:4/388: dread d0/d7/d52/f67 [0,4194304] 0 2024-09-16T08:57:09.470 INFO:tasks.workunit.client.1.smithi097.stdout:8/353: fsync d8/dd/d15/d2c/d4a/d4d/f49 0 2024-09-16T08:57:09.470 INFO:tasks.workunit.client.0.smithi031.stdout:0/83: dwrite d6/f1e [0,4194304] 0 2024-09-16T08:57:09.470 INFO:tasks.workunit.client.0.smithi031.stdout:8/75: rename d4/f10 to d4/f17 0 2024-09-16T08:57:09.470 INFO:tasks.workunit.client.0.smithi031.stdout:8/76: truncate d4/f14 730068 0 2024-09-16T08:57:09.471 INFO:tasks.workunit.client.1.smithi097.stdout:9/339: truncate d2/d5/f35 994018 0 2024-09-16T08:57:09.474 INFO:tasks.workunit.client.0.smithi031.stdout:1/105: write d2/d4/f12 [684168,105420] 0 2024-09-16T08:57:09.475 INFO:tasks.workunit.client.1.smithi097.stdout:6/379: link d1/d3/d3e/l43 d1/dc/l73 0 2024-09-16T08:57:09.477 INFO:tasks.workunit.client.1.smithi097.stdout:3/356: creat d7/de/d29/d35/d3d/d57/d64/f78 x:0 0 0 2024-09-16T08:57:09.477 INFO:tasks.workunit.client.1.smithi097.stdout:3/357: creat d7/de/d3c/f79 x:0 0 0 2024-09-16T08:57:09.478 INFO:tasks.workunit.client.0.smithi031.stdout:0/84: rename d6/d12/d13 to d6/d12/d13/d20 22 2024-09-16T08:57:09.479 INFO:tasks.workunit.client.1.smithi097.stdout:0/389: mkdir da/df/d8a 0 2024-09-16T08:57:09.479 INFO:tasks.workunit.client.1.smithi097.stdout:0/390: write da/df/d3d/f66 [622600,7539] 0 2024-09-16T08:57:09.479 INFO:tasks.workunit.client.1.smithi097.stdout:0/391: read da/d1a/d26/f59 [210805,50447] 0 2024-09-16T08:57:09.483 INFO:tasks.workunit.client.1.smithi097.stdout:5/397: creat db/d18/d1c/d42/d46/d7e/f80 x:0 0 0 2024-09-16T08:57:09.485 INFO:tasks.workunit.client.0.smithi031.stdout:1/106: mkdir d2/d15/d1f 0 2024-09-16T08:57:09.486 INFO:tasks.workunit.client.1.smithi097.stdout:2/367: rename d3/f37 to d3/d4/dd/f5e 0 2024-09-16T08:57:09.489 INFO:tasks.workunit.client.1.smithi097.stdout:7/375: link d4/d1b/d41/f52 d4/d1b/d41/f81 0 2024-09-16T08:57:09.490 INFO:tasks.workunit.client.1.smithi097.stdout:7/376: fsync d4/d9/d2b/f7e 0 2024-09-16T08:57:09.490 INFO:tasks.workunit.client.0.smithi031.stdout:1/107: symlink d2/de/l20 0 2024-09-16T08:57:09.491 INFO:tasks.workunit.client.0.smithi031.stdout:1/108: fdatasync f0 0 2024-09-16T08:57:09.491 INFO:tasks.workunit.client.0.smithi031.stdout:1/109: chown d2/d4/f5 20420419 1 2024-09-16T08:57:09.492 INFO:tasks.workunit.client.1.smithi097.stdout:4/389: symlink d0/d23/l8b 0 2024-09-16T08:57:09.493 INFO:tasks.workunit.client.1.smithi097.stdout:1/387: dread d5/db/f20 [0,4194304] 0 2024-09-16T08:57:09.493 INFO:tasks.workunit.client.1.smithi097.stdout:1/388: chown d5/d17/d54/d6b/l75 359 1 2024-09-16T08:57:09.493 INFO:tasks.workunit.client.0.smithi031.stdout:0/85: dwrite d6/f11 [0,4194304] 0 2024-09-16T08:57:09.494 INFO:tasks.workunit.client.1.smithi097.stdout:9/340: symlink d2/d30/l6a 0 2024-09-16T08:57:09.495 INFO:tasks.workunit.client.0.smithi031.stdout:1/110: mknod d2/d15/d1f/c21 0 2024-09-16T08:57:09.499 INFO:tasks.workunit.client.1.smithi097.stdout:6/380: mknod d1/d3/d16/c74 0 2024-09-16T08:57:09.499 INFO:tasks.workunit.client.1.smithi097.stdout:6/381: truncate d1/d3/d16/d4f/f55 4650844 0 2024-09-16T08:57:09.502 INFO:tasks.workunit.client.0.smithi031.stdout:0/86: rmdir d6/d12 39 2024-09-16T08:57:09.504 INFO:tasks.workunit.client.1.smithi097.stdout:3/358: mkdir d7/de/d3c/d3f/d60/d7a 0 2024-09-16T08:57:09.505 INFO:tasks.workunit.client.0.smithi031.stdout:1/111: mkdir d2/d4/d22 0 2024-09-16T08:57:09.506 INFO:tasks.workunit.client.1.smithi097.stdout:0/392: creat da/d24/d76/f8b x:0 0 0 2024-09-16T08:57:09.506 INFO:tasks.workunit.client.1.smithi097.stdout:9/341: dread d2/d5/f11 [0,4194304] 0 2024-09-16T08:57:09.506 INFO:tasks.workunit.client.1.smithi097.stdout:2/368: dwrite d3/d4/dd/f35 [0,4194304] 0 2024-09-16T08:57:09.507 INFO:tasks.workunit.client.1.smithi097.stdout:5/398: mkdir db/d6e/d81 0 2024-09-16T08:57:09.507 INFO:tasks.workunit.client.1.smithi097.stdout:0/393: dread da/d24/f46 [0,4194304] 0 2024-09-16T08:57:09.507 INFO:tasks.workunit.client.1.smithi097.stdout:0/394: chown da/d24/d4c/d80/d42/d5b/l7a 23260388 1 2024-09-16T08:57:09.509 INFO:tasks.workunit.client.0.smithi031.stdout:1/112: creat d2/d4/d22/f23 x:0 0 0 2024-09-16T08:57:09.511 INFO:tasks.workunit.client.1.smithi097.stdout:6/382: read d1/d3/f4b [2734477,78044] 0 2024-09-16T08:57:09.515 INFO:tasks.workunit.client.1.smithi097.stdout:3/359: dread d7/d28/f2f [0,4194304] 0 2024-09-16T08:57:09.515 INFO:tasks.workunit.client.1.smithi097.stdout:7/377: stat d4/d1b/l26 0 2024-09-16T08:57:09.518 INFO:tasks.workunit.client.1.smithi097.stdout:9/342: mkdir d2/d5/d56/d6b 0 2024-09-16T08:57:09.519 INFO:tasks.workunit.client.1.smithi097.stdout:2/369: creat d3/d1b/f5f x:0 0 0 2024-09-16T08:57:09.520 INFO:tasks.workunit.client.1.smithi097.stdout:5/399: unlink db/l4a 0 2024-09-16T08:57:09.521 INFO:tasks.workunit.client.1.smithi097.stdout:0/395: mknod da/d24/d4c/d80/d42/d5b/d60/d68/c8c 0 2024-09-16T08:57:09.521 INFO:tasks.workunit.client.1.smithi097.stdout:6/383: rmdir d1/d3/d16/d6a 39 2024-09-16T08:57:09.522 INFO:tasks.workunit.client.1.smithi097.stdout:6/384: write d1/f64 [4575293,95619] 0 2024-09-16T08:57:09.525 INFO:tasks.workunit.client.0.smithi031.stdout:0/87: dwrite d6/f1a [0,4194304] 0 2024-09-16T08:57:09.526 INFO:tasks.workunit.client.1.smithi097.stdout:4/390: mkdir d0/d7/d52/d26/d8c 0 2024-09-16T08:57:09.526 INFO:tasks.workunit.client.1.smithi097.stdout:1/389: rename d5/d17/f6c to d5/db/f7c 0 2024-09-16T08:57:09.529 INFO:tasks.workunit.client.1.smithi097.stdout:2/370: dread d3/f40 [0,4194304] 0 2024-09-16T08:57:09.529 INFO:tasks.workunit.client.1.smithi097.stdout:2/371: chown d3/d4/d8/d34 0 1 2024-09-16T08:57:09.529 INFO:tasks.workunit.client.1.smithi097.stdout:3/360: dread d7/d15/f17 [0,4194304] 0 2024-09-16T08:57:09.530 INFO:tasks.workunit.client.0.smithi031.stdout:0/88: dread d6/d12/f17 [0,4194304] 0 2024-09-16T08:57:09.530 INFO:tasks.workunit.client.1.smithi097.stdout:7/378: getdents d4/d1b 0 2024-09-16T08:57:09.533 INFO:tasks.workunit.client.1.smithi097.stdout:4/391: unlink d0/d7/f24 0 2024-09-16T08:57:09.533 INFO:tasks.workunit.client.1.smithi097.stdout:4/392: read - d0/d23/d5f/d7d/f85 zero size 2024-09-16T08:57:09.535 INFO:tasks.workunit.client.1.smithi097.stdout:1/390: link d5/db/dc/d19/f29 d5/db/dc/f7d 0 2024-09-16T08:57:09.537 INFO:tasks.workunit.client.1.smithi097.stdout:9/343: mknod d2/d5/c6c 0 2024-09-16T08:57:09.539 INFO:tasks.workunit.client.1.smithi097.stdout:3/361: truncate d7/de/d29/f2d 966634 0 2024-09-16T08:57:09.539 INFO:tasks.workunit.client.1.smithi097.stdout:3/362: chown d7/de/d29/d35/f27 14 1 2024-09-16T08:57:09.540 INFO:tasks.workunit.client.0.smithi031.stdout:0/89: dread d6/f1e [0,4194304] 0 2024-09-16T08:57:09.540 INFO:tasks.workunit.client.0.smithi031.stdout:0/90: chown d6/f8 49 1 2024-09-16T08:57:09.540 INFO:tasks.workunit.client.1.smithi097.stdout:4/393: creat d0/d7/d52/f8d x:0 0 0 2024-09-16T08:57:09.541 INFO:tasks.workunit.client.1.smithi097.stdout:4/394: write d0/d36/f51 [114977,65309] 0 2024-09-16T08:57:09.542 INFO:tasks.workunit.client.1.smithi097.stdout:9/344: creat d2/d5/d3a/d3c/f6d x:0 0 0 2024-09-16T08:57:09.542 INFO:tasks.workunit.client.1.smithi097.stdout:9/345: write d2/d5/d56/f5d [221603,128092] 0 2024-09-16T08:57:09.544 INFO:tasks.workunit.client.0.smithi031.stdout:0/91: creat d6/d12/d13/f21 x:0 0 0 2024-09-16T08:57:09.545 INFO:tasks.workunit.client.1.smithi097.stdout:4/395: mknod d0/d36/c8e 0 2024-09-16T08:57:09.545 INFO:tasks.workunit.client.1.smithi097.stdout:3/363: write d7/de/d3c/d3f/f51 [2234319,45213] 0 2024-09-16T08:57:09.547 INFO:tasks.workunit.client.1.smithi097.stdout:4/396: truncate d0/f2a 1309571 0 2024-09-16T08:57:09.549 INFO:tasks.workunit.client.1.smithi097.stdout:9/346: dread d2/d3f/f43 [0,4194304] 0 2024-09-16T08:57:09.550 INFO:tasks.workunit.client.1.smithi097.stdout:9/347: fsync d2/f46 0 2024-09-16T08:57:09.550 INFO:tasks.workunit.client.1.smithi097.stdout:9/348: chown d2/d5/d3a/d3c/f53 1172 1 2024-09-16T08:57:09.550 INFO:tasks.workunit.client.1.smithi097.stdout:9/349: read d2/d5/d3a/d3c/f5f [1095265,97945] 0 2024-09-16T08:57:09.551 INFO:tasks.workunit.client.1.smithi097.stdout:9/350: mkdir d2/d5/d6e 0 2024-09-16T08:57:09.554 INFO:tasks.workunit.client.1.smithi097.stdout:1/391: write d5/db/f74 [3100831,5566] 0 2024-09-16T08:57:09.556 INFO:tasks.workunit.client.1.smithi097.stdout:1/392: truncate d5/db/f3c 1006256 0 2024-09-16T08:57:09.559 INFO:tasks.workunit.client.1.smithi097.stdout:1/393: rename d5/d2c/f6a to d5/f7e 0 2024-09-16T08:57:09.560 INFO:tasks.workunit.client.0.smithi031.stdout:0/92: write d6/f1a [1544632,14207] 0 2024-09-16T08:57:09.562 INFO:tasks.workunit.client.0.smithi031.stdout:0/93: chown d6/c10 234 1 2024-09-16T08:57:09.563 INFO:tasks.workunit.client.0.smithi031.stdout:0/94: creat d6/d12/f22 x:0 0 0 2024-09-16T08:57:09.566 INFO:tasks.workunit.client.1.smithi097.stdout:7/379: dwrite d4/d9/dc/f72 [0,4194304] 0 2024-09-16T08:57:09.566 INFO:tasks.workunit.client.1.smithi097.stdout:7/380: truncate d4/f64 217528 0 2024-09-16T08:57:09.573 INFO:tasks.workunit.client.0.smithi031.stdout:0/95: rmdir d6/d12/d13 39 2024-09-16T08:57:09.575 INFO:tasks.workunit.client.1.smithi097.stdout:0/396: dwrite da/df/f16 [4194304,4194304] 0 2024-09-16T08:57:09.577 INFO:tasks.workunit.client.1.smithi097.stdout:6/385: dwrite d1/d3/d16/f2e [0,4194304] 0 2024-09-16T08:57:09.577 INFO:tasks.workunit.client.1.smithi097.stdout:5/400: dwrite db/d1a/f1f [0,4194304] 0 2024-09-16T08:57:09.578 INFO:tasks.workunit.client.0.smithi031.stdout:0/96: creat d6/d12/d13/f23 x:0 0 0 2024-09-16T08:57:09.578 INFO:tasks.workunit.client.1.smithi097.stdout:3/364: dwrite d7/d28/d30/f53 [0,4194304] 0 2024-09-16T08:57:09.578 INFO:tasks.workunit.client.1.smithi097.stdout:4/397: dwrite d0/d7/df/d15/d1f/f81 [0,4194304] 0 2024-09-16T08:57:09.579 INFO:tasks.workunit.client.1.smithi097.stdout:0/397: creat da/d24/d4c/d80/d42/d5b/d60/d73/d85/f8d x:0 0 0 2024-09-16T08:57:09.580 INFO:tasks.workunit.client.1.smithi097.stdout:0/398: creat da/d1a/f8e x:0 0 0 2024-09-16T08:57:09.580 INFO:tasks.workunit.client.0.smithi031.stdout:0/97: creat d6/d12/d13/f24 x:0 0 0 2024-09-16T08:57:09.580 INFO:tasks.workunit.client.0.smithi031.stdout:0/98: truncate d6/d12/d13/f23 689033 0 2024-09-16T08:57:09.581 INFO:tasks.workunit.client.0.smithi031.stdout:0/99: chown d6/d12/d13/c19 19 1 2024-09-16T08:57:09.586 INFO:tasks.workunit.client.0.smithi031.stdout:0/100: symlink d6/l25 0 2024-09-16T08:57:09.587 INFO:tasks.workunit.client.1.smithi097.stdout:5/401: read db/d18/d20/d2b/f5c [1495008,12549] 0 2024-09-16T08:57:09.587 INFO:tasks.workunit.client.1.smithi097.stdout:5/402: truncate db/d18/d20/d2b/f75 261639 0 2024-09-16T08:57:09.587 INFO:tasks.workunit.client.1.smithi097.stdout:5/403: chown db/d18/d1c/d42/d46/d7e 756 1 2024-09-16T08:57:09.587 INFO:tasks.workunit.client.1.smithi097.stdout:5/404: dread - db/d18/d1c/d42/d46/f66 zero size 2024-09-16T08:57:09.588 INFO:tasks.workunit.client.1.smithi097.stdout:6/386: truncate d1/d3/d3e/d5a/f4a 3820334 0 2024-09-16T08:57:09.590 INFO:tasks.workunit.client.1.smithi097.stdout:7/381: dwrite d4/d9/dc/f48 [0,4194304] 0 2024-09-16T08:57:09.592 INFO:tasks.workunit.client.1.smithi097.stdout:3/365: rename d7/de/d3c/d3f/f5b to d7/de/d3c/d3f/f7b 0 2024-09-16T08:57:09.594 INFO:tasks.workunit.client.1.smithi097.stdout:0/399: getdents da/d24/d4c/d80/d42/d5b 0 2024-09-16T08:57:09.596 INFO:tasks.workunit.client.1.smithi097.stdout:3/366: read d7/de/d29/d35/f22 [555171,42749] 0 2024-09-16T08:57:09.596 INFO:tasks.workunit.client.1.smithi097.stdout:5/405: rmdir db/d18/d20/d74 0 2024-09-16T08:57:09.596 INFO:tasks.workunit.client.1.smithi097.stdout:5/406: dread - db/d18/d1c/d42/d46/d7e/f80 zero size 2024-09-16T08:57:09.598 INFO:tasks.workunit.client.1.smithi097.stdout:7/382: creat d4/d7/d79/f82 x:0 0 0 2024-09-16T08:57:09.599 INFO:tasks.workunit.client.1.smithi097.stdout:6/387: dread d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:09.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/400: creat da/d1a/d26/d75/f8f x:0 0 0 2024-09-16T08:57:09.601 INFO:tasks.workunit.client.1.smithi097.stdout:3/367: creat d7/de/d3c/d4f/f7c x:0 0 0 2024-09-16T08:57:09.601 INFO:tasks.workunit.client.1.smithi097.stdout:3/368: truncate d7/de/d6c/f5a 410225 0 2024-09-16T08:57:09.602 INFO:tasks.workunit.client.1.smithi097.stdout:7/383: mknod d4/c83 0 2024-09-16T08:57:09.602 INFO:tasks.workunit.client.1.smithi097.stdout:5/407: read db/d18/f34 [114470,94741] 0 2024-09-16T08:57:09.603 INFO:tasks.workunit.client.1.smithi097.stdout:0/401: symlink da/d1a/d26/d75/l90 0 2024-09-16T08:57:09.603 INFO:tasks.workunit.client.1.smithi097.stdout:3/369: mkdir d7/d15/d7d 0 2024-09-16T08:57:09.604 INFO:tasks.workunit.client.1.smithi097.stdout:7/384: creat d4/d7/d77/f84 x:0 0 0 2024-09-16T08:57:09.604 INFO:tasks.workunit.client.1.smithi097.stdout:7/385: write d4/d9/d1c/f38 [732827,27540] 0 2024-09-16T08:57:09.606 INFO:tasks.workunit.client.1.smithi097.stdout:5/408: unlink db/d18/d20/d2b/f5c 0 2024-09-16T08:57:09.607 INFO:tasks.workunit.client.1.smithi097.stdout:4/398: dwrite d0/d39/d71/f77 [0,4194304] 0 2024-09-16T08:57:09.608 INFO:tasks.workunit.client.1.smithi097.stdout:0/402: link da/d1a/f3a da/d24/d4c/d80/d3b/f91 0 2024-09-16T08:57:09.609 INFO:tasks.workunit.client.1.smithi097.stdout:3/370: rename d7/l12 to d7/de/d3c/d3f/d60/d7a/l7e 0 2024-09-16T08:57:09.609 INFO:tasks.workunit.client.1.smithi097.stdout:3/371: creat d7/de/d3c/d4f/f7f x:0 0 0 2024-09-16T08:57:09.609 INFO:tasks.workunit.client.0.smithi031.stdout:0/101: dwrite d6/f9 [0,4194304] 0 2024-09-16T08:57:09.610 INFO:tasks.workunit.client.0.smithi031.stdout:0/102: chown d6/d12/f22 251907002 1 2024-09-16T08:57:09.610 INFO:tasks.workunit.client.1.smithi097.stdout:5/409: truncate db/d1a/f3a 20526 0 2024-09-16T08:57:09.613 INFO:tasks.workunit.client.1.smithi097.stdout:6/388: dread d1/d3/d16/f4d [0,4194304] 0 2024-09-16T08:57:09.616 INFO:tasks.workunit.client.1.smithi097.stdout:5/410: link db/d18/d1c/c54 db/d18/d1c/d42/d46/d7e/d76/c82 0 2024-09-16T08:57:09.616 INFO:tasks.workunit.client.0.smithi031.stdout:0/103: dread d6/fa [0,4194304] 0 2024-09-16T08:57:09.618 INFO:tasks.workunit.client.1.smithi097.stdout:6/389: unlink d1/ce 0 2024-09-16T08:57:09.618 INFO:tasks.workunit.client.1.smithi097.stdout:8/354: sync 2024-09-16T08:57:09.619 INFO:tasks.workunit.client.1.smithi097.stdout:5/411: mknod db/d18/d1c/d42/d46/d7e/d64/c83 0 2024-09-16T08:57:09.619 INFO:tasks.workunit.client.1.smithi097.stdout:5/412: creat db/f84 x:0 0 0 2024-09-16T08:57:09.620 INFO:tasks.workunit.client.1.smithi097.stdout:3/372: write d7/d28/f2f [3145013,91442] 0 2024-09-16T08:57:09.620 INFO:tasks.workunit.client.0.smithi031.stdout:0/104: write d6/d12/f17 [4009210,128278] 0 2024-09-16T08:57:09.621 INFO:tasks.workunit.client.1.smithi097.stdout:6/390: rename d1/c15 to d1/d3/d3e/d5a/c75 0 2024-09-16T08:57:09.622 INFO:tasks.workunit.client.1.smithi097.stdout:6/391: chown d1/d3/d3e/d5a/c18 25 1 2024-09-16T08:57:09.622 INFO:tasks.workunit.client.1.smithi097.stdout:6/392: creat d1/dc/d3c/f76 x:0 0 0 2024-09-16T08:57:09.623 INFO:tasks.workunit.client.1.smithi097.stdout:3/373: dread d7/de/d29/f2d [0,4194304] 0 2024-09-16T08:57:09.623 INFO:tasks.workunit.client.1.smithi097.stdout:3/374: creat d7/de/d3c/d4f/f80 x:0 0 0 2024-09-16T08:57:09.626 INFO:tasks.workunit.client.1.smithi097.stdout:8/355: dread d8/dd/f19 [0,4194304] 0 2024-09-16T08:57:09.627 INFO:tasks.workunit.client.1.smithi097.stdout:6/393: symlink d1/dc/l77 0 2024-09-16T08:57:09.627 INFO:tasks.workunit.client.1.smithi097.stdout:6/394: chown d1/d3/d16/d6a 9 1 2024-09-16T08:57:09.629 INFO:tasks.workunit.client.1.smithi097.stdout:3/375: mkdir d7/d15/d52/d81 0 2024-09-16T08:57:09.629 INFO:tasks.workunit.client.1.smithi097.stdout:3/376: chown d7/d28/f2f 471 1 2024-09-16T08:57:09.629 INFO:tasks.workunit.client.1.smithi097.stdout:0/403: fsync da/d1a/f8e 0 2024-09-16T08:57:09.631 INFO:tasks.workunit.client.1.smithi097.stdout:6/395: rmdir d1/d3/d3e 39 2024-09-16T08:57:09.632 INFO:tasks.workunit.client.1.smithi097.stdout:6/396: write d1/f22 [1673326,2719] 0 2024-09-16T08:57:09.632 INFO:tasks.workunit.client.1.smithi097.stdout:6/397: write d1/d3/d16/d6a/f6b [2675111,122022] 0 2024-09-16T08:57:09.633 INFO:tasks.workunit.client.1.smithi097.stdout:3/377: symlink d7/d28/d63/l82 0 2024-09-16T08:57:09.633 INFO:tasks.workunit.client.1.smithi097.stdout:7/386: dwrite d4/d9/dc/f48 [0,4194304] 0 2024-09-16T08:57:09.634 INFO:tasks.workunit.client.1.smithi097.stdout:4/399: read d0/d7/d52/f50 [3267315,89686] 0 2024-09-16T08:57:09.636 INFO:tasks.workunit.client.1.smithi097.stdout:0/404: symlink da/d1a/d26/l92 0 2024-09-16T08:57:09.636 INFO:tasks.workunit.client.1.smithi097.stdout:6/398: getdents d1/dc/d1e 0 2024-09-16T08:57:09.638 INFO:tasks.workunit.client.1.smithi097.stdout:4/400: creat d0/d7/df/f8f x:0 0 0 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:0/405: mknod da/d24/d4c/d80/d3b/c93 0 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:0/406: dread - da/d24/d76/f8b zero size 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:0/407: readlink da/d24/d4c/d80/d3b/l5c 0 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:0/408: write da/d1a/d41/f71 [975505,45805] 0 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:0/409: truncate da/d24/d4c/d80/f3c 2960948 0 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:6/399: creat d1/d3/d16/d4f/f78 x:0 0 0 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:7/387: dread d4/d9/f24 [0,4194304] 0 2024-09-16T08:57:09.645 INFO:tasks.workunit.client.1.smithi097.stdout:5/413: dwrite db/d1a/f57 [0,4194304] 0 2024-09-16T08:57:09.647 INFO:tasks.workunit.client.1.smithi097.stdout:0/410: truncate da/df/f16 739303 0 2024-09-16T08:57:09.648 INFO:tasks.workunit.client.1.smithi097.stdout:0/411: chown da/d24/d4c/d80/d42/d5b/d60/d73/d85 66 1 2024-09-16T08:57:09.648 INFO:tasks.workunit.client.1.smithi097.stdout:0/412: stat da/d1a/d41/f71 0 2024-09-16T08:57:09.648 INFO:tasks.workunit.client.1.smithi097.stdout:0/413: fdatasync da/df/f16 0 2024-09-16T08:57:09.649 INFO:tasks.workunit.client.1.smithi097.stdout:6/400: dread d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:09.650 INFO:tasks.workunit.client.1.smithi097.stdout:8/356: dwrite d8/dd/d15/d2c/d4a/d4d/f5c [0,4194304] 0 2024-09-16T08:57:09.652 INFO:tasks.workunit.client.1.smithi097.stdout:7/388: symlink d4/d9/d2b/l85 0 2024-09-16T08:57:09.654 INFO:tasks.workunit.client.1.smithi097.stdout:3/378: dwrite d7/de/d3c/d3f/f6d [0,4194304] 0 2024-09-16T08:57:09.654 INFO:tasks.workunit.client.1.smithi097.stdout:4/401: truncate d0/d7/d52/f1d 6589232 0 2024-09-16T08:57:09.654 INFO:tasks.workunit.client.1.smithi097.stdout:5/414: creat db/d18/d1c/d42/d46/d7e/d64/d7f/f85 x:0 0 0 2024-09-16T08:57:09.654 INFO:tasks.workunit.client.1.smithi097.stdout:5/415: fsync db/d18/d1c/d42/d46/d7e/f51 0 2024-09-16T08:57:09.657 INFO:tasks.workunit.client.1.smithi097.stdout:6/401: creat d1/d3/d16/f79 x:0 0 0 2024-09-16T08:57:09.666 INFO:tasks.workunit.client.1.smithi097.stdout:3/379: symlink d7/de/d29/d35/d3d/d57/l83 0 2024-09-16T08:57:09.666 INFO:tasks.workunit.client.1.smithi097.stdout:3/380: truncate d7/de/d3c/f45 1435144 0 2024-09-16T08:57:09.666 INFO:tasks.workunit.client.1.smithi097.stdout:3/381: chown d7/d15 1 1 2024-09-16T08:57:09.666 INFO:tasks.workunit.client.1.smithi097.stdout:3/382: read - d7/de/d3c/f4d zero size 2024-09-16T08:57:09.667 INFO:tasks.workunit.client.1.smithi097.stdout:4/402: mknod d0/d23/c90 0 2024-09-16T08:57:09.669 INFO:tasks.workunit.client.1.smithi097.stdout:5/416: creat db/d1a/f86 x:0 0 0 2024-09-16T08:57:09.669 INFO:tasks.workunit.client.1.smithi097.stdout:6/402: dread d1/d3/d3e/d5a/f4a [0,4194304] 0 2024-09-16T08:57:09.669 INFO:tasks.workunit.client.1.smithi097.stdout:6/403: truncate d1/d3/d16/f79 666008 0 2024-09-16T08:57:09.669 INFO:tasks.workunit.client.1.smithi097.stdout:6/404: fsync d1/d3/d16/f21 0 2024-09-16T08:57:09.670 INFO:tasks.workunit.client.1.smithi097.stdout:6/405: dread - d1/dc/f72 zero size 2024-09-16T08:57:09.670 INFO:tasks.workunit.client.1.smithi097.stdout:6/406: chown d1/c70 8742512 1 2024-09-16T08:57:09.670 INFO:tasks.workunit.client.1.smithi097.stdout:6/407: chown d1/d3/d3e/d5a/l45 29 1 2024-09-16T08:57:09.672 INFO:tasks.workunit.client.1.smithi097.stdout:3/383: rmdir d7/d28 39 2024-09-16T08:57:09.673 INFO:tasks.workunit.client.1.smithi097.stdout:5/417: getdents db/d22/d58 0 2024-09-16T08:57:09.673 INFO:tasks.workunit.client.1.smithi097.stdout:5/418: chown db/d18/d1c/c72 1757107294 1 2024-09-16T08:57:09.673 INFO:tasks.workunit.client.1.smithi097.stdout:5/419: creat db/d18/d20/d2b/f87 x:0 0 0 2024-09-16T08:57:09.673 INFO:tasks.workunit.client.1.smithi097.stdout:6/408: dread d1/f5b [0,4194304] 0 2024-09-16T08:57:09.674 INFO:tasks.workunit.client.1.smithi097.stdout:3/384: creat d7/d28/d63/d70/f84 x:0 0 0 2024-09-16T08:57:09.675 INFO:tasks.workunit.client.1.smithi097.stdout:6/409: mknod d1/dc/d3c/c7a 0 2024-09-16T08:57:09.676 INFO:tasks.workunit.client.1.smithi097.stdout:6/410: creat d1/d3/d16/f7b x:0 0 0 2024-09-16T08:57:09.676 INFO:tasks.workunit.client.1.smithi097.stdout:0/414: dwrite da/f3e [4194304,4194304] 0 2024-09-16T08:57:09.677 INFO:tasks.workunit.client.1.smithi097.stdout:8/357: dwrite d8/fc [4194304,4194304] 0 2024-09-16T08:57:09.677 INFO:tasks.workunit.client.1.smithi097.stdout:7/389: dwrite d4/f6f [0,4194304] 0 2024-09-16T08:57:09.678 INFO:tasks.workunit.client.1.smithi097.stdout:6/411: rename d1/dc/d3c/c7a to d1/d3/c7c 0 2024-09-16T08:57:09.680 INFO:tasks.workunit.client.1.smithi097.stdout:0/415: mkdir da/d1a/d26/d94 0 2024-09-16T08:57:09.683 INFO:tasks.workunit.client.1.smithi097.stdout:4/403: write d0/d23/f3a [4171122,101456] 0 2024-09-16T08:57:09.688 INFO:tasks.workunit.client.1.smithi097.stdout:4/404: creat d0/d36/d4c/f91 x:0 0 0 2024-09-16T08:57:09.691 INFO:tasks.workunit.client.1.smithi097.stdout:4/405: symlink d0/d7/d52/l92 0 2024-09-16T08:57:09.691 INFO:tasks.workunit.client.1.smithi097.stdout:4/406: dread - d0/d39/d71/f87 zero size 2024-09-16T08:57:09.694 INFO:tasks.workunit.client.1.smithi097.stdout:8/358: truncate d8/dd/d15/d2c/d4a/d4d/f5c 511261 0 2024-09-16T08:57:09.695 INFO:tasks.workunit.client.1.smithi097.stdout:8/359: mknod d8/dd/d15/d2c/d4a/d4e/c67 0 2024-09-16T08:57:09.742 INFO:tasks.workunit.client.1.smithi097.stdout:3/385: dwrite d7/d28/d30/d69/f6a [0,4194304] 0 2024-09-16T08:57:09.745 INFO:tasks.workunit.client.1.smithi097.stdout:3/386: creat d7/de/d3c/d3f/d60/d7a/f85 x:0 0 0 2024-09-16T08:57:09.745 INFO:tasks.workunit.client.1.smithi097.stdout:3/387: fdatasync d7/de/d3c/d3f/f5e 0 2024-09-16T08:57:09.746 INFO:tasks.workunit.client.1.smithi097.stdout:0/416: write da/d24/d4c/d80/d42/d5b/d60/d68/f39 [3587952,63041] 0 2024-09-16T08:57:09.746 INFO:tasks.workunit.client.1.smithi097.stdout:0/417: truncate da/d24/d4c/d80/f62 554226 0 2024-09-16T08:57:09.752 INFO:tasks.workunit.client.1.smithi097.stdout:3/388: dread d7/d28/f2f [0,4194304] 0 2024-09-16T08:57:09.752 INFO:tasks.workunit.client.1.smithi097.stdout:3/389: write d7/de/d6c/f5a [746733,122442] 0 2024-09-16T08:57:09.753 INFO:tasks.workunit.client.1.smithi097.stdout:4/407: read d0/d7/df/d15/f41 [2928899,111508] 0 2024-09-16T08:57:09.757 INFO:tasks.workunit.client.1.smithi097.stdout:3/390: unlink d7/de/d3c/d3f/d60/d7a/f85 0 2024-09-16T08:57:09.758 INFO:tasks.workunit.client.1.smithi097.stdout:4/408: creat d0/d23/d5f/f93 x:0 0 0 2024-09-16T08:57:09.758 INFO:tasks.workunit.client.1.smithi097.stdout:0/418: rmdir da/df/d13 39 2024-09-16T08:57:09.758 INFO:tasks.workunit.client.1.smithi097.stdout:0/419: chown da/d1a 417108 1 2024-09-16T08:57:09.760 INFO:tasks.workunit.client.1.smithi097.stdout:7/390: dwrite d4/d7/d7b/d43/f44 [0,4194304] 0 2024-09-16T08:57:09.760 INFO:tasks.workunit.client.1.smithi097.stdout:7/391: dread - d4/d9/d1c/d5f/f6c zero size 2024-09-16T08:57:09.760 INFO:tasks.workunit.client.1.smithi097.stdout:7/392: read d4/d7/d19/f33 [86921,39453] 0 2024-09-16T08:57:09.760 INFO:tasks.workunit.client.1.smithi097.stdout:7/393: chown d4/d9/d1c/f28 5631652 1 2024-09-16T08:57:09.760 INFO:tasks.workunit.client.1.smithi097.stdout:7/394: truncate d4/d9/d1c/f38 1322175 0 2024-09-16T08:57:09.761 INFO:tasks.workunit.client.1.smithi097.stdout:0/420: mknod da/d1a/d26/d75/c95 0 2024-09-16T08:57:09.761 INFO:tasks.workunit.client.1.smithi097.stdout:6/412: dwrite d1/d3/f2f [4194304,4194304] 0 2024-09-16T08:57:09.762 INFO:tasks.workunit.client.1.smithi097.stdout:8/360: dwrite d8/dd/d15/d2c/d4a/d4d/f5c [0,4194304] 0 2024-09-16T08:57:09.762 INFO:tasks.workunit.client.1.smithi097.stdout:8/361: creat d8/dd/d15/d2c/d4a/d4d/f68 x:0 0 0 2024-09-16T08:57:09.762 INFO:tasks.workunit.client.1.smithi097.stdout:6/413: write d1/d3/f1c [1041877,22446] 0 2024-09-16T08:57:09.762 INFO:tasks.workunit.client.1.smithi097.stdout:6/414: dread - d1/d3/d16/f7b zero size 2024-09-16T08:57:09.762 INFO:tasks.workunit.client.1.smithi097.stdout:6/415: truncate d1/d3/d16/d4f/f78 161363 0 2024-09-16T08:57:09.762 INFO:tasks.workunit.client.1.smithi097.stdout:6/416: readlink d1/l13 0 2024-09-16T08:57:09.762 INFO:tasks.workunit.client.1.smithi097.stdout:6/417: chown d1/dc/d1e/c30 0 1 2024-09-16T08:57:09.764 INFO:tasks.workunit.client.1.smithi097.stdout:7/395: mkdir d4/d1b/d41/d86 0 2024-09-16T08:57:09.766 INFO:tasks.workunit.client.1.smithi097.stdout:0/421: creat da/d7e/f96 x:0 0 0 2024-09-16T08:57:09.774 INFO:tasks.workunit.client.1.smithi097.stdout:6/418: truncate d1/d3/d16/d4f/f55 2652527 0 2024-09-16T08:57:09.775 INFO:tasks.workunit.client.1.smithi097.stdout:7/396: symlink d4/d7/d7b/d43/l87 0 2024-09-16T08:57:09.777 INFO:tasks.workunit.client.1.smithi097.stdout:6/419: getdents d1/d3 0 2024-09-16T08:57:09.777 INFO:tasks.workunit.client.1.smithi097.stdout:6/420: write d1/d3/f7 [4502339,30261] 0 2024-09-16T08:57:09.778 INFO:tasks.workunit.client.1.smithi097.stdout:7/397: mkdir d4/d9/d51/d88 0 2024-09-16T08:57:09.778 INFO:tasks.workunit.client.1.smithi097.stdout:6/421: write d1/d3/d3e/d5a/f32 [413296,95521] 0 2024-09-16T08:57:09.778 INFO:tasks.workunit.client.1.smithi097.stdout:6/422: readlink d1/dc/d1e/l3f 0 2024-09-16T08:57:09.778 INFO:tasks.workunit.client.1.smithi097.stdout:6/423: creat d1/dc/f7d x:0 0 0 2024-09-16T08:57:09.780 INFO:tasks.workunit.client.1.smithi097.stdout:7/398: symlink d4/d7/d7b/l89 0 2024-09-16T08:57:09.780 INFO:tasks.workunit.client.1.smithi097.stdout:3/391: dwrite d7/d28/f2f [0,4194304] 0 2024-09-16T08:57:09.780 INFO:tasks.workunit.client.1.smithi097.stdout:8/362: dwrite d8/dd/d15/d16/f3a [4194304,4194304] 0 2024-09-16T08:57:09.782 INFO:tasks.workunit.client.1.smithi097.stdout:0/422: rename da/d1a/d26/d75 to da/d1a/d26/d47/d97 0 2024-09-16T08:57:09.784 INFO:tasks.workunit.client.1.smithi097.stdout:3/392: creat d7/de/d3c/d3f/d60/d6b/f86 x:0 0 0 2024-09-16T08:57:09.784 INFO:tasks.workunit.client.1.smithi097.stdout:3/393: dread - d7/de/d29/f76 zero size 2024-09-16T08:57:09.785 INFO:tasks.workunit.client.1.smithi097.stdout:3/394: stat d7/de/f1e 0 2024-09-16T08:57:09.785 INFO:tasks.workunit.client.1.smithi097.stdout:3/395: truncate d7/de/d3c/d4f/f7f 959232 0 2024-09-16T08:57:09.785 INFO:tasks.workunit.client.1.smithi097.stdout:3/396: read d7/de/d29/f49 [92130,39791] 0 2024-09-16T08:57:09.786 INFO:tasks.workunit.client.1.smithi097.stdout:0/423: symlink da/d24/d4c/d80/d42/d5b/d60/d68/l98 0 2024-09-16T08:57:09.786 INFO:tasks.workunit.client.1.smithi097.stdout:0/424: write da/f1b [36387,22163] 0 2024-09-16T08:57:09.786 INFO:tasks.workunit.client.1.smithi097.stdout:0/425: write da/d24/d4c/d80/d3b/f91 [4380559,128371] 0 2024-09-16T08:57:09.789 INFO:tasks.workunit.client.1.smithi097.stdout:1/394: sync 2024-09-16T08:57:09.789 INFO:tasks.workunit.client.1.smithi097.stdout:9/351: sync 2024-09-16T08:57:09.789 INFO:tasks.workunit.client.1.smithi097.stdout:2/372: sync 2024-09-16T08:57:09.795 INFO:tasks.workunit.client.1.smithi097.stdout:0/426: mknod da/d24/d4c/d80/d42/d5b/d60/c99 0 2024-09-16T08:57:09.796 INFO:tasks.workunit.client.1.smithi097.stdout:0/427: creat da/d24/d76/f9a x:0 0 0 2024-09-16T08:57:09.798 INFO:tasks.workunit.client.1.smithi097.stdout:2/373: rename d3/d4/d8/f33 to d3/d4/d8/d34/d48/f60 0 2024-09-16T08:57:09.798 INFO:tasks.workunit.client.1.smithi097.stdout:3/397: write d7/d28/d30/d69/f6a [771234,46279] 0 2024-09-16T08:57:09.798 INFO:tasks.workunit.client.1.smithi097.stdout:3/398: creat d7/de/d29/d35/d3d/f87 x:0 0 0 2024-09-16T08:57:09.799 INFO:tasks.workunit.client.1.smithi097.stdout:1/395: mkdir d5/d2c/d3b/d7f 0 2024-09-16T08:57:09.802 INFO:tasks.workunit.client.1.smithi097.stdout:9/352: creat d2/d5/d3a/f6f x:0 0 0 2024-09-16T08:57:09.803 INFO:tasks.workunit.client.1.smithi097.stdout:0/428: symlink da/l9b 0 2024-09-16T08:57:09.807 INFO:tasks.workunit.client.1.smithi097.stdout:9/353: mkdir d2/d5/d3a/d70 0 2024-09-16T08:57:09.808 INFO:tasks.workunit.client.1.smithi097.stdout:0/429: mkdir da/d1a/d9c 0 2024-09-16T08:57:09.809 INFO:tasks.workunit.client.1.smithi097.stdout:0/430: chown da/d24/d4c/d80/d42/d5b/d60/d68/f5d 785038022 1 2024-09-16T08:57:09.810 INFO:tasks.workunit.client.1.smithi097.stdout:6/424: dwrite d1/f37 [0,4194304] 0 2024-09-16T08:57:09.810 INFO:tasks.workunit.client.1.smithi097.stdout:6/425: fsync d1/dc/d3c/f6d 0 2024-09-16T08:57:09.810 INFO:tasks.workunit.client.1.smithi097.stdout:0/431: getdents da/df/d13 0 2024-09-16T08:57:09.811 INFO:tasks.workunit.client.1.smithi097.stdout:7/399: dwrite d4/d7/d7b/f7c [0,4194304] 0 2024-09-16T08:57:09.811 INFO:tasks.workunit.client.1.smithi097.stdout:2/374: write d3/d4/d8/f16 [5927133,126408] 0 2024-09-16T08:57:09.811 INFO:tasks.workunit.client.1.smithi097.stdout:2/375: write d3/f23 [3928370,20025] 0 2024-09-16T08:57:09.812 INFO:tasks.workunit.client.1.smithi097.stdout:8/363: dwrite d8/dd/f61 [0,4194304] 0 2024-09-16T08:57:09.812 INFO:tasks.workunit.client.1.smithi097.stdout:1/396: dread d5/f9 [0,4194304] 0 2024-09-16T08:57:09.812 INFO:tasks.workunit.client.1.smithi097.stdout:9/354: rename d2/f51 to d2/d5/f71 0 2024-09-16T08:57:09.812 INFO:tasks.workunit.client.1.smithi097.stdout:9/355: read - d2/d3f/f4b zero size 2024-09-16T08:57:09.814 INFO:tasks.workunit.client.1.smithi097.stdout:7/400: dread d4/d1b/f69 [0,4194304] 0 2024-09-16T08:57:09.814 INFO:tasks.workunit.client.1.smithi097.stdout:6/426: truncate d1/dc/d1e/f3a 4097426 0 2024-09-16T08:57:09.816 INFO:tasks.workunit.client.1.smithi097.stdout:3/399: write d7/d28/d30/f53 [2583162,23767] 0 2024-09-16T08:57:09.816 INFO:tasks.workunit.client.1.smithi097.stdout:3/400: dread - d7/de/d3c/f79 zero size 2024-09-16T08:57:09.819 INFO:tasks.workunit.client.1.smithi097.stdout:0/432: link da/df/d3d/f66 da/d24/d4c/f9d 0 2024-09-16T08:57:09.819 INFO:tasks.workunit.client.1.smithi097.stdout:0/433: dread - da/d1a/f58 zero size 2024-09-16T08:57:09.819 INFO:tasks.workunit.client.1.smithi097.stdout:0/434: chown da/l4f 153 1 2024-09-16T08:57:09.819 INFO:tasks.workunit.client.1.smithi097.stdout:0/435: write da/df/d3d/f66 [1171449,70317] 0 2024-09-16T08:57:09.826 INFO:tasks.workunit.client.1.smithi097.stdout:1/397: truncate d5/db/f74 1821330 0 2024-09-16T08:57:09.826 INFO:tasks.workunit.client.1.smithi097.stdout:1/398: creat d5/d2c/d3b/f80 x:0 0 0 2024-09-16T08:57:09.827 INFO:tasks.workunit.client.1.smithi097.stdout:1/399: chown d5/db/dc/d19/c60 40463147 1 2024-09-16T08:57:09.827 INFO:tasks.workunit.client.1.smithi097.stdout:1/400: write d5/db/dc/d19/f29 [6789018,121275] 0 2024-09-16T08:57:09.827 INFO:tasks.workunit.client.1.smithi097.stdout:9/356: truncate d2/f13 2063281 0 2024-09-16T08:57:09.830 INFO:tasks.workunit.client.1.smithi097.stdout:7/401: link d4/d12/l67 d4/d9/d51/d88/l8a 0 2024-09-16T08:57:09.830 INFO:tasks.workunit.client.1.smithi097.stdout:6/427: mknod d1/d3/d16/d4f/c7e 0 2024-09-16T08:57:09.831 INFO:tasks.workunit.client.1.smithi097.stdout:3/401: mkdir d7/d28/d30/d88 0 2024-09-16T08:57:09.831 INFO:tasks.workunit.client.1.smithi097.stdout:3/402: creat d7/de/d3c/d3f/d60/d6b/f89 x:0 0 0 2024-09-16T08:57:09.832 INFO:tasks.workunit.client.1.smithi097.stdout:0/436: read f8 [2778556,3904] 0 2024-09-16T08:57:09.832 INFO:tasks.workunit.client.1.smithi097.stdout:0/437: fsync da/d24/d4c/d80/d3b/f5a 0 2024-09-16T08:57:09.834 INFO:tasks.workunit.client.1.smithi097.stdout:2/376: dwrite d3/d4/d8/f2b [4194304,4194304] 0 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:2/377: readlink d3/d4/d8/d51/l52 0 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:2/378: fsync d3/d4/d9/f36 0 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:2/379: chown d3/d4/d8/d51/f59 6583018 1 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:8/364: dwrite d8/dd/d15/f28 [0,4194304] 0 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:6/428: dread d1/d3/d3e/d5a/f4a [0,4194304] 0 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:1/401: link d5/d14/c44 d5/d2c/d3b/c81 0 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:1/402: truncate d5/d2c/d3b/f61 714739 0 2024-09-16T08:57:09.835 INFO:tasks.workunit.client.1.smithi097.stdout:1/403: write d5/db/f5d [371471,63252] 0 2024-09-16T08:57:09.837 INFO:tasks.workunit.client.1.smithi097.stdout:7/402: mkdir d4/d9/d51/d8b 0 2024-09-16T08:57:09.839 INFO:tasks.workunit.client.1.smithi097.stdout:6/429: dread d1/d3/f2c [0,4194304] 0 2024-09-16T08:57:09.839 INFO:tasks.workunit.client.1.smithi097.stdout:3/403: dread d7/d28/d30/d69/f6a [0,4194304] 0 2024-09-16T08:57:09.841 INFO:tasks.workunit.client.1.smithi097.stdout:2/380: mkdir d3/d4/d8/d34/d48/d61 0 2024-09-16T08:57:09.849 INFO:tasks.workunit.client.1.smithi097.stdout:8/365: link d8/dd/d15/l1d d8/dd/d15/d2c/d48/l69 0 2024-09-16T08:57:09.850 INFO:tasks.workunit.client.1.smithi097.stdout:1/404: getdents d5/d2c 0 2024-09-16T08:57:09.850 INFO:tasks.workunit.client.1.smithi097.stdout:1/405: write d5/db/dc/f33 [188680,109440] 0 2024-09-16T08:57:09.851 INFO:tasks.workunit.client.1.smithi097.stdout:1/406: write d5/d17/f5b [805007,52255] 0 2024-09-16T08:57:09.851 INFO:tasks.workunit.client.1.smithi097.stdout:1/407: stat d5/d17/d54/d6b 0 2024-09-16T08:57:09.852 INFO:tasks.workunit.client.1.smithi097.stdout:9/357: mkdir d2/d5/d72 0 2024-09-16T08:57:09.853 INFO:tasks.workunit.client.1.smithi097.stdout:7/403: creat d4/d12/d1f/d22/f8c x:0 0 0 2024-09-16T08:57:09.853 INFO:tasks.workunit.client.1.smithi097.stdout:7/404: read d4/d9/f24 [2203450,122248] 0 2024-09-16T08:57:09.853 INFO:tasks.workunit.client.1.smithi097.stdout:7/405: write d4/fb [3470160,115561] 0 2024-09-16T08:57:09.854 INFO:tasks.workunit.client.1.smithi097.stdout:0/438: rmdir da/d1a/d41 39 2024-09-16T08:57:09.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/430: symlink d1/d3/d16/l7f 0 2024-09-16T08:57:09.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/431: truncate d1/dc/d3c/f44 1036459 0 2024-09-16T08:57:09.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/432: readlink d1/d3/l46 0 2024-09-16T08:57:09.860 INFO:tasks.workunit.client.1.smithi097.stdout:9/358: write d2/d5/d3a/d3c/f54 [1548353,60730] 0 2024-09-16T08:57:09.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/433: stat d1/d3/d16/f21 0 2024-09-16T08:57:09.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/434: creat d1/dc/f80 x:0 0 0 2024-09-16T08:57:09.861 INFO:tasks.workunit.client.1.smithi097.stdout:3/404: truncate d7/de/d3c/d3f/f6d 3249207 0 2024-09-16T08:57:09.861 INFO:tasks.workunit.client.1.smithi097.stdout:2/381: mknod d3/d4/c62 0 2024-09-16T08:57:09.862 INFO:tasks.workunit.client.1.smithi097.stdout:0/439: read da/d1a/f1c [1175179,84666] 0 2024-09-16T08:57:09.862 INFO:tasks.workunit.client.1.smithi097.stdout:0/440: chown da/d1a/d41/f50 4779 1 2024-09-16T08:57:09.862 INFO:tasks.workunit.client.1.smithi097.stdout:1/408: symlink d5/d14/l82 0 2024-09-16T08:57:09.863 INFO:tasks.workunit.client.1.smithi097.stdout:7/406: mknod d4/d7/d77/c8d 0 2024-09-16T08:57:09.869 INFO:tasks.workunit.client.1.smithi097.stdout:1/409: unlink d5/f7e 0 2024-09-16T08:57:09.870 INFO:tasks.workunit.client.1.smithi097.stdout:1/410: mknod d5/db/dc/c83 0 2024-09-16T08:57:09.871 INFO:tasks.workunit.client.1.smithi097.stdout:1/411: write d5/f77 [607253,115960] 0 2024-09-16T08:57:09.873 INFO:tasks.workunit.client.1.smithi097.stdout:1/412: getdents d5/d17/d54/d6b 0 2024-09-16T08:57:09.873 INFO:tasks.workunit.client.1.smithi097.stdout:1/413: truncate d5/d2c/f5e 7388 0 2024-09-16T08:57:09.873 INFO:tasks.workunit.client.1.smithi097.stdout:1/414: chown d5/d17/f26 4858 1 2024-09-16T08:57:09.875 INFO:tasks.workunit.client.1.smithi097.stdout:1/415: truncate d5/db/f74 1293817 0 2024-09-16T08:57:09.875 INFO:tasks.workunit.client.1.smithi097.stdout:1/416: readlink d5/d17/d54/d6b/l6e 0 2024-09-16T08:57:09.876 INFO:tasks.workunit.client.1.smithi097.stdout:8/366: dwrite d8/dd/d15/d2c/d4a/d4d/f35 [0,4194304] 0 2024-09-16T08:57:09.876 INFO:tasks.workunit.client.1.smithi097.stdout:8/367: write d8/dd/d15/d2c/d4a/d4d/d3d/f43 [250116,66212] 0 2024-09-16T08:57:09.876 INFO:tasks.workunit.client.1.smithi097.stdout:8/368: creat d8/dd/f6a x:0 0 0 2024-09-16T08:57:09.878 INFO:tasks.workunit.client.1.smithi097.stdout:6/435: dread d1/d3/f4b [0,4194304] 0 2024-09-16T08:57:09.879 INFO:tasks.workunit.client.1.smithi097.stdout:8/369: dread d8/dd/f26 [0,4194304] 0 2024-09-16T08:57:09.879 INFO:tasks.workunit.client.1.smithi097.stdout:9/359: dread f0 [0,4194304] 0 2024-09-16T08:57:09.880 INFO:tasks.workunit.client.1.smithi097.stdout:9/360: readlink d2/d30/l6a 0 2024-09-16T08:57:09.881 INFO:tasks.workunit.client.1.smithi097.stdout:1/417: mknod d5/d2c/c84 0 2024-09-16T08:57:09.881 INFO:tasks.workunit.client.1.smithi097.stdout:1/418: chown d5/d17/d54/d6b/l6e 14359 1 2024-09-16T08:57:09.888 INFO:tasks.workunit.client.1.smithi097.stdout:8/370: dread d8/f1e [0,4194304] 0 2024-09-16T08:57:09.894 INFO:tasks.workunit.client.1.smithi097.stdout:1/419: dread d5/db/dc/d19/f3f [0,4194304] 0 2024-09-16T08:57:09.894 INFO:tasks.workunit.client.1.smithi097.stdout:8/371: symlink d8/dd/d15/d2c/d4a/d4d/l6b 0 2024-09-16T08:57:09.895 INFO:tasks.workunit.client.1.smithi097.stdout:6/436: getdents d1/d3/d16 0 2024-09-16T08:57:09.897 INFO:tasks.workunit.client.1.smithi097.stdout:1/420: mkdir d5/d70/d85 0 2024-09-16T08:57:09.897 INFO:tasks.workunit.client.1.smithi097.stdout:1/421: fsync d5/db/dc/d19/f4d 0 2024-09-16T08:57:09.898 INFO:tasks.workunit.client.1.smithi097.stdout:8/372: creat d8/dd/d15/d2c/d48/d56/f6c x:0 0 0 2024-09-16T08:57:09.899 INFO:tasks.workunit.client.1.smithi097.stdout:6/437: mkdir d1/d3/d81 0 2024-09-16T08:57:09.900 INFO:tasks.workunit.client.1.smithi097.stdout:1/422: mknod d5/db/d25/c86 0 2024-09-16T08:57:09.900 INFO:tasks.workunit.client.1.smithi097.stdout:1/423: chown d5/db/d25/l31 251384 1 2024-09-16T08:57:09.902 INFO:tasks.workunit.client.1.smithi097.stdout:6/438: creat d1/d3/d16/f82 x:0 0 0 2024-09-16T08:57:09.903 INFO:tasks.workunit.client.1.smithi097.stdout:1/424: mknod d5/db/dc/c87 0 2024-09-16T08:57:09.903 INFO:tasks.workunit.client.1.smithi097.stdout:1/425: creat d5/db/dc/d19/f88 x:0 0 0 2024-09-16T08:57:09.905 INFO:tasks.workunit.client.1.smithi097.stdout:6/439: creat d1/d3/d3e/d5a/f83 x:0 0 0 2024-09-16T08:57:09.906 INFO:tasks.workunit.client.1.smithi097.stdout:2/382: dwrite d3/d4/d8/d34/f3c [0,4194304] 0 2024-09-16T08:57:09.906 INFO:tasks.workunit.client.1.smithi097.stdout:9/361: dwrite d2/fc [0,4194304] 0 2024-09-16T08:57:09.906 INFO:tasks.workunit.client.1.smithi097.stdout:2/383: creat d3/d4/d8/d51/f63 x:0 0 0 2024-09-16T08:57:09.907 INFO:tasks.workunit.client.1.smithi097.stdout:1/426: mknod d5/d17/d54/d6b/d71/c89 0 2024-09-16T08:57:09.907 INFO:tasks.workunit.client.1.smithi097.stdout:6/440: getdents d1/d3/d3e 0 2024-09-16T08:57:09.911 INFO:tasks.workunit.client.1.smithi097.stdout:9/362: mkdir d2/d5/d3a/d3c/d73 0 2024-09-16T08:57:09.912 INFO:tasks.workunit.client.1.smithi097.stdout:1/427: dread d5/f12 [0,4194304] 0 2024-09-16T08:57:09.912 INFO:tasks.workunit.client.1.smithi097.stdout:6/441: creat d1/dc/d1e/f84 x:0 0 0 2024-09-16T08:57:09.912 INFO:tasks.workunit.client.1.smithi097.stdout:6/442: stat d1/d3/d3e/d5a/l45 0 2024-09-16T08:57:09.914 INFO:tasks.workunit.client.1.smithi097.stdout:9/363: mkdir d2/d30/d74 0 2024-09-16T08:57:09.914 INFO:tasks.workunit.client.1.smithi097.stdout:9/364: chown d2/d5/c26 310000 1 2024-09-16T08:57:09.916 INFO:tasks.workunit.client.1.smithi097.stdout:1/428: mknod d5/d70/c8a 0 2024-09-16T08:57:09.916 INFO:tasks.workunit.client.1.smithi097.stdout:1/429: write d5/db/dc/d19/f38 [700148,52659] 0 2024-09-16T08:57:09.917 INFO:tasks.workunit.client.1.smithi097.stdout:9/365: link d2/ld d2/d3f/d62/l75 0 2024-09-16T08:57:09.918 INFO:tasks.workunit.client.1.smithi097.stdout:1/430: symlink d5/db/d25/l8b 0 2024-09-16T08:57:09.918 INFO:tasks.workunit.client.1.smithi097.stdout:1/431: write d5/d2c/d3b/f61 [1400756,45950] 0 2024-09-16T08:57:09.918 INFO:tasks.workunit.client.1.smithi097.stdout:1/432: creat d5/db/dc/d19/f8c x:0 0 0 2024-09-16T08:57:09.919 INFO:tasks.workunit.client.1.smithi097.stdout:9/366: creat d2/d5/d72/f76 x:0 0 0 2024-09-16T08:57:09.919 INFO:tasks.workunit.client.1.smithi097.stdout:9/367: chown d2/d30/d74 767292395 1 2024-09-16T08:57:09.919 INFO:tasks.workunit.client.1.smithi097.stdout:6/443: write d1/f12 [3539085,74775] 0 2024-09-16T08:57:09.920 INFO:tasks.workunit.client.1.smithi097.stdout:1/433: creat d5/db/d25/f8d x:0 0 0 2024-09-16T08:57:09.929 INFO:tasks.workunit.client.1.smithi097.stdout:2/384: dwrite d3/d4/dd/d53/d47/f49 [0,4194304] 0 2024-09-16T08:57:09.931 INFO:tasks.workunit.client.1.smithi097.stdout:6/444: write d1/f4e [1834774,67056] 0 2024-09-16T08:57:09.933 INFO:tasks.workunit.client.1.smithi097.stdout:1/434: dread d5/d2c/f4b [0,4194304] 0 2024-09-16T08:57:09.934 INFO:tasks.workunit.client.1.smithi097.stdout:6/445: getdents d1/d3/d16 0 2024-09-16T08:57:09.934 INFO:tasks.workunit.client.1.smithi097.stdout:6/446: fdatasync d1/dc/f7d 0 2024-09-16T08:57:09.934 INFO:tasks.workunit.client.1.smithi097.stdout:6/447: truncate d1/d3/d16/f7b 871786 0 2024-09-16T08:57:09.935 INFO:tasks.workunit.client.1.smithi097.stdout:6/448: rename d1/c28 to d1/c85 0 2024-09-16T08:57:09.935 INFO:tasks.workunit.client.1.smithi097.stdout:6/449: mkdir d1/d3/d16/d86 0 2024-09-16T08:57:09.936 INFO:tasks.workunit.client.1.smithi097.stdout:6/450: mknod d1/d3/d3e/c87 0 2024-09-16T08:57:09.944 INFO:tasks.workunit.client.1.smithi097.stdout:6/451: dread d1/f64 [0,4194304] 0 2024-09-16T08:57:09.945 INFO:tasks.workunit.client.1.smithi097.stdout:6/452: rename d1/dc/d1e/f3a to d1/d3/d81/f88 0 2024-09-16T08:57:09.946 INFO:tasks.workunit.client.1.smithi097.stdout:6/453: truncate d1/d3/d16/f82 118781 0 2024-09-16T08:57:09.946 INFO:tasks.workunit.client.1.smithi097.stdout:6/454: write d1/dc/f7d [757388,121578] 0 2024-09-16T08:57:09.946 INFO:tasks.workunit.client.1.smithi097.stdout:6/455: creat d1/d3/d16/f89 x:0 0 0 2024-09-16T08:57:09.947 INFO:tasks.workunit.client.1.smithi097.stdout:6/456: truncate d1/f4e 336111 0 2024-09-16T08:57:09.949 INFO:tasks.workunit.client.1.smithi097.stdout:6/457: rename d1/d3/d16/d6a/f6b to d1/d3/d16/d86/f8a 0 2024-09-16T08:57:09.950 INFO:tasks.workunit.client.1.smithi097.stdout:6/458: creat d1/d3/d16/d86/f8b x:0 0 0 2024-09-16T08:57:09.950 INFO:tasks.workunit.client.1.smithi097.stdout:6/459: chown d1/dc/d1e 2251 1 2024-09-16T08:57:09.951 INFO:tasks.workunit.client.1.smithi097.stdout:6/460: rename d1/d3/d3e/c87 to d1/d3/d3e/c8c 0 2024-09-16T08:57:09.952 INFO:tasks.workunit.client.1.smithi097.stdout:6/461: creat d1/d3/f8d x:0 0 0 2024-09-16T08:57:09.952 INFO:tasks.workunit.client.1.smithi097.stdout:6/462: chown d1/dc/d1e/c5d 7 1 2024-09-16T08:57:09.973 INFO:tasks.workunit.client.1.smithi097.stdout:5/420: sync 2024-09-16T08:57:09.973 INFO:tasks.workunit.client.1.smithi097.stdout:4/409: sync 2024-09-16T08:57:09.975 INFO:tasks.workunit.client.1.smithi097.stdout:4/410: link d0/d35/f38 d0/d7/d52/f94 0 2024-09-16T08:57:09.983 INFO:tasks.workunit.client.1.smithi097.stdout:3/405: dwrite d7/de/d6c/f5a [0,4194304] 0 2024-09-16T08:57:09.986 INFO:tasks.workunit.client.1.smithi097.stdout:2/385: dread d3/d4/d8/f2b [4194304,4194304] 0 2024-09-16T08:57:09.988 INFO:tasks.workunit.client.1.smithi097.stdout:2/386: mknod d3/d4/c64 0 2024-09-16T08:57:09.989 INFO:tasks.workunit.client.1.smithi097.stdout:3/406: dread d7/d28/d30/f53 [0,4194304] 0 2024-09-16T08:57:09.989 INFO:tasks.workunit.client.1.smithi097.stdout:3/407: write d7/de/d3c/d4f/f80 [426167,111844] 0 2024-09-16T08:57:09.990 INFO:tasks.workunit.client.1.smithi097.stdout:2/387: unlink d3/d4/f2d 0 2024-09-16T08:57:09.994 INFO:tasks.workunit.client.1.smithi097.stdout:3/408: rename d7/de/d3c/d3f/l5d to d7/de/d3c/d3f/d60/d6b/l8a 0 2024-09-16T08:57:09.995 INFO:tasks.workunit.client.1.smithi097.stdout:3/409: creat d7/de/d29/d35/d3d/d57/d64/f8b x:0 0 0 2024-09-16T08:57:09.995 INFO:tasks.workunit.client.1.smithi097.stdout:3/410: fsync d7/de/d29/d35/f22 0 2024-09-16T08:57:09.996 INFO:tasks.workunit.client.1.smithi097.stdout:2/388: mknod d3/d4/dd/d53/d47/c65 0 2024-09-16T08:57:09.997 INFO:tasks.workunit.client.1.smithi097.stdout:2/389: symlink d3/l66 0 2024-09-16T08:57:10.022 INFO:tasks.workunit.client.1.smithi097.stdout:7/407: dwrite d4/d1b/f35 [0,4194304] 0 2024-09-16T08:57:10.024 INFO:tasks.workunit.client.1.smithi097.stdout:7/408: mkdir d4/d9/dc/d8e 0 2024-09-16T08:57:10.026 INFO:tasks.workunit.client.1.smithi097.stdout:3/411: dwrite d7/de/d3c/d3f/f7b [0,4194304] 0 2024-09-16T08:57:10.027 INFO:tasks.workunit.client.1.smithi097.stdout:7/409: read - d4/d7/f32 zero size 2024-09-16T08:57:10.027 INFO:tasks.workunit.client.1.smithi097.stdout:7/410: fsync d4/d12/d1f/f21 0 2024-09-16T08:57:10.027 INFO:tasks.workunit.client.1.smithi097.stdout:7/411: write d4/d7/ff [229213,96078] 0 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:8/373: dwrite d8/dd/d15/d2c/d4a/d4d/d3d/f5b [0,4194304] 0 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:5/421: dwrite db/d18/f34 [0,4194304] 0 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:5/422: dread - db/d18/d1c/d42/d46/f47 zero size 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:5/423: chown db/d18/f62 77790 1 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:8/374: creat d8/dd/d15/d2c/d48/d56/f6d x:0 0 0 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:8/375: chown d8/dd/d15/d2c/d4a/d4d/f68 73 1 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:7/412: link d4/f23 d4/d7/d7b/f8f 0 2024-09-16T08:57:10.032 INFO:tasks.workunit.client.1.smithi097.stdout:1/435: dwrite d5/d2c/f2e [4194304,4194304] 0 2024-09-16T08:57:10.034 INFO:tasks.workunit.client.1.smithi097.stdout:6/463: dwrite d1/d3/f7 [0,4194304] 0 2024-09-16T08:57:10.034 INFO:tasks.workunit.client.1.smithi097.stdout:6/464: dread - d1/d3/f8d zero size 2024-09-16T08:57:10.034 INFO:tasks.workunit.client.1.smithi097.stdout:6/465: stat d1/f64 0 2024-09-16T08:57:10.036 INFO:tasks.workunit.client.1.smithi097.stdout:2/390: dwrite d3/d4/dd/d53/f58 [0,4194304] 0 2024-09-16T08:57:10.036 INFO:tasks.workunit.client.1.smithi097.stdout:2/391: write d3/d1b/f5f [909624,23217] 0 2024-09-16T08:57:10.041 INFO:tasks.workunit.client.1.smithi097.stdout:5/424: dread db/d18/d1c/f39 [0,4194304] 0 2024-09-16T08:57:10.042 INFO:tasks.workunit.client.1.smithi097.stdout:5/425: dread - db/d18/d1c/d42/d46/f47 zero size 2024-09-16T08:57:10.076 INFO:tasks.workunit.client.1.smithi097.stdout:8/376: mknod d8/dd/d15/c6e 0 2024-09-16T08:57:10.085 INFO:tasks.workunit.client.1.smithi097.stdout:6/466: symlink d1/d3/d16/d6a/l8e 0 2024-09-16T08:57:10.086 INFO:tasks.workunit.client.1.smithi097.stdout:3/412: dwrite d7/de/d3c/f45 [0,4194304] 0 2024-09-16T08:57:10.087 INFO:tasks.workunit.client.1.smithi097.stdout:2/392: mkdir d3/d4/d9/d67 0 2024-09-16T08:57:10.088 INFO:tasks.workunit.client.1.smithi097.stdout:5/426: mknod db/d18/d1c/d42/d46/d7e/d64/c88 0 2024-09-16T08:57:10.091 INFO:tasks.workunit.client.1.smithi097.stdout:6/467: symlink d1/d3/d16/d86/l8f 0 2024-09-16T08:57:10.094 INFO:tasks.workunit.client.1.smithi097.stdout:3/413: symlink d7/d28/d30/d88/l8c 0 2024-09-16T08:57:10.095 INFO:tasks.workunit.client.1.smithi097.stdout:2/393: mkdir d3/d4/d68 0 2024-09-16T08:57:10.096 INFO:tasks.workunit.client.1.smithi097.stdout:8/377: dwrite d8/dd/d42/f51 [0,4194304] 0 2024-09-16T08:57:10.097 INFO:tasks.workunit.client.1.smithi097.stdout:6/468: rename d1/dc/d1e/c26 to d1/dc/d3c/c90 0 2024-09-16T08:57:10.100 INFO:tasks.workunit.client.1.smithi097.stdout:3/414: rmdir d7/de/d29/d35/d3d/d57/d75 0 2024-09-16T08:57:10.100 INFO:tasks.workunit.client.1.smithi097.stdout:3/415: write d7/de/d29/d35/f22 [794399,10590] 0 2024-09-16T08:57:10.100 INFO:tasks.workunit.client.1.smithi097.stdout:3/416: stat d7/de/d29/f2d 0 2024-09-16T08:57:10.100 INFO:tasks.workunit.client.1.smithi097.stdout:3/417: write d7/de/d6c/f66 [86452,118420] 0 2024-09-16T08:57:10.102 INFO:tasks.workunit.client.1.smithi097.stdout:8/378: link d8/dd/l66 d8/dd/d15/l6f 0 2024-09-16T08:57:10.106 INFO:tasks.workunit.client.1.smithi097.stdout:8/379: rmdir d8/dd/d15/d2c/d4a/d4d 39 2024-09-16T08:57:10.106 INFO:tasks.workunit.client.1.smithi097.stdout:8/380: chown d8/dd/d15/d2c/d4a/d4d/d3d/f57 15783568 1 2024-09-16T08:57:10.111 INFO:tasks.workunit.client.1.smithi097.stdout:5/427: dwrite db/d18/f34 [0,4194304] 0 2024-09-16T08:57:10.111 INFO:tasks.workunit.client.1.smithi097.stdout:5/428: truncate db/d18/d1c/d42/d46/f66 974715 0 2024-09-16T08:57:10.116 INFO:tasks.workunit.client.1.smithi097.stdout:3/418: write d7/de/d29/d35/f27 [2260356,90967] 0 2024-09-16T08:57:10.116 INFO:tasks.workunit.client.1.smithi097.stdout:3/419: truncate d7/f2b 2668722 0 2024-09-16T08:57:10.117 INFO:tasks.workunit.client.1.smithi097.stdout:5/429: mkdir db/d18/d1c/d42/d46/d7e/d64/d7f/d89 0 2024-09-16T08:57:10.117 INFO:tasks.workunit.client.1.smithi097.stdout:5/430: write db/d22/f6a [277861,69576] 0 2024-09-16T08:57:10.117 INFO:tasks.workunit.client.1.smithi097.stdout:5/431: chown db/d6e/d79 14 1 2024-09-16T08:57:10.130 INFO:tasks.workunit.client.1.smithi097.stdout:6/469: dwrite d1/d3/f34 [0,4194304] 0 2024-09-16T08:57:10.131 INFO:tasks.workunit.client.1.smithi097.stdout:3/420: dread d7/de/d3c/d3f/f50 [0,4194304] 0 2024-09-16T08:57:10.133 INFO:tasks.workunit.client.1.smithi097.stdout:8/381: dwrite d8/dd/f1f [0,4194304] 0 2024-09-16T08:57:10.133 INFO:tasks.workunit.client.1.smithi097.stdout:8/382: chown d8/dd/l38 12087391 1 2024-09-16T08:57:10.133 INFO:tasks.workunit.client.1.smithi097.stdout:8/383: readlink d8/dd/l66 0 2024-09-16T08:57:10.133 INFO:tasks.workunit.client.1.smithi097.stdout:8/384: fsync d8/fa 0 2024-09-16T08:57:10.135 INFO:tasks.workunit.client.1.smithi097.stdout:6/470: symlink d1/dc/d1e/l91 0 2024-09-16T08:57:10.135 INFO:tasks.workunit.client.1.smithi097.stdout:6/471: fdatasync d1/d3/d16/f89 0 2024-09-16T08:57:10.135 INFO:tasks.workunit.client.1.smithi097.stdout:3/421: mkdir d7/de/d8d 0 2024-09-16T08:57:10.135 INFO:tasks.workunit.client.1.smithi097.stdout:3/422: truncate d7/de/d3c/d3f/d60/d6b/f86 928190 0 2024-09-16T08:57:10.135 INFO:tasks.workunit.client.1.smithi097.stdout:3/423: chown d7/de/d3c/d3f/f7b 253511094 1 2024-09-16T08:57:10.135 INFO:tasks.workunit.client.1.smithi097.stdout:8/385: creat d8/dd/d15/d2c/d4a/d4e/f70 x:0 0 0 2024-09-16T08:57:10.137 INFO:tasks.workunit.client.1.smithi097.stdout:3/424: creat d7/d28/d30/f8e x:0 0 0 2024-09-16T08:57:10.138 INFO:tasks.workunit.client.1.smithi097.stdout:3/425: symlink d7/d15/l8f 0 2024-09-16T08:57:10.140 INFO:tasks.workunit.client.1.smithi097.stdout:3/426: mkdir d7/de/d8d/d90 0 2024-09-16T08:57:10.140 INFO:tasks.workunit.client.1.smithi097.stdout:9/368: sync 2024-09-16T08:57:10.141 INFO:tasks.workunit.client.1.smithi097.stdout:9/369: read d2/d30/f47 [47659,74694] 0 2024-09-16T08:57:10.142 INFO:tasks.workunit.client.1.smithi097.stdout:3/427: getdents d7/de/d3c/d4f 0 2024-09-16T08:57:10.144 INFO:tasks.workunit.client.1.smithi097.stdout:9/370: unlink d2/fc 0 2024-09-16T08:57:10.144 INFO:tasks.workunit.client.1.smithi097.stdout:8/386: dread d8/dd/f1f [0,4194304] 0 2024-09-16T08:57:10.145 INFO:tasks.workunit.client.1.smithi097.stdout:3/428: rename d7/de/d3c/d3f/f51 to d7/de/d29/d35/d3d/d57/f91 0 2024-09-16T08:57:10.147 INFO:tasks.workunit.client.1.smithi097.stdout:3/429: mkdir d7/de/d8d/d90/d92 0 2024-09-16T08:57:10.147 INFO:tasks.workunit.client.1.smithi097.stdout:3/430: chown d7/de/d6c 319311 1 2024-09-16T08:57:10.148 INFO:tasks.workunit.client.1.smithi097.stdout:9/371: rmdir d2/d5/d72 39 2024-09-16T08:57:10.148 INFO:tasks.workunit.client.1.smithi097.stdout:9/372: getdents d2/d5/d56/d6b 0 2024-09-16T08:57:10.151 INFO:tasks.workunit.client.1.smithi097.stdout:5/432: dwrite db/d18/d1c/d42/d46/d7e/f53 [0,4194304] 0 2024-09-16T08:57:10.154 INFO:tasks.workunit.client.1.smithi097.stdout:5/433: symlink db/d1a/l8a 0 2024-09-16T08:57:10.155 INFO:tasks.workunit.client.1.smithi097.stdout:5/434: readlink db/d18/d20/l77 0 2024-09-16T08:57:10.155 INFO:tasks.workunit.client.1.smithi097.stdout:5/435: fdatasync db/d22/d58/f7d 0 2024-09-16T08:57:10.155 INFO:tasks.workunit.client.1.smithi097.stdout:5/436: creat db/f8b x:0 0 0 2024-09-16T08:57:10.156 INFO:tasks.workunit.client.1.smithi097.stdout:6/472: dwrite d1/dc/d3c/f76 [0,4194304] 0 2024-09-16T08:57:10.156 INFO:tasks.workunit.client.1.smithi097.stdout:6/473: creat d1/d3/d16/d4f/f92 x:0 0 0 2024-09-16T08:57:10.156 INFO:tasks.workunit.client.1.smithi097.stdout:8/387: dread d8/dd/d15/d2c/d4a/d4d/d3d/f5b [0,4194304] 0 2024-09-16T08:57:10.158 INFO:tasks.workunit.client.1.smithi097.stdout:5/437: symlink db/d6e/d79/l8c 0 2024-09-16T08:57:10.160 INFO:tasks.workunit.client.1.smithi097.stdout:9/373: write d2/d5/f21 [3955716,21206] 0 2024-09-16T08:57:10.165 INFO:tasks.workunit.client.1.smithi097.stdout:3/431: dread d7/de/d29/d35/f27 [4194304,4194304] 0 2024-09-16T08:57:10.165 INFO:tasks.workunit.client.1.smithi097.stdout:3/432: truncate d7/de/d29/d35/d3d/f59 797214 0 2024-09-16T08:57:10.166 INFO:tasks.workunit.client.1.smithi097.stdout:3/433: chown d7/de/d8d 40398804 1 2024-09-16T08:57:10.169 INFO:tasks.workunit.client.1.smithi097.stdout:8/388: dread d8/dd/d42/f51 [0,4194304] 0 2024-09-16T08:57:10.171 INFO:tasks.workunit.client.1.smithi097.stdout:5/438: mkdir db/d18/d1c/d42/d46/d7e/d64/d8d 0 2024-09-16T08:57:10.172 INFO:tasks.workunit.client.1.smithi097.stdout:6/474: getdents d1/dc 0 2024-09-16T08:57:10.174 INFO:tasks.workunit.client.1.smithi097.stdout:3/434: mknod d7/de/d29/d35/d3d/d57/d64/c93 0 2024-09-16T08:57:10.175 INFO:tasks.workunit.client.1.smithi097.stdout:8/389: mkdir d8/dd/d71 0 2024-09-16T08:57:10.176 INFO:tasks.workunit.client.1.smithi097.stdout:5/439: truncate db/d18/d1c/d42/d46/d7e/f51 3661292 0 2024-09-16T08:57:10.177 INFO:tasks.workunit.client.1.smithi097.stdout:8/390: write d8/dd/d15/d2c/f4b [134389,53814] 0 2024-09-16T08:57:10.178 INFO:tasks.workunit.client.1.smithi097.stdout:6/475: creat d1/d3/d81/f93 x:0 0 0 2024-09-16T08:57:10.178 INFO:tasks.workunit.client.1.smithi097.stdout:6/476: truncate d1/dc/d3c/f6f 487349 0 2024-09-16T08:57:10.178 INFO:tasks.workunit.client.1.smithi097.stdout:6/477: read d1/d3/d3e/d5a/f32 [341521,52086] 0 2024-09-16T08:57:10.180 INFO:tasks.workunit.client.1.smithi097.stdout:9/374: dread d2/d5/fb [0,4194304] 0 2024-09-16T08:57:10.180 INFO:tasks.workunit.client.1.smithi097.stdout:9/375: truncate d2/d30/f32 4560626 0 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:7/413: sync 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:0/441: sync 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:1/436: sync 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:7/414: stat d4/d7/d7b 0 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:4/411: sync 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:2/394: sync 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:2/395: truncate d3/d4/d8/f25 1159521 0 2024-09-16T08:57:10.189 INFO:tasks.workunit.client.1.smithi097.stdout:2/396: chown d3/d4/d8/d51/l52 102608922 1 2024-09-16T08:57:10.190 INFO:tasks.workunit.client.1.smithi097.stdout:7/415: rename d4/d7/f50 to d4/d9/dc/f90 0 2024-09-16T08:57:10.190 INFO:tasks.workunit.client.1.smithi097.stdout:8/391: write d8/fa [3131119,130489] 0 2024-09-16T08:57:10.190 INFO:tasks.workunit.client.1.smithi097.stdout:8/392: truncate d8/f31 807893 0 2024-09-16T08:57:10.191 INFO:tasks.workunit.client.1.smithi097.stdout:0/442: symlink da/d1a/d26/d47/d97/d84/l9e 0 2024-09-16T08:57:10.192 INFO:tasks.workunit.client.1.smithi097.stdout:1/437: mknod d5/d2c/d3b/d7f/c8e 0 2024-09-16T08:57:10.192 INFO:tasks.workunit.client.1.smithi097.stdout:4/412: mknod d0/d35/d6c/c95 0 2024-09-16T08:57:10.193 INFO:tasks.workunit.client.1.smithi097.stdout:2/397: rename d3/d4/c44 to d3/c69 0 2024-09-16T08:57:10.193 INFO:tasks.workunit.client.1.smithi097.stdout:2/398: dread - d3/d4/dd/d53/f50 zero size 2024-09-16T08:57:10.194 INFO:tasks.workunit.client.1.smithi097.stdout:8/393: mkdir d8/dd/d15/d2c/d72 0 2024-09-16T08:57:10.194 INFO:tasks.workunit.client.1.smithi097.stdout:1/438: symlink d5/l8f 0 2024-09-16T08:57:10.196 INFO:tasks.workunit.client.1.smithi097.stdout:4/413: unlink d0/d39/d71/f79 0 2024-09-16T08:57:10.197 INFO:tasks.workunit.client.1.smithi097.stdout:0/443: rename da/d24/d4c/d80/d42/d5b/d60/d68/f39 to da/d24/d4c/d80/d42/d5b/d60/d73/d85/f9f 0 2024-09-16T08:57:10.197 INFO:tasks.workunit.client.1.smithi097.stdout:0/444: read da/d24/d4c/d80/d42/d5b/d60/d73/d85/f9f [2500994,67014] 0 2024-09-16T08:57:10.198 INFO:tasks.workunit.client.1.smithi097.stdout:0/445: readlink da/d24/d4c/d80/l48 0 2024-09-16T08:57:10.198 INFO:tasks.workunit.client.1.smithi097.stdout:2/399: link d3/d4/c64 d3/d1b/c6a 0 2024-09-16T08:57:10.199 INFO:tasks.workunit.client.1.smithi097.stdout:8/394: link d8/dd/d15/f21 d8/dd/d71/f73 0 2024-09-16T08:57:10.200 INFO:tasks.workunit.client.1.smithi097.stdout:4/414: mknod d0/d7/df/d15/c96 0 2024-09-16T08:57:10.201 INFO:tasks.workunit.client.1.smithi097.stdout:7/416: write d4/d7/d7b/f46 [3316996,81140] 0 2024-09-16T08:57:10.202 INFO:tasks.workunit.client.1.smithi097.stdout:2/400: creat d3/d4/dd/f6b x:0 0 0 2024-09-16T08:57:10.203 INFO:tasks.workunit.client.1.smithi097.stdout:8/395: rename d8/dd/f1f to d8/dd/d15/d2c/f74 0 2024-09-16T08:57:10.205 INFO:tasks.workunit.client.1.smithi097.stdout:2/401: creat d3/d4/dd/d53/f6c x:0 0 0 2024-09-16T08:57:10.205 INFO:tasks.workunit.client.1.smithi097.stdout:2/402: write d3/d4/d9/f5a [411726,48781] 0 2024-09-16T08:57:10.205 INFO:tasks.workunit.client.1.smithi097.stdout:2/403: write d3/d4/dd/d53/f6c [551596,69815] 0 2024-09-16T08:57:10.206 INFO:tasks.workunit.client.1.smithi097.stdout:8/396: symlink d8/dd/d15/d16/l75 0 2024-09-16T08:57:10.210 INFO:tasks.workunit.client.1.smithi097.stdout:0/446: read da/d1a/f63 [4124557,129195] 0 2024-09-16T08:57:10.211 INFO:tasks.workunit.client.1.smithi097.stdout:4/415: dread d0/d7/d52/d26/d3d/f44 [0,4194304] 0 2024-09-16T08:57:10.212 INFO:tasks.workunit.client.1.smithi097.stdout:0/447: dread da/d24/d4c/d80/f62 [0,4194304] 0 2024-09-16T08:57:10.212 INFO:tasks.workunit.client.1.smithi097.stdout:0/448: readlink da/df/d3d/l67 0 2024-09-16T08:57:10.212 INFO:tasks.workunit.client.1.smithi097.stdout:4/416: rmdir d0/d7/d52/d26/d3d 39 2024-09-16T08:57:10.212 INFO:tasks.workunit.client.1.smithi097.stdout:4/417: creat d0/d23/d5f/d7d/f97 x:0 0 0 2024-09-16T08:57:10.213 INFO:tasks.workunit.client.1.smithi097.stdout:0/449: mknod da/d24/d4c/d80/d42/d5b/d60/ca0 0 2024-09-16T08:57:10.214 INFO:tasks.workunit.client.1.smithi097.stdout:5/440: dwrite db/d1a/f38 [0,4194304] 0 2024-09-16T08:57:10.216 INFO:tasks.workunit.client.1.smithi097.stdout:5/441: creat db/d18/d1c/d42/d46/d7e/d76/f8e x:0 0 0 2024-09-16T08:57:10.216 INFO:tasks.workunit.client.1.smithi097.stdout:5/442: write db/d18/d1c/d42/d46/f47 [996897,49639] 0 2024-09-16T08:57:10.216 INFO:tasks.workunit.client.1.smithi097.stdout:3/435: dwrite d7/de/d3c/f79 [0,4194304] 0 2024-09-16T08:57:10.217 INFO:tasks.workunit.client.1.smithi097.stdout:3/436: creat d7/d28/d30/f94 x:0 0 0 2024-09-16T08:57:10.217 INFO:tasks.workunit.client.1.smithi097.stdout:3/437: fdatasync d7/de/d3c/d3f/d60/d6b/f89 0 2024-09-16T08:57:10.217 INFO:tasks.workunit.client.1.smithi097.stdout:5/443: mkdir db/d18/d1c/d42/d46/d7e/d8f 0 2024-09-16T08:57:10.217 INFO:tasks.workunit.client.1.smithi097.stdout:5/444: read - db/f84 zero size 2024-09-16T08:57:10.218 INFO:tasks.workunit.client.1.smithi097.stdout:9/376: dwrite d2/d3f/f4b [0,4194304] 0 2024-09-16T08:57:10.218 INFO:tasks.workunit.client.1.smithi097.stdout:9/377: truncate d2/f4 3284558 0 2024-09-16T08:57:10.219 INFO:tasks.workunit.client.1.smithi097.stdout:2/404: dread d3/d4/d8/f39 [0,4194304] 0 2024-09-16T08:57:10.220 INFO:tasks.workunit.client.1.smithi097.stdout:2/405: fsync d3/d4/dd/f18 0 2024-09-16T08:57:10.220 INFO:tasks.workunit.client.1.smithi097.stdout:3/438: symlink d7/de/d3c/d3f/d60/d6b/l95 0 2024-09-16T08:57:10.220 INFO:tasks.workunit.client.1.smithi097.stdout:1/439: dwrite d5/db/dc/f41 [0,4194304] 0 2024-09-16T08:57:10.223 INFO:tasks.workunit.client.1.smithi097.stdout:8/397: dread d8/dd/d15/d2c/d4a/d4d/f49 [0,4194304] 0 2024-09-16T08:57:10.224 INFO:tasks.workunit.client.1.smithi097.stdout:3/439: mkdir d7/de/d8d/d90/d96 0 2024-09-16T08:57:10.224 INFO:tasks.workunit.client.1.smithi097.stdout:3/440: readlink d7/d15/l1b 0 2024-09-16T08:57:10.229 INFO:tasks.workunit.client.1.smithi097.stdout:2/406: write d3/d4/d9/f1e [1677619,8111] 0 2024-09-16T08:57:10.229 INFO:tasks.workunit.client.1.smithi097.stdout:2/407: creat d3/d4/dd/f6d x:0 0 0 2024-09-16T08:57:10.230 INFO:tasks.workunit.client.1.smithi097.stdout:8/398: read d8/dd/d71/f73 [1147991,128561] 0 2024-09-16T08:57:10.232 INFO:tasks.workunit.client.1.smithi097.stdout:1/440: creat d5/d17/f90 x:0 0 0 2024-09-16T08:57:10.247 INFO:tasks.workunit.client.1.smithi097.stdout:8/399: rmdir d8/dd/d15/d2c/d4a/d4d/d3d 39 2024-09-16T08:57:10.247 INFO:tasks.workunit.client.1.smithi097.stdout:1/441: read f4 [370402,80203] 0 2024-09-16T08:57:10.249 INFO:tasks.workunit.client.1.smithi097.stdout:8/400: creat d8/dd/d15/d2c/d4a/d4d/d62/f76 x:0 0 0 2024-09-16T08:57:10.250 INFO:tasks.workunit.client.1.smithi097.stdout:1/442: mknod d5/d17/d54/d6b/d71/c91 0 2024-09-16T08:57:10.251 INFO:tasks.workunit.client.1.smithi097.stdout:8/401: creat d8/dd/d15/d2c/d4a/d4d/d3f/f77 x:0 0 0 2024-09-16T08:57:10.253 INFO:tasks.workunit.client.1.smithi097.stdout:1/443: getdents d5/d2c 0 2024-09-16T08:57:10.253 INFO:tasks.workunit.client.1.smithi097.stdout:1/444: stat d5/l8f 0 2024-09-16T08:57:10.253 INFO:tasks.workunit.client.1.smithi097.stdout:1/445: chown d5/d17/f24 633 1 2024-09-16T08:57:10.253 INFO:tasks.workunit.client.1.smithi097.stdout:1/446: chown d5/db/l50 6 1 2024-09-16T08:57:10.253 INFO:tasks.workunit.client.1.smithi097.stdout:2/408: dread d3/f32 [0,4194304] 0 2024-09-16T08:57:10.253 INFO:tasks.workunit.client.1.smithi097.stdout:2/409: creat d3/d4/f6e x:0 0 0 2024-09-16T08:57:10.254 INFO:tasks.workunit.client.1.smithi097.stdout:8/402: rename d8/dd/l1a to d8/dd/d15/d16/l78 0 2024-09-16T08:57:10.254 INFO:tasks.workunit.client.1.smithi097.stdout:1/447: creat d5/d2c/f92 x:0 0 0 2024-09-16T08:57:10.255 INFO:tasks.workunit.client.1.smithi097.stdout:1/448: creat d5/db/d25/f93 x:0 0 0 2024-09-16T08:57:10.256 INFO:tasks.workunit.client.1.smithi097.stdout:8/403: dread d8/dd/d71/f73 [0,4194304] 0 2024-09-16T08:57:10.257 INFO:tasks.workunit.client.1.smithi097.stdout:2/410: unlink d3/d4/d9/f1e 0 2024-09-16T08:57:10.264 INFO:tasks.workunit.client.1.smithi097.stdout:0/450: dwrite da/d1a/d26/f6b [0,4194304] 0 2024-09-16T08:57:10.264 INFO:tasks.workunit.client.1.smithi097.stdout:0/451: fsync f8 0 2024-09-16T08:57:10.265 INFO:tasks.workunit.client.1.smithi097.stdout:5/445: dwrite db/f14 [0,4194304] 0 2024-09-16T08:57:10.265 INFO:tasks.workunit.client.1.smithi097.stdout:9/378: dwrite d2/f31 [0,4194304] 0 2024-09-16T08:57:10.265 INFO:tasks.workunit.client.1.smithi097.stdout:3/441: dwrite d7/f2b [0,4194304] 0 2024-09-16T08:57:10.267 INFO:tasks.workunit.client.1.smithi097.stdout:0/452: write da/df/d3d/f66 [778506,50103] 0 2024-09-16T08:57:10.267 INFO:tasks.workunit.client.1.smithi097.stdout:5/446: read db/d22/f4b [262867,76653] 0 2024-09-16T08:57:10.267 INFO:tasks.workunit.client.1.smithi097.stdout:5/447: chown db/d18/d1c 5 1 2024-09-16T08:57:10.267 INFO:tasks.workunit.client.1.smithi097.stdout:5/448: read db/d18/d20/d2b/f75 [34722,59710] 0 2024-09-16T08:57:10.267 INFO:tasks.workunit.client.1.smithi097.stdout:2/411: mknod d3/d4/dd/c6f 0 2024-09-16T08:57:10.267 INFO:tasks.workunit.client.1.smithi097.stdout:2/412: getdents d3/d4/d9/d67 0 2024-09-16T08:57:10.268 INFO:tasks.workunit.client.1.smithi097.stdout:2/413: chown d3/d4/d8/c2e 115275752 1 2024-09-16T08:57:10.272 INFO:tasks.workunit.client.1.smithi097.stdout:9/379: creat d2/d30/d63/f77 x:0 0 0 2024-09-16T08:57:10.273 INFO:tasks.workunit.client.1.smithi097.stdout:3/442: mkdir d7/d28/d97 0 2024-09-16T08:57:10.273 INFO:tasks.workunit.client.1.smithi097.stdout:3/443: write d7/de/d3c/d4f/f7f [891478,65821] 0 2024-09-16T08:57:10.274 INFO:tasks.workunit.client.1.smithi097.stdout:0/453: creat da/d24/d4b/fa1 x:0 0 0 2024-09-16T08:57:10.274 INFO:tasks.workunit.client.1.smithi097.stdout:0/454: dread - da/d24/d4c/d80/d42/d5b/d60/d68/f5d zero size 2024-09-16T08:57:10.275 INFO:tasks.workunit.client.1.smithi097.stdout:5/449: mkdir db/d18/d1c/d42/d46/d70/d90 0 2024-09-16T08:57:10.275 INFO:tasks.workunit.client.1.smithi097.stdout:5/450: chown db/d1a/c69 7736937 1 2024-09-16T08:57:10.275 INFO:tasks.workunit.client.1.smithi097.stdout:5/451: fsync db/f14 0 2024-09-16T08:57:10.276 INFO:tasks.workunit.client.1.smithi097.stdout:1/449: dwrite d5/db/f74 [0,4194304] 0 2024-09-16T08:57:10.276 INFO:tasks.workunit.client.1.smithi097.stdout:9/380: mknod d2/d5/d56/c78 0 2024-09-16T08:57:10.278 INFO:tasks.workunit.client.1.smithi097.stdout:0/455: getdents da/d1a/d26/d47/d97/d84 0 2024-09-16T08:57:10.278 INFO:tasks.workunit.client.1.smithi097.stdout:0/456: write da/d1a/f58 [585212,23050] 0 2024-09-16T08:57:10.281 INFO:tasks.workunit.client.1.smithi097.stdout:2/414: getdents d3/d4/d8/d34 0 2024-09-16T08:57:10.281 INFO:tasks.workunit.client.1.smithi097.stdout:2/415: stat d3/d4/dd/d53/f50 0 2024-09-16T08:57:10.283 INFO:tasks.workunit.client.1.smithi097.stdout:3/444: rename d7/de/d3c/d3f/c42 to d7/c98 0 2024-09-16T08:57:10.283 INFO:tasks.workunit.client.1.smithi097.stdout:3/445: chown d7/de/d29/d35/d3d/c58 123 1 2024-09-16T08:57:10.284 INFO:tasks.workunit.client.1.smithi097.stdout:3/446: creat d7/de/d3c/f99 x:0 0 0 2024-09-16T08:57:10.284 INFO:tasks.workunit.client.1.smithi097.stdout:3/447: fsync d7/fa 0 2024-09-16T08:57:10.284 INFO:tasks.workunit.client.1.smithi097.stdout:9/381: write d2/f50 [2509617,39546] 0 2024-09-16T08:57:10.284 INFO:tasks.workunit.client.1.smithi097.stdout:9/382: write d2/d5/d3a/d3c/f54 [1140148,92924] 0 2024-09-16T08:57:10.285 INFO:tasks.workunit.client.1.smithi097.stdout:9/383: truncate d2/d30/f47 424854 0 2024-09-16T08:57:10.287 INFO:tasks.workunit.client.1.smithi097.stdout:8/404: rmdir d8/dd/d15/d2c 39 2024-09-16T08:57:10.288 INFO:tasks.workunit.client.1.smithi097.stdout:1/450: dread d5/f21 [0,4194304] 0 2024-09-16T08:57:10.288 INFO:tasks.workunit.client.1.smithi097.stdout:1/451: write d5/db/dc/d19/f38 [1548105,27937] 0 2024-09-16T08:57:10.288 INFO:tasks.workunit.client.1.smithi097.stdout:1/452: dread - d5/d17/f90 zero size 2024-09-16T08:57:10.288 INFO:tasks.workunit.client.1.smithi097.stdout:1/453: write d5/db/dc/d19/f29 [360422,57982] 0 2024-09-16T08:57:10.288 INFO:tasks.workunit.client.1.smithi097.stdout:1/454: dread - d5/db/d25/f93 zero size 2024-09-16T08:57:10.288 INFO:tasks.workunit.client.1.smithi097.stdout:0/457: dread da/d1a/d41/f50 [0,4194304] 0 2024-09-16T08:57:10.288 INFO:tasks.workunit.client.1.smithi097.stdout:0/458: write da/d24/d4c/d80/d3b/f4e [966637,98636] 0 2024-09-16T08:57:10.289 INFO:tasks.workunit.client.1.smithi097.stdout:0/459: readlink da/d24/d4c/d80/d42/l54 0 2024-09-16T08:57:10.290 INFO:tasks.workunit.client.1.smithi097.stdout:9/384: mkdir d2/d3f/d62/d79 0 2024-09-16T08:57:10.292 INFO:tasks.workunit.client.1.smithi097.stdout:3/448: symlink d7/de/d29/l9a 0 2024-09-16T08:57:10.292 INFO:tasks.workunit.client.1.smithi097.stdout:3/449: truncate d7/de/d3c/f73 128407 0 2024-09-16T08:57:10.292 INFO:tasks.workunit.client.1.smithi097.stdout:3/450: chown d7/de/d29/d35/d3d/d57/l83 59 1 2024-09-16T08:57:10.293 INFO:tasks.workunit.client.1.smithi097.stdout:1/455: mkdir d5/d17/d54/d6b/d71/d94 0 2024-09-16T08:57:10.294 INFO:tasks.workunit.client.1.smithi097.stdout:0/460: creat da/d1a/d26/d47/d97/fa2 x:0 0 0 2024-09-16T08:57:10.294 INFO:tasks.workunit.client.1.smithi097.stdout:0/461: creat da/d24/d4b/fa3 x:0 0 0 2024-09-16T08:57:10.295 INFO:tasks.workunit.client.1.smithi097.stdout:9/385: rename d2/d5/d72/f76 to d2/d5/d3a/d3c/d73/f7a 0 2024-09-16T08:57:10.296 INFO:tasks.workunit.client.1.smithi097.stdout:3/451: symlink d7/d28/d63/d70/l9b 0 2024-09-16T08:57:10.297 INFO:tasks.workunit.client.1.smithi097.stdout:1/456: unlink d5/d2c/d3b/f5a 0 2024-09-16T08:57:10.299 INFO:tasks.workunit.client.1.smithi097.stdout:0/462: rename da/d24/d4c/d80/f62 to da/d24/d4c/d80/d42/d5b/d60/d73/fa4 0 2024-09-16T08:57:10.300 INFO:tasks.workunit.client.1.smithi097.stdout:3/452: mkdir d7/de/d9c 0 2024-09-16T08:57:10.300 INFO:tasks.workunit.client.1.smithi097.stdout:3/453: chown d7/de/d29/d35/d3d/d57 1432005 1 2024-09-16T08:57:10.301 INFO:tasks.workunit.client.1.smithi097.stdout:3/454: chown d7/de/d29/d35/d3d/d57 49005497 1 2024-09-16T08:57:10.301 INFO:tasks.workunit.client.1.smithi097.stdout:3/455: creat d7/de/d8d/d90/f9d x:0 0 0 2024-09-16T08:57:10.302 INFO:tasks.workunit.client.1.smithi097.stdout:3/456: write d7/de/d6c/f66 [2041089,12005] 0 2024-09-16T08:57:10.306 INFO:tasks.workunit.client.1.smithi097.stdout:2/416: dwrite d3/d4/d9/f5a [0,4194304] 0 2024-09-16T08:57:10.306 INFO:tasks.workunit.client.1.smithi097.stdout:2/417: write d3/d4/d8/d51/f5d [1693359,70164] 0 2024-09-16T08:57:10.307 INFO:tasks.workunit.client.1.smithi097.stdout:5/452: dwrite fa [8388608,4194304] 0 2024-09-16T08:57:10.311 INFO:tasks.workunit.client.1.smithi097.stdout:8/405: dwrite d8/f1e [4194304,4194304] 0 2024-09-16T08:57:10.311 INFO:tasks.workunit.client.1.smithi097.stdout:5/453: rename db/d18/d1c/l44 to db/d6e/d81/l91 0 2024-09-16T08:57:10.312 INFO:tasks.workunit.client.1.smithi097.stdout:9/386: write d2/fe [2658920,44804] 0 2024-09-16T08:57:10.313 INFO:tasks.workunit.client.1.smithi097.stdout:8/406: symlink d8/dd/d15/l79 0 2024-09-16T08:57:10.313 INFO:tasks.workunit.client.1.smithi097.stdout:2/418: dread d3/d4/d8/d34/d48/f60 [0,4194304] 0 2024-09-16T08:57:10.315 INFO:tasks.workunit.client.1.smithi097.stdout:2/419: creat d3/d4/d8/d34/f70 x:0 0 0 2024-09-16T08:57:10.322 INFO:tasks.workunit.client.1.smithi097.stdout:8/407: dread d8/fa [0,4194304] 0 2024-09-16T08:57:10.323 INFO:tasks.workunit.client.1.smithi097.stdout:8/408: unlink d8/dd/d15/d2c/d4a/d4d/d3d/f57 0 2024-09-16T08:57:10.324 INFO:tasks.workunit.client.1.smithi097.stdout:8/409: mknod d8/dd/d15/d2c/d4a/d4d/d3d/c7a 0 2024-09-16T08:57:10.324 INFO:tasks.workunit.client.1.smithi097.stdout:8/410: write d8/dd/d15/d2c/d4a/d4e/f70 [253619,63206] 0 2024-09-16T08:57:10.325 INFO:tasks.workunit.client.1.smithi097.stdout:5/454: dread db/d22/d58/f7d [0,4194304] 0 2024-09-16T08:57:10.325 INFO:tasks.workunit.client.1.smithi097.stdout:8/411: mkdir d8/dd/d15/d2c/d4a/d4d/d3d/d7b 0 2024-09-16T08:57:10.328 INFO:tasks.workunit.client.1.smithi097.stdout:9/387: dread d2/d5/f11 [0,4194304] 0 2024-09-16T08:57:10.328 INFO:tasks.workunit.client.1.smithi097.stdout:9/388: fsync d2/d5/d3a/d3c/f53 0 2024-09-16T08:57:10.330 INFO:tasks.workunit.client.1.smithi097.stdout:9/389: creat d2/d5/d3a/d3c/d73/f7b x:0 0 0 2024-09-16T08:57:10.330 INFO:tasks.workunit.client.1.smithi097.stdout:9/390: write d2/d3f/f4b [4802346,86873] 0 2024-09-16T08:57:10.333 INFO:tasks.workunit.client.1.smithi097.stdout:3/457: dwrite d7/de/d3c/d4f/f80 [0,4194304] 0 2024-09-16T08:57:10.333 INFO:tasks.workunit.client.1.smithi097.stdout:3/458: truncate d7/de/d8d/d90/f9d 925821 0 2024-09-16T08:57:10.334 INFO:tasks.workunit.client.1.smithi097.stdout:0/463: dwrite da/d24/d4c/d80/d42/d5b/f7b [0,4194304] 0 2024-09-16T08:57:10.334 INFO:tasks.workunit.client.1.smithi097.stdout:1/457: dwrite d5/d2c/f2e [0,4194304] 0 2024-09-16T08:57:10.339 INFO:tasks.workunit.client.1.smithi097.stdout:8/412: write d8/dd/d15/f28 [2062399,61543] 0 2024-09-16T08:57:10.343 INFO:tasks.workunit.client.1.smithi097.stdout:0/464: mknod da/d24/d4b/ca5 0 2024-09-16T08:57:10.343 INFO:tasks.workunit.client.1.smithi097.stdout:0/465: fdatasync da/d24/d4b/fa1 0 2024-09-16T08:57:10.346 INFO:tasks.workunit.client.1.smithi097.stdout:1/458: mkdir d5/d14/d95 0 2024-09-16T08:57:10.347 INFO:tasks.workunit.client.1.smithi097.stdout:1/459: write d5/db/dc/f48 [135446,117374] 0 2024-09-16T08:57:10.347 INFO:tasks.workunit.client.1.smithi097.stdout:1/460: write d5/d2c/d3b/f61 [2106121,74537] 0 2024-09-16T08:57:10.347 INFO:tasks.workunit.client.1.smithi097.stdout:1/461: readlink d5/l8f 0 2024-09-16T08:57:10.347 INFO:tasks.workunit.client.1.smithi097.stdout:1/462: chown d5/d17/f90 1800 1 2024-09-16T08:57:10.347 INFO:tasks.workunit.client.1.smithi097.stdout:3/459: getdents d7/de/d3c/d3f/d60/d7a 0 2024-09-16T08:57:10.347 INFO:tasks.workunit.client.1.smithi097.stdout:3/460: readlink d7/de/l24 0 2024-09-16T08:57:10.348 INFO:tasks.workunit.client.1.smithi097.stdout:8/413: mkdir d8/dd/d15/d16/d7c 0 2024-09-16T08:57:10.348 INFO:tasks.workunit.client.1.smithi097.stdout:8/414: stat d8/dd/d15/d2c/d4a/d4d/d3d/d7b 0 2024-09-16T08:57:10.353 INFO:tasks.workunit.client.1.smithi097.stdout:0/466: unlink da/d24/d4c/d80/d42/d5b/d60/l61 0 2024-09-16T08:57:10.353 INFO:tasks.workunit.client.1.smithi097.stdout:0/467: stat da/f1b 0 2024-09-16T08:57:10.354 INFO:tasks.workunit.client.1.smithi097.stdout:5/455: dwrite db/d18/d1c/d42/d46/d7e/f51 [0,4194304] 0 2024-09-16T08:57:10.354 INFO:tasks.workunit.client.1.smithi097.stdout:5/456: chown db/d1a/c69 1 1 2024-09-16T08:57:10.355 INFO:tasks.workunit.client.1.smithi097.stdout:5/457: chown db/d18/d1c/d42/d46/f6c 920315 1 2024-09-16T08:57:10.355 INFO:tasks.workunit.client.1.smithi097.stdout:1/463: link d5/d14/l7a d5/d17/d54/l96 0 2024-09-16T08:57:10.355 INFO:tasks.workunit.client.1.smithi097.stdout:9/391: dwrite d2/d3f/f43 [0,4194304] 0 2024-09-16T08:57:10.355 INFO:tasks.workunit.client.1.smithi097.stdout:9/392: fdatasync d2/f31 0 2024-09-16T08:57:10.355 INFO:tasks.workunit.client.1.smithi097.stdout:8/415: creat d8/dd/d71/f7d x:0 0 0 2024-09-16T08:57:10.355 INFO:tasks.workunit.client.1.smithi097.stdout:8/416: stat d8/fc 0 2024-09-16T08:57:10.356 INFO:tasks.workunit.client.1.smithi097.stdout:0/468: symlink da/d24/d4c/la6 0 2024-09-16T08:57:10.358 INFO:tasks.workunit.client.1.smithi097.stdout:5/458: rename db/d18/d1c/d42/d46/f66 to db/d1a/f92 0 2024-09-16T08:57:10.363 INFO:tasks.workunit.client.1.smithi097.stdout:9/393: symlink d2/d5/d72/l7c 0 2024-09-16T08:57:10.363 INFO:tasks.workunit.client.1.smithi097.stdout:9/394: rename d2 to d2/d3f/d7d 22 2024-09-16T08:57:10.365 INFO:tasks.workunit.client.1.smithi097.stdout:0/469: creat da/df/d13/fa7 x:0 0 0 2024-09-16T08:57:10.366 INFO:tasks.workunit.client.1.smithi097.stdout:5/459: symlink db/d18/d1c/d42/d46/l93 0 2024-09-16T08:57:10.367 INFO:tasks.workunit.client.1.smithi097.stdout:5/460: write db/fe [13544185,74943] 0 2024-09-16T08:57:10.367 INFO:tasks.workunit.client.1.smithi097.stdout:5/461: fsync db/d18/d20/d2b/f87 0 2024-09-16T08:57:10.367 INFO:tasks.workunit.client.1.smithi097.stdout:1/464: rmdir d5/db/d25 39 2024-09-16T08:57:10.368 INFO:tasks.workunit.client.1.smithi097.stdout:9/395: truncate d2/d30/f3b 638829 0 2024-09-16T08:57:10.368 INFO:tasks.workunit.client.1.smithi097.stdout:9/396: fsync d2/d30/f47 0 2024-09-16T08:57:10.368 INFO:tasks.workunit.client.1.smithi097.stdout:9/397: chown d2/d5/c26 3 1 2024-09-16T08:57:10.369 INFO:tasks.workunit.client.1.smithi097.stdout:0/470: mknod da/d1a/d26/d47/d97/d84/ca8 0 2024-09-16T08:57:10.369 INFO:tasks.workunit.client.1.smithi097.stdout:1/465: dread d5/db/f5d [0,4194304] 0 2024-09-16T08:57:10.370 INFO:tasks.workunit.client.1.smithi097.stdout:9/398: dread d2/d5/d56/f57 [0,4194304] 0 2024-09-16T08:57:10.370 INFO:tasks.workunit.client.1.smithi097.stdout:5/462: symlink db/d18/d1c/l94 0 2024-09-16T08:57:10.370 INFO:tasks.workunit.client.1.smithi097.stdout:5/463: chown db/d18/d1c/l2a 1 1 2024-09-16T08:57:10.370 INFO:tasks.workunit.client.1.smithi097.stdout:5/464: write db/d18/d1c/d42/d46/f6c [1026021,51688] 0 2024-09-16T08:57:10.371 INFO:tasks.workunit.client.1.smithi097.stdout:1/466: link d5/l51 d5/d2c/d3b/d7f/l97 0 2024-09-16T08:57:10.372 INFO:tasks.workunit.client.1.smithi097.stdout:9/399: mknod d2/d3f/d62/d79/c7e 0 2024-09-16T08:57:10.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:10 smithi031 ceph-mon[27844]: Standby manager daemon smithi031.dcmqpv started 2024-09-16T08:57:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/crt"}]: dispatch 2024-09-16T08:57:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:57:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/key"}]: dispatch 2024-09-16T08:57:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:57:10.373 INFO:tasks.workunit.client.1.smithi097.stdout:5/465: mkdir db/d6e/d95 0 2024-09-16T08:57:10.373 INFO:tasks.workunit.client.1.smithi097.stdout:5/466: chown db/c45 22081 1 2024-09-16T08:57:10.374 INFO:tasks.workunit.client.1.smithi097.stdout:5/467: chown db/d18/d1c/d42/d46/d7e/d64/d7f/f85 6834493 1 2024-09-16T08:57:10.374 INFO:tasks.workunit.client.1.smithi097.stdout:5/468: creat db/d22/f96 x:0 0 0 2024-09-16T08:57:10.374 INFO:tasks.workunit.client.1.smithi097.stdout:1/467: rename d5/db/dc/d19/f29 to d5/f98 0 2024-09-16T08:57:10.378 INFO:tasks.workunit.client.1.smithi097.stdout:5/469: getdents db/d18/d1c/d42/d46/d70 0 2024-09-16T08:57:10.380 INFO:tasks.workunit.client.1.smithi097.stdout:9/400: rmdir d2/d5/d3a/d3c 39 2024-09-16T08:57:10.384 INFO:tasks.workunit.client.1.smithi097.stdout:9/401: dread d2/d5/f11 [0,4194304] 0 2024-09-16T08:57:10.385 INFO:tasks.workunit.client.1.smithi097.stdout:9/402: getdents d2/d5 0 2024-09-16T08:57:10.391 INFO:tasks.workunit.client.1.smithi097.stdout:3/461: dwrite d7/de/d6c/f4a [0,4194304] 0 2024-09-16T08:57:10.393 INFO:tasks.workunit.client.1.smithi097.stdout:5/470: dread db/d18/d1c/d42/d46/d7e/f53 [0,4194304] 0 2024-09-16T08:57:10.393 INFO:tasks.workunit.client.1.smithi097.stdout:8/417: dwrite d8/dd/f6a [0,4194304] 0 2024-09-16T08:57:10.393 INFO:tasks.workunit.client.1.smithi097.stdout:8/418: creat d8/dd/f7e x:0 0 0 2024-09-16T08:57:10.393 INFO:tasks.workunit.client.1.smithi097.stdout:8/419: stat d8/dd/d15/d2c/f41 0 2024-09-16T08:57:10.393 INFO:tasks.workunit.client.1.smithi097.stdout:8/420: readlink d8/dd/d15/d2c/d48/l69 0 2024-09-16T08:57:10.394 INFO:tasks.workunit.client.1.smithi097.stdout:0/471: dwrite da/d1a/f49 [0,4194304] 0 2024-09-16T08:57:10.400 INFO:tasks.workunit.client.1.smithi097.stdout:8/421: creat d8/f7f x:0 0 0 2024-09-16T08:57:10.400 INFO:tasks.workunit.client.1.smithi097.stdout:8/422: creat d8/dd/f80 x:0 0 0 2024-09-16T08:57:10.406 INFO:tasks.workunit.client.1.smithi097.stdout:5/471: write fa [7343533,125285] 0 2024-09-16T08:57:10.408 INFO:tasks.workunit.client.1.smithi097.stdout:1/468: getdents d5/db/dc/d19 0 2024-09-16T08:57:10.408 INFO:tasks.workunit.client.1.smithi097.stdout:1/469: readlink d5/d17/d54/l56 0 2024-09-16T08:57:10.411 INFO:tasks.workunit.client.1.smithi097.stdout:3/462: getdents d7/de 0 2024-09-16T08:57:10.412 INFO:tasks.workunit.client.1.smithi097.stdout:3/463: dread d7/de/d29/f49 [0,4194304] 0 2024-09-16T08:57:10.413 INFO:tasks.workunit.client.1.smithi097.stdout:8/423: symlink d8/dd/d15/d16/d7c/l81 0 2024-09-16T08:57:10.413 INFO:tasks.workunit.client.1.smithi097.stdout:1/470: creat d5/d17/d54/d6b/d71/d94/f99 x:0 0 0 2024-09-16T08:57:10.414 INFO:tasks.workunit.client.1.smithi097.stdout:0/472: dwrite da/d24/d4c/d80/d42/d5b/d60/d68/f74 [0,4194304] 0 2024-09-16T08:57:10.414 INFO:tasks.workunit.client.1.smithi097.stdout:3/464: link d7/de/d3c/d3f/l4c d7/de/d3c/d4f/l9e 0 2024-09-16T08:57:10.415 INFO:tasks.workunit.client.1.smithi097.stdout:3/465: chown d7/c26 53393 1 2024-09-16T08:57:10.417 INFO:tasks.workunit.client.1.smithi097.stdout:1/471: truncate d5/f12 3583022 0 2024-09-16T08:57:10.417 INFO:tasks.workunit.client.1.smithi097.stdout:1/472: write d5/d2c/f5e [478412,53361] 0 2024-09-16T08:57:10.420 INFO:tasks.workunit.client.1.smithi097.stdout:0/473: creat da/df/d8a/fa9 x:0 0 0 2024-09-16T08:57:10.420 INFO:tasks.workunit.client.1.smithi097.stdout:0/474: fsync da/d1a/f22 0 2024-09-16T08:57:10.421 INFO:tasks.workunit.client.1.smithi097.stdout:2/420: sync 2024-09-16T08:57:10.421 INFO:tasks.workunit.client.1.smithi097.stdout:7/417: sync 2024-09-16T08:57:10.421 INFO:tasks.workunit.client.1.smithi097.stdout:4/418: sync 2024-09-16T08:57:10.422 INFO:tasks.workunit.client.1.smithi097.stdout:6/478: sync 2024-09-16T08:57:10.422 INFO:tasks.workunit.client.1.smithi097.stdout:6/479: creat d1/d3/d16/d4f/f94 x:0 0 0 2024-09-16T08:57:10.423 INFO:tasks.workunit.client.1.smithi097.stdout:3/466: mkdir d7/de/d8d/d90/d96/d9f 0 2024-09-16T08:57:10.423 INFO:tasks.workunit.client.1.smithi097.stdout:3/467: chown d7/de/d8d 8579 1 2024-09-16T08:57:10.423 INFO:tasks.workunit.client.1.smithi097.stdout:1/473: rename d5/f98 to d5/d2c/d3b/f9a 0 2024-09-16T08:57:10.423 INFO:tasks.workunit.client.1.smithi097.stdout:1/474: fdatasync d5/db/f3c 0 2024-09-16T08:57:10.424 INFO:tasks.workunit.client.1.smithi097.stdout:8/424: getdents d8/dd 0 2024-09-16T08:57:10.424 INFO:tasks.workunit.client.1.smithi097.stdout:8/425: chown d8/dd/d15/d16/f3c 2415 1 2024-09-16T08:57:10.425 INFO:tasks.workunit.client.1.smithi097.stdout:0/475: creat da/d24/d4c/d80/d3b/d86/faa x:0 0 0 2024-09-16T08:57:10.426 INFO:tasks.workunit.client.1.smithi097.stdout:4/419: rmdir d0/d35 39 2024-09-16T08:57:10.427 INFO:tasks.workunit.client.1.smithi097.stdout:7/418: rmdir d4/d1b/d41/d86 0 2024-09-16T08:57:10.432 INFO:tasks.workunit.client.1.smithi097.stdout:2/421: rename d3/d4/d8/d51/f5d to d3/f71 0 2024-09-16T08:57:10.433 INFO:tasks.workunit.client.1.smithi097.stdout:8/426: creat d8/dd/f82 x:0 0 0 2024-09-16T08:57:10.434 INFO:tasks.workunit.client.1.smithi097.stdout:0/476: mkdir da/d24/d4c/d80/dab 0 2024-09-16T08:57:10.434 INFO:tasks.workunit.client.1.smithi097.stdout:4/420: write d0/d7/d52/f78 [2332290,34263] 0 2024-09-16T08:57:10.437 INFO:tasks.workunit.client.1.smithi097.stdout:6/480: rename d1/d3/d3e/d5a/l71 to d1/d3/d16/d86/l95 0 2024-09-16T08:57:10.438 INFO:tasks.workunit.client.1.smithi097.stdout:2/422: creat d3/d4/d68/f72 x:0 0 0 2024-09-16T08:57:10.440 INFO:tasks.workunit.client.1.smithi097.stdout:0/477: mkdir da/df/d8a/dac 0 2024-09-16T08:57:10.444 INFO:tasks.workunit.client.1.smithi097.stdout:1/475: rename d5/db/f20 to d5/d2c/f9b 0 2024-09-16T08:57:10.444 INFO:tasks.workunit.client.1.smithi097.stdout:1/476: write d5/db/dc/f33 [3133119,34128] 0 2024-09-16T08:57:10.445 INFO:tasks.workunit.client.1.smithi097.stdout:8/427: dread d8/dd/d15/f28 [0,4194304] 0 2024-09-16T08:57:10.445 INFO:tasks.workunit.client.1.smithi097.stdout:8/428: dread - d8/dd/f82 zero size 2024-09-16T08:57:10.445 INFO:tasks.workunit.client.1.smithi097.stdout:8/429: fdatasync d8/dd/d15/d2c/f74 0 2024-09-16T08:57:10.446 INFO:tasks.workunit.client.1.smithi097.stdout:6/481: unlink d1/d3/f3d 0 2024-09-16T08:57:10.448 INFO:tasks.workunit.client.1.smithi097.stdout:2/423: fdatasync d3/d4/d8/d34/d48/f4d 0 2024-09-16T08:57:10.450 INFO:tasks.workunit.client.1.smithi097.stdout:8/430: dread d8/f13 [0,4194304] 0 2024-09-16T08:57:10.450 INFO:tasks.workunit.client.1.smithi097.stdout:6/482: rename d1/d3/d3e/d5a/f4a to d1/d3/d3e/d5a/f96 0 2024-09-16T08:57:10.450 INFO:tasks.workunit.client.1.smithi097.stdout:6/483: dread - d1/d3/d16/d4f/f94 zero size 2024-09-16T08:57:10.452 INFO:tasks.workunit.client.1.smithi097.stdout:2/424: rename d3/d4/dd/c6f to d3/d4/d9/d67/c73 0 2024-09-16T08:57:10.453 INFO:tasks.workunit.client.1.smithi097.stdout:2/425: fdatasync d3/d4/d8/d34/f70 0 2024-09-16T08:57:10.454 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:10 smithi097 ceph-mon[35478]: Standby manager daemon smithi031.dcmqpv started 2024-09-16T08:57:10.454 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/crt"}]: dispatch 2024-09-16T08:57:10.454 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:57:10.454 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/key"}]: dispatch 2024-09-16T08:57:10.454 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/3225052862' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:57:10.455 INFO:tasks.workunit.client.1.smithi097.stdout:1/477: dread d5/db/dc/f7d [0,4194304] 0 2024-09-16T08:57:10.457 INFO:tasks.workunit.client.1.smithi097.stdout:4/421: dwrite d0/d7/df/f37 [0,4194304] 0 2024-09-16T08:57:10.457 INFO:tasks.workunit.client.1.smithi097.stdout:6/484: rename d1/d3/d3e/d5a/l45 to d1/d3/l97 0 2024-09-16T08:57:10.458 INFO:tasks.workunit.client.1.smithi097.stdout:7/419: dwrite d4/d9/d1c/f38 [0,4194304] 0 2024-09-16T08:57:10.458 INFO:tasks.workunit.client.1.smithi097.stdout:7/420: fdatasync d4/d7/d7b/f7c 0 2024-09-16T08:57:10.458 INFO:tasks.workunit.client.1.smithi097.stdout:7/421: truncate d4/d7/d30/f5c 1729442 0 2024-09-16T08:57:10.459 INFO:tasks.workunit.client.1.smithi097.stdout:2/426: rename d3/l66 to d3/l74 0 2024-09-16T08:57:10.460 INFO:tasks.workunit.client.1.smithi097.stdout:4/422: creat d0/d7/df/d83/f98 x:0 0 0 2024-09-16T08:57:10.463 INFO:tasks.workunit.client.1.smithi097.stdout:4/423: symlink d0/d7/d52/d26/l99 0 2024-09-16T08:57:10.463 INFO:tasks.workunit.client.1.smithi097.stdout:0/478: dwrite da/df/d3d/f44 [0,4194304] 0 2024-09-16T08:57:10.466 INFO:tasks.workunit.client.1.smithi097.stdout:0/479: mkdir da/d24/d4c/d80/d42/d5b/d60/d73/d85/dad 0 2024-09-16T08:57:10.468 INFO:tasks.workunit.client.1.smithi097.stdout:0/480: mknod da/d24/d4c/d80/d42/d5b/d60/d73/cae 0 2024-09-16T08:57:10.468 INFO:tasks.workunit.client.1.smithi097.stdout:0/481: dread - da/d24/d4c/d80/d42/d5b/d60/d73/d85/f8d zero size 2024-09-16T08:57:10.469 INFO:tasks.workunit.client.1.smithi097.stdout:4/424: read d0/d35/f38 [199726,95928] 0 2024-09-16T08:57:10.470 INFO:tasks.workunit.client.1.smithi097.stdout:0/482: creat da/d1a/d26/d47/faf x:0 0 0 2024-09-16T08:57:10.470 INFO:tasks.workunit.client.1.smithi097.stdout:0/483: fsync f6 0 2024-09-16T08:57:10.470 INFO:tasks.workunit.client.1.smithi097.stdout:0/484: stat da/d24/d4c/d80/f3c 0 2024-09-16T08:57:10.470 INFO:tasks.workunit.client.1.smithi097.stdout:0/485: chown da/df/d13/c7d 19516 1 2024-09-16T08:57:10.470 INFO:tasks.workunit.client.1.smithi097.stdout:0/486: chown da/d24/d4c/d80/d42/d5b/d60/d73/f77 673556 1 2024-09-16T08:57:10.476 INFO:tasks.workunit.client.1.smithi097.stdout:2/427: write d3/d4/dd/d53/f58 [726488,111425] 0 2024-09-16T08:57:10.477 INFO:tasks.workunit.client.1.smithi097.stdout:2/428: mknod d3/d4/d8/c75 0 2024-09-16T08:57:10.477 INFO:tasks.workunit.client.1.smithi097.stdout:2/429: chown d3/d4/d8/d51 814678 1 2024-09-16T08:57:10.479 INFO:tasks.workunit.client.1.smithi097.stdout:0/487: dread f8 [4194304,4194304] 0 2024-09-16T08:57:10.480 INFO:tasks.workunit.client.1.smithi097.stdout:0/488: dread da/d1a/d41/f50 [0,4194304] 0 2024-09-16T08:57:10.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/489: dread - da/d24/d4b/fa1 zero size 2024-09-16T08:57:10.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/490: readlink da/l4f 0 2024-09-16T08:57:10.485 INFO:tasks.workunit.client.1.smithi097.stdout:7/422: dwrite d4/d1b/f35 [0,4194304] 0 2024-09-16T08:57:10.485 INFO:tasks.workunit.client.1.smithi097.stdout:7/423: chown d4/d9/d1c/d5f/l80 114599 1 2024-09-16T08:57:10.485 INFO:tasks.workunit.client.1.smithi097.stdout:1/478: dwrite d5/d2c/f2e [0,4194304] 0 2024-09-16T08:57:10.486 INFO:tasks.workunit.client.1.smithi097.stdout:1/479: write d5/d2c/d3b/f9a [7375813,93729] 0 2024-09-16T08:57:10.486 INFO:tasks.workunit.client.1.smithi097.stdout:1/480: truncate d5/d2c/d3b/f80 421895 0 2024-09-16T08:57:10.487 INFO:tasks.workunit.client.1.smithi097.stdout:8/431: dwrite d8/dd/f61 [0,4194304] 0 2024-09-16T08:57:10.489 INFO:tasks.workunit.client.1.smithi097.stdout:1/481: stat d5/db/d25/f45 0 2024-09-16T08:57:10.489 INFO:tasks.workunit.client.1.smithi097.stdout:1/482: readlink d5/db/d25/l8b 0 2024-09-16T08:57:10.490 INFO:tasks.workunit.client.1.smithi097.stdout:1/483: stat d5/d2c/d3b/d7f 0 2024-09-16T08:57:10.492 INFO:tasks.workunit.client.1.smithi097.stdout:1/484: creat d5/d17/d54/d6b/f9c x:0 0 0 2024-09-16T08:57:10.493 INFO:tasks.workunit.client.1.smithi097.stdout:1/485: dread - d5/db/dc/d19/f4d zero size 2024-09-16T08:57:10.494 INFO:tasks.workunit.client.1.smithi097.stdout:1/486: mkdir d5/d70/d9d 0 2024-09-16T08:57:10.495 INFO:tasks.workunit.client.1.smithi097.stdout:1/487: write d5/db/dc/d19/f4d [549861,17854] 0 2024-09-16T08:57:10.495 INFO:tasks.workunit.client.1.smithi097.stdout:1/488: fsync d5/d2c/f2e 0 2024-09-16T08:57:10.496 INFO:tasks.workunit.client.1.smithi097.stdout:4/425: rmdir d0/d7/d52 39 2024-09-16T08:57:10.496 INFO:tasks.workunit.client.1.smithi097.stdout:4/426: stat d0/d39/d71/f77 0 2024-09-16T08:57:10.498 INFO:tasks.workunit.client.1.smithi097.stdout:8/432: dread d8/dd/d15/d2c/d4a/d4e/f70 [0,4194304] 0 2024-09-16T08:57:10.499 INFO:tasks.workunit.client.1.smithi097.stdout:1/489: rename d5/l1b to d5/db/dc/d19/l9e 0 2024-09-16T08:57:10.499 INFO:tasks.workunit.client.1.smithi097.stdout:1/490: readlink d5/l3d 0 2024-09-16T08:57:10.501 INFO:tasks.workunit.client.1.smithi097.stdout:8/433: mkdir d8/dd/d15/d83 0 2024-09-16T08:57:10.501 INFO:tasks.workunit.client.1.smithi097.stdout:8/434: read - d8/dd/d15/d2c/d4a/d4d/f68 zero size 2024-09-16T08:57:10.504 INFO:tasks.workunit.client.1.smithi097.stdout:8/435: link d8/dd/d15/d2c/d4a/d4d/f68 d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f84 0 2024-09-16T08:57:10.505 INFO:tasks.workunit.client.1.smithi097.stdout:8/436: rmdir d8/dd/d15/d83 0 2024-09-16T08:57:10.506 INFO:tasks.workunit.client.1.smithi097.stdout:8/437: write d8/dd/d71/f73 [2059392,10630] 0 2024-09-16T08:57:10.514 INFO:tasks.workunit.client.1.smithi097.stdout:0/491: dwrite da/d24/d4c/d80/d42/d5b/d60/d68/f5d [0,4194304] 0 2024-09-16T08:57:10.515 INFO:tasks.workunit.client.1.smithi097.stdout:7/424: dwrite d4/d1b/f78 [0,4194304] 0 2024-09-16T08:57:10.517 INFO:tasks.workunit.client.1.smithi097.stdout:0/492: mkdir da/df/d13/db0 0 2024-09-16T08:57:10.519 INFO:tasks.workunit.client.1.smithi097.stdout:0/493: mknod da/d24/d4c/d80/d42/d5b/cb1 0 2024-09-16T08:57:10.519 INFO:tasks.workunit.client.1.smithi097.stdout:0/494: mkdir da/d1a/d41/db2 0 2024-09-16T08:57:10.532 INFO:tasks.workunit.client.1.smithi097.stdout:0/495: dread da/d1a/d26/f6b [0,4194304] 0 2024-09-16T08:57:10.532 INFO:tasks.workunit.client.1.smithi097.stdout:0/496: write da/d1a/f3a [4558930,63399] 0 2024-09-16T08:57:10.535 INFO:tasks.workunit.client.1.smithi097.stdout:8/438: dwrite d8/dd/d15/d2c/d48/d56/f6c [0,4194304] 0 2024-09-16T08:57:10.535 INFO:tasks.workunit.client.1.smithi097.stdout:4/427: dwrite d0/d35/f5c [0,4194304] 0 2024-09-16T08:57:10.535 INFO:tasks.workunit.client.1.smithi097.stdout:1/491: dwrite d5/d2c/f4c [0,4194304] 0 2024-09-16T08:57:10.537 INFO:tasks.workunit.client.1.smithi097.stdout:8/439: mkdir d8/dd/d15/d2c/d4a/d4e/d85 0 2024-09-16T08:57:10.539 INFO:tasks.workunit.client.1.smithi097.stdout:0/497: rename da/d24/d4c/d80/d42/d5b to da/d1a/d26/d47/db3 0 2024-09-16T08:57:10.544 INFO:tasks.workunit.client.1.smithi097.stdout:8/440: dread d8/f1e [0,4194304] 0 2024-09-16T08:57:10.544 INFO:tasks.workunit.client.1.smithi097.stdout:0/498: link da/df/d3d/l67 da/df/d13/lb4 0 2024-09-16T08:57:10.544 INFO:tasks.workunit.client.1.smithi097.stdout:0/499: stat da/d1a/d26/d47/db3/d60/d68/c8c 0 2024-09-16T08:57:10.545 INFO:tasks.workunit.client.1.smithi097.stdout:8/441: chown d8/dd/f7e 4724 1 2024-09-16T08:57:10.546 INFO:tasks.workunit.client.1.smithi097.stdout:4/428: dread d0/d39/f3c [0,4194304] 0 2024-09-16T08:57:10.546 INFO:tasks.workunit.client.1.smithi097.stdout:4/429: mkdir d0/d7/d52/d26/d9a 0 2024-09-16T08:57:10.547 INFO:tasks.workunit.client.1.smithi097.stdout:8/442: dread d8/dd/d15/f34 [0,4194304] 0 2024-09-16T08:57:10.573 INFO:tasks.workunit.client.1.smithi097.stdout:4/430: dwrite d0/f64 [0,4194304] 0 2024-09-16T08:57:10.574 INFO:tasks.workunit.client.1.smithi097.stdout:4/431: chown d0/d35 1 1 2024-09-16T08:57:10.574 INFO:tasks.workunit.client.1.smithi097.stdout:8/443: dwrite d8/dd/d71/f73 [0,4194304] 0 2024-09-16T08:57:10.576 INFO:tasks.workunit.client.1.smithi097.stdout:4/432: rename d0/d7/lc to d0/d39/l9b 0 2024-09-16T08:57:10.576 INFO:tasks.workunit.client.1.smithi097.stdout:4/433: readlink d0/d39/d71/l6d 0 2024-09-16T08:57:10.577 INFO:tasks.workunit.client.1.smithi097.stdout:8/444: chown d8/dd/d42/f51 2 1 2024-09-16T08:57:10.577 INFO:tasks.workunit.client.0.smithi031.stdout:4/14: sync 2024-09-16T08:57:10.577 INFO:tasks.workunit.client.0.smithi031.stdout:5/86: sync 2024-09-16T08:57:10.578 INFO:tasks.workunit.client.1.smithi097.stdout:4/434: link d0/d7/l7a d0/d23/l9c 0 2024-09-16T08:57:10.579 INFO:tasks.workunit.client.1.smithi097.stdout:8/445: mkdir d8/dd/d15/d16/d7c/d86 0 2024-09-16T08:57:10.579 INFO:tasks.workunit.client.0.smithi031.stdout:4/15: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:10.580 INFO:tasks.workunit.client.0.smithi031.stdout:4/16: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:10.581 INFO:tasks.workunit.client.0.smithi031.stdout:4/17: symlink d0/l5 0 2024-09-16T08:57:10.581 INFO:tasks.workunit.client.0.smithi031.stdout:4/18: unlink d0/d2/c4 0 2024-09-16T08:57:10.581 INFO:tasks.workunit.client.0.smithi031.stdout:4/19: fsync d0/f1 0 2024-09-16T08:57:10.582 INFO:tasks.workunit.client.1.smithi097.stdout:4/435: dread d0/d7/df/d15/f41 [0,4194304] 0 2024-09-16T08:57:10.582 INFO:tasks.workunit.client.0.smithi031.stdout:4/20: mknod d0/d2/c6 0 2024-09-16T08:57:10.583 INFO:tasks.workunit.client.0.smithi031.stdout:4/21: mknod d0/c7 0 2024-09-16T08:57:10.583 INFO:tasks.workunit.client.0.smithi031.stdout:4/22: link d0/l3 d0/d2/l8 0 2024-09-16T08:57:10.584 INFO:tasks.workunit.client.0.smithi031.stdout:4/23: symlink d0/l9 0 2024-09-16T08:57:10.584 INFO:tasks.workunit.client.0.smithi031.stdout:4/24: creat d0/fa x:0 0 0 2024-09-16T08:57:10.585 INFO:tasks.workunit.client.0.smithi031.stdout:4/25: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:10.586 INFO:tasks.workunit.client.0.smithi031.stdout:4/26: write d0/fa [869395,16882] 0 2024-09-16T08:57:10.587 INFO:tasks.workunit.client.0.smithi031.stdout:4/27: link d0/fa d0/fb 0 2024-09-16T08:57:10.589 INFO:tasks.workunit.client.1.smithi097.stdout:8/446: dread d8/fc [0,4194304] 0 2024-09-16T08:57:10.591 INFO:tasks.workunit.client.1.smithi097.stdout:8/447: stat d8/dd/d15/l6f 0 2024-09-16T08:57:10.592 INFO:tasks.workunit.client.1.smithi097.stdout:8/448: write d8/dd/f26 [693748,8121] 0 2024-09-16T08:57:10.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/28: dread d0/fb [0,4194304] 0 2024-09-16T08:57:10.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/29: creat d0/fc x:0 0 0 2024-09-16T08:57:10.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/30: mknod d0/d2/cd 0 2024-09-16T08:57:10.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/31: dread - d0/fc zero size 2024-09-16T08:57:10.596 INFO:tasks.workunit.client.0.smithi031.stdout:4/32: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:10.598 INFO:tasks.workunit.client.0.smithi031.stdout:4/33: rename d0/fc to d0/d2/fe 0 2024-09-16T08:57:10.599 INFO:tasks.workunit.client.0.smithi031.stdout:4/34: mknod d0/d2/cf 0 2024-09-16T08:57:10.600 INFO:tasks.workunit.client.0.smithi031.stdout:4/35: creat d0/d2/f10 x:0 0 0 2024-09-16T08:57:10.605 INFO:tasks.workunit.client.1.smithi097.stdout:8/449: dread d8/f10 [0,4194304] 0 2024-09-16T08:57:10.605 INFO:tasks.workunit.client.1.smithi097.stdout:8/450: chown d8/dd 563806672 1 2024-09-16T08:57:10.605 INFO:tasks.workunit.client.1.smithi097.stdout:8/451: chown d8/dd/d15/d2c/d4a/d4d/d3d/f43 206 1 2024-09-16T08:57:10.606 INFO:tasks.workunit.client.1.smithi097.stdout:8/452: fsync d8/dd/d15/f28 0 2024-09-16T08:57:10.607 INFO:tasks.workunit.client.1.smithi097.stdout:8/453: rename d8/dd/d15/d2c/d4a/d4d/f68 to d8/dd/d15/d2c/d4a/f87 0 2024-09-16T08:57:10.609 INFO:tasks.workunit.client.1.smithi097.stdout:4/436: dwrite d0/f64 [0,4194304] 0 2024-09-16T08:57:10.609 INFO:tasks.workunit.client.1.smithi097.stdout:4/437: truncate d0/d35/f5c 4632027 0 2024-09-16T08:57:10.609 INFO:tasks.workunit.client.1.smithi097.stdout:4/438: fdatasync d0/d23/d5f/f93 0 2024-09-16T08:57:10.611 INFO:tasks.workunit.client.1.smithi097.stdout:4/439: rename d0/d7/df/d15/c45 to d0/d7/d52/d26/d3d/c9d 0 2024-09-16T08:57:10.611 INFO:tasks.workunit.client.1.smithi097.stdout:4/440: fsync d0/d7/d52/f94 0 2024-09-16T08:57:10.611 INFO:tasks.workunit.client.1.smithi097.stdout:4/441: write d0/d23/f3a [4428043,82875] 0 2024-09-16T08:57:10.614 INFO:tasks.workunit.client.1.smithi097.stdout:4/442: dread d0/d36/d4c/f80 [0,4194304] 0 2024-09-16T08:57:10.614 INFO:tasks.workunit.client.1.smithi097.stdout:4/443: write d0/d7/df/f8f [309194,121203] 0 2024-09-16T08:57:10.615 INFO:tasks.workunit.client.1.smithi097.stdout:4/444: symlink d0/d7/d52/d26/d8c/l9e 0 2024-09-16T08:57:10.617 INFO:tasks.workunit.client.1.smithi097.stdout:4/445: getdents d0 0 2024-09-16T08:57:10.617 INFO:tasks.workunit.client.1.smithi097.stdout:4/446: stat d0/d23/c60 0 2024-09-16T08:57:10.617 INFO:tasks.workunit.client.1.smithi097.stdout:4/447: write d0/d23/f3a [5456610,95517] 0 2024-09-16T08:57:10.617 INFO:tasks.workunit.client.1.smithi097.stdout:4/448: truncate d0/d35/f38 955985 0 2024-09-16T08:57:10.617 INFO:tasks.workunit.client.1.smithi097.stdout:4/449: creat d0/d23/f9f x:0 0 0 2024-09-16T08:57:10.620 INFO:tasks.workunit.client.1.smithi097.stdout:4/450: mknod d0/d35/d6c/ca0 0 2024-09-16T08:57:10.620 INFO:tasks.workunit.client.1.smithi097.stdout:4/451: fsync d0/d39/f6f 0 2024-09-16T08:57:10.620 INFO:tasks.workunit.client.1.smithi097.stdout:4/452: stat d0/d7/d52/d26/d8c 0 2024-09-16T08:57:10.621 INFO:tasks.workunit.client.1.smithi097.stdout:8/454: dread d8/dd/f6a [0,4194304] 0 2024-09-16T08:57:10.621 INFO:tasks.workunit.client.1.smithi097.stdout:8/455: fsync d8/dd/d15/d2c/d48/d56/f6c 0 2024-09-16T08:57:10.621 INFO:tasks.workunit.client.0.smithi031.stdout:5/87: dwrite d2/dc/f12 [0,4194304] 0 2024-09-16T08:57:10.621 INFO:tasks.workunit.client.0.smithi031.stdout:5/88: truncate d2/dc/f19 322818 0 2024-09-16T08:57:10.621 INFO:tasks.workunit.client.0.smithi031.stdout:5/89: fdatasync d2/f1a 0 2024-09-16T08:57:10.621 INFO:tasks.workunit.client.0.smithi031.stdout:5/90: dread - d2/f1a zero size 2024-09-16T08:57:10.621 INFO:tasks.workunit.client.0.smithi031.stdout:5/91: creat d2/f1b x:0 0 0 2024-09-16T08:57:10.622 INFO:tasks.workunit.client.0.smithi031.stdout:4/36: dwrite d0/fb [0,4194304] 0 2024-09-16T08:57:10.622 INFO:tasks.workunit.client.1.smithi097.stdout:4/453: rmdir d0/d7/d52/d26 39 2024-09-16T08:57:10.622 INFO:tasks.workunit.client.1.smithi097.stdout:4/454: truncate d0/d7/d52/d26/d3d/f76 837186 0 2024-09-16T08:57:10.625 INFO:tasks.workunit.client.1.smithi097.stdout:8/456: symlink d8/dd/d15/d16/l88 0 2024-09-16T08:57:10.629 INFO:tasks.workunit.client.0.smithi031.stdout:4/37: mknod d0/d2/c11 0 2024-09-16T08:57:10.629 INFO:tasks.workunit.client.0.smithi031.stdout:4/38: chown d0/d2/fe 0 1 2024-09-16T08:57:10.630 INFO:tasks.workunit.client.0.smithi031.stdout:5/92: write d2/f5 [2753061,30003] 0 2024-09-16T08:57:10.632 INFO:tasks.workunit.client.0.smithi031.stdout:4/39: symlink d0/d2/l12 0 2024-09-16T08:57:10.632 INFO:tasks.workunit.client.0.smithi031.stdout:4/40: creat d0/d2/f13 x:0 0 0 2024-09-16T08:57:10.633 INFO:tasks.workunit.client.1.smithi097.stdout:4/455: unlink d0/d39/d71/f87 0 2024-09-16T08:57:10.633 INFO:tasks.workunit.client.1.smithi097.stdout:4/456: write d0/d7/df/d15/f41 [1110869,16445] 0 2024-09-16T08:57:10.634 INFO:tasks.workunit.client.1.smithi097.stdout:4/457: fdatasync d0/d7/d52/d26/f6b 0 2024-09-16T08:57:10.635 INFO:tasks.workunit.client.1.smithi097.stdout:8/457: stat d8/dd/c27 0 2024-09-16T08:57:10.635 INFO:tasks.workunit.client.1.smithi097.stdout:8/458: fdatasync d8/dd/d15/f28 0 2024-09-16T08:57:10.635 INFO:tasks.workunit.client.1.smithi097.stdout:8/459: write d8/dd/d15/d2c/d4a/d4d/d62/f76 [276915,75145] 0 2024-09-16T08:57:10.639 INFO:tasks.workunit.client.0.smithi031.stdout:4/41: link d0/d2/cd d0/c14 0 2024-09-16T08:57:10.643 INFO:tasks.workunit.client.1.smithi097.stdout:4/458: creat d0/d7/d52/d26/fa1 x:0 0 0 2024-09-16T08:57:10.645 INFO:tasks.workunit.client.1.smithi097.stdout:8/460: creat d8/dd/d42/f89 x:0 0 0 2024-09-16T08:57:10.645 INFO:tasks.workunit.client.1.smithi097.stdout:8/461: readlink d8/dd/d15/d2c/d4a/l52 0 2024-09-16T08:57:10.649 INFO:tasks.workunit.client.1.smithi097.stdout:4/459: write d0/d39/f3c [3512988,63175] 0 2024-09-16T08:57:10.650 INFO:tasks.workunit.client.1.smithi097.stdout:8/462: symlink d8/l8a 0 2024-09-16T08:57:10.653 INFO:tasks.workunit.client.0.smithi031.stdout:4/42: getdents d0/d2 0 2024-09-16T08:57:10.653 INFO:tasks.workunit.client.1.smithi097.stdout:8/463: dread d8/dd/d15/f28 [0,4194304] 0 2024-09-16T08:57:10.654 INFO:tasks.workunit.client.1.smithi097.stdout:8/464: chown d8/dd/d15/f29 9 1 2024-09-16T08:57:10.654 INFO:tasks.workunit.client.0.smithi031.stdout:4/43: rmdir d0/d2 39 2024-09-16T08:57:10.654 INFO:tasks.workunit.client.0.smithi031.stdout:4/44: write d0/d2/f13 [619219,76946] 0 2024-09-16T08:57:10.654 INFO:tasks.workunit.client.0.smithi031.stdout:4/45: truncate d0/d2/fe 406752 0 2024-09-16T08:57:10.655 INFO:tasks.workunit.client.0.smithi031.stdout:5/93: dwrite d2/dc/f19 [0,4194304] 0 2024-09-16T08:57:10.655 INFO:tasks.workunit.client.0.smithi031.stdout:5/94: fdatasync d2/dc/f19 0 2024-09-16T08:57:10.656 INFO:tasks.workunit.client.0.smithi031.stdout:5/95: stat d2/dc/f12 0 2024-09-16T08:57:10.656 INFO:tasks.workunit.client.0.smithi031.stdout:4/46: rename d0/c7 to d0/c15 0 2024-09-16T08:57:10.657 INFO:tasks.workunit.client.1.smithi097.stdout:8/465: creat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f8b x:0 0 0 2024-09-16T08:57:10.658 INFO:tasks.workunit.client.0.smithi031.stdout:5/96: dread d2/dc/f16 [0,4194304] 0 2024-09-16T08:57:10.658 INFO:tasks.workunit.client.0.smithi031.stdout:5/97: chown d2/f5 5414026 1 2024-09-16T08:57:10.658 INFO:tasks.workunit.client.0.smithi031.stdout:5/98: write d2/dc/f12 [5089751,20267] 0 2024-09-16T08:57:10.660 INFO:tasks.workunit.client.1.smithi097.stdout:8/466: mknod d8/dd/d15/d2c/d4a/d4d/c8c 0 2024-09-16T08:57:10.664 INFO:tasks.workunit.client.0.smithi031.stdout:5/99: dread d2/f5 [0,4194304] 0 2024-09-16T08:57:10.665 INFO:tasks.workunit.client.0.smithi031.stdout:5/100: creat d2/f1c x:0 0 0 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:9/403: sync 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:0/500: sync 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:3/468: sync 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:1/492: sync 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:9/404: chown d2/l23 22289482 1 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:7/425: sync 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:7/426: chown d4/d1b/f35 112965553 1 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:1/493: chown d5/d17/d54/d6b/l6e 425048990 1 2024-09-16T08:57:10.669 INFO:tasks.workunit.client.1.smithi097.stdout:7/427: readlink d4/d7/d7b/d43/l54 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:6/485: sync 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:7/428: write d4/d9/d1c/d5f/f6c [10717,59040] 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:2/430: sync 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:5/472: sync 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:2/431: creat d3/d1b/f76 x:0 0 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:5/473: creat db/d18/d1c/d42/d46/d7e/f97 x:0 0 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:5/474: readlink db/l1e 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:2/432: fsync d3/d4/dd/d53/f50 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:2/433: truncate d3/d4/dd/d53/f6c 1515996 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:5/475: read db/d1a/f5b [143401,122375] 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:8/467: mknod d8/dd/c8d 0 2024-09-16T08:57:10.670 INFO:tasks.workunit.client.1.smithi097.stdout:3/469: mknod d7/d28/ca0 0 2024-09-16T08:57:10.671 INFO:tasks.workunit.client.0.smithi031.stdout:5/101: write d2/fe [96785,39737] 0 2024-09-16T08:57:10.671 INFO:tasks.workunit.client.1.smithi097.stdout:9/405: creat d2/d3f/d62/f7f x:0 0 0 2024-09-16T08:57:10.672 INFO:tasks.workunit.client.1.smithi097.stdout:1/494: mknod d5/d70/c9f 0 2024-09-16T08:57:10.672 INFO:tasks.workunit.client.1.smithi097.stdout:4/460: dwrite d0/fe [0,4194304] 0 2024-09-16T08:57:10.672 INFO:tasks.workunit.client.1.smithi097.stdout:1/495: write d5/d70/f72 [944083,35508] 0 2024-09-16T08:57:10.672 INFO:tasks.workunit.client.1.smithi097.stdout:1/496: truncate d5/d2c/d3b/f80 942075 0 2024-09-16T08:57:10.683 INFO:tasks.workunit.client.0.smithi031.stdout:5/102: read d2/dc/f12 [2088444,81193] 0 2024-09-16T08:57:10.683 INFO:tasks.workunit.client.0.smithi031.stdout:5/103: write d2/f11 [4693955,98579] 0 2024-09-16T08:57:10.686 INFO:tasks.workunit.client.0.smithi031.stdout:3/59: sync 2024-09-16T08:57:10.688 INFO:tasks.workunit.client.1.smithi097.stdout:5/476: creat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 x:0 0 0 2024-09-16T08:57:10.691 INFO:tasks.workunit.client.0.smithi031.stdout:0/105: sync 2024-09-16T08:57:10.691 INFO:tasks.workunit.client.0.smithi031.stdout:6/88: sync 2024-09-16T08:57:10.692 INFO:tasks.workunit.client.1.smithi097.stdout:3/470: creat d7/de/d3c/d3f/d60/d7a/fa1 x:0 0 0 2024-09-16T08:57:10.692 INFO:tasks.workunit.client.1.smithi097.stdout:3/471: readlink d7/de/d3c/l3e 0 2024-09-16T08:57:10.692 INFO:tasks.workunit.client.1.smithi097.stdout:3/472: fdatasync d7/de/d6c/f4a 0 2024-09-16T08:57:10.692 INFO:tasks.workunit.client.1.smithi097.stdout:3/473: creat d7/de/d8d/d90/fa2 x:0 0 0 2024-09-16T08:57:10.692 INFO:tasks.workunit.client.1.smithi097.stdout:3/474: truncate d7/de/d29/f76 674455 0 2024-09-16T08:57:10.693 INFO:tasks.workunit.client.0.smithi031.stdout:0/106: creat d6/f26 x:0 0 0 2024-09-16T08:57:10.696 INFO:tasks.workunit.client.0.smithi031.stdout:6/89: mkdir d0/d10/d14 0 2024-09-16T08:57:10.696 INFO:tasks.workunit.client.0.smithi031.stdout:6/90: truncate d0/fc 886931 0 2024-09-16T08:57:10.696 INFO:tasks.workunit.client.0.smithi031.stdout:6/91: chown d0/c5 11285 1 2024-09-16T08:57:10.696 INFO:tasks.workunit.client.0.smithi031.stdout:6/92: fsync d0/fc 0 2024-09-16T08:57:10.696 INFO:tasks.workunit.client.0.smithi031.stdout:6/93: chown d0/dd/ce 24735848 1 2024-09-16T08:57:10.697 INFO:tasks.workunit.client.0.smithi031.stdout:7/131: sync 2024-09-16T08:57:10.697 INFO:tasks.workunit.client.0.smithi031.stdout:8/77: sync 2024-09-16T08:57:10.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/113: sync 2024-09-16T08:57:10.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/114: creat d2/f24 x:0 0 0 2024-09-16T08:57:10.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/41: sync 2024-09-16T08:57:10.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/42: fsync f4 0 2024-09-16T08:57:10.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/43: creat d6/da/ff x:0 0 0 2024-09-16T08:57:10.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/44: dread - d6/da/ff zero size 2024-09-16T08:57:10.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/45: dread - d6/da/ff zero size 2024-09-16T08:57:10.699 INFO:tasks.workunit.client.0.smithi031.stdout:2/46: write f4 [3788636,91689] 0 2024-09-16T08:57:10.699 INFO:tasks.workunit.client.1.smithi097.stdout:9/406: getdents d2/d3f/d62 0 2024-09-16T08:57:10.699 INFO:tasks.workunit.client.1.smithi097.stdout:9/407: write d2/d30/f5a [8633840,123214] 0 2024-09-16T08:57:10.700 INFO:tasks.workunit.client.1.smithi097.stdout:0/501: dwrite da/df/f3f [0,4194304] 0 2024-09-16T08:57:10.702 INFO:tasks.workunit.client.1.smithi097.stdout:2/434: rename d3/d4/d8/d34 to d3/d4/d77 0 2024-09-16T08:57:10.702 INFO:tasks.workunit.client.0.smithi031.stdout:8/78: write d4/f14 [111180,44556] 0 2024-09-16T08:57:10.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/79: creat d4/f18 x:0 0 0 2024-09-16T08:57:10.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/80: creat d4/f19 x:0 0 0 2024-09-16T08:57:10.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/81: write d4/f19 [262211,29767] 0 2024-09-16T08:57:10.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/82: read - d4/f17 zero size 2024-09-16T08:57:10.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/132: creat d2/d8/f30 x:0 0 0 2024-09-16T08:57:10.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/133: readlink d2/d8/d11/l1c 0 2024-09-16T08:57:10.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/104: dread d2/dc/f19 [0,4194304] 0 2024-09-16T08:57:10.705 INFO:tasks.workunit.client.0.smithi031.stdout:3/60: dread d3/fa [0,4194304] 0 2024-09-16T08:57:10.705 INFO:tasks.workunit.client.1.smithi097.stdout:2/435: dread d3/d4/d77/d48/f60 [0,4194304] 0 2024-09-16T08:57:10.708 INFO:tasks.workunit.client.0.smithi031.stdout:1/115: write d2/d4/f5 [2017990,399] 0 2024-09-16T08:57:10.708 INFO:tasks.workunit.client.0.smithi031.stdout:1/116: fdatasync f0 0 2024-09-16T08:57:10.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/30: sync 2024-09-16T08:57:10.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/31: write d3/d5/f6 [145529,295] 0 2024-09-16T08:57:10.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/32: rename d3 to d3/d5/d8 22 2024-09-16T08:57:10.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/33: chown d3/d5 317429524 1 2024-09-16T08:57:10.710 INFO:tasks.workunit.client.1.smithi097.stdout:7/429: dwrite d4/d7/d30/f5b [0,4194304] 0 2024-09-16T08:57:10.710 INFO:tasks.workunit.client.1.smithi097.stdout:7/430: chown d4/d7/d79 391665975 1 2024-09-16T08:57:10.710 INFO:tasks.workunit.client.0.smithi031.stdout:7/134: dread d2/d8/d11/f24 [0,4194304] 0 2024-09-16T08:57:10.711 INFO:tasks.workunit.client.1.smithi097.stdout:6/486: dwrite d1/d3/d16/f82 [0,4194304] 0 2024-09-16T08:57:10.711 INFO:tasks.workunit.client.1.smithi097.stdout:0/502: truncate da/d24/d4c/f9d 774691 0 2024-09-16T08:57:10.713 INFO:tasks.workunit.client.1.smithi097.stdout:1/497: rename d5/d2c/f4c to d5/d14/d95/fa0 0 2024-09-16T08:57:10.713 INFO:tasks.workunit.client.1.smithi097.stdout:1/498: read d5/db/dc/d19/f3f [3149997,35997] 0 2024-09-16T08:57:10.715 INFO:tasks.workunit.client.0.smithi031.stdout:3/61: dread d3/dc/f13 [0,4194304] 0 2024-09-16T08:57:10.715 INFO:tasks.workunit.client.0.smithi031.stdout:3/62: write d3/fb [375664,42929] 0 2024-09-16T08:57:10.715 INFO:tasks.workunit.client.1.smithi097.stdout:2/436: unlink d3/d4/dd/f35 0 2024-09-16T08:57:10.721 INFO:tasks.workunit.client.1.smithi097.stdout:8/468: dwrite d8/dd/f19 [4194304,4194304] 0 2024-09-16T08:57:10.721 INFO:tasks.workunit.client.0.smithi031.stdout:8/83: unlink d4/fa 0 2024-09-16T08:57:10.723 INFO:tasks.workunit.client.0.smithi031.stdout:5/105: mkdir d2/dc/d1d 0 2024-09-16T08:57:10.725 INFO:tasks.workunit.client.1.smithi097.stdout:6/487: read d1/d3/d16/f2e [1669507,96978] 0 2024-09-16T08:57:10.725 INFO:tasks.workunit.client.0.smithi031.stdout:9/34: creat d3/f9 x:0 0 0 2024-09-16T08:57:10.726 INFO:tasks.workunit.client.0.smithi031.stdout:7/135: dread d2/d8/d11/f1b [0,4194304] 0 2024-09-16T08:57:10.727 INFO:tasks.workunit.client.1.smithi097.stdout:0/503: mknod da/d1a/d26/d47/db3/d60/d68/cb5 0 2024-09-16T08:57:10.728 INFO:tasks.workunit.client.0.smithi031.stdout:2/47: getdents d6 0 2024-09-16T08:57:10.736 INFO:tasks.workunit.client.1.smithi097.stdout:5/477: dwrite db/d1a/f3a [0,4194304] 0 2024-09-16T08:57:10.738 INFO:tasks.workunit.client.1.smithi097.stdout:7/431: rename d4/d7/d19/f33 to d4/d1b/f91 0 2024-09-16T08:57:10.740 INFO:tasks.workunit.client.1.smithi097.stdout:1/499: symlink d5/d2c/d3b/d7f/la1 0 2024-09-16T08:57:10.741 INFO:tasks.workunit.client.0.smithi031.stdout:5/106: symlink d2/dc/d1d/l1e 0 2024-09-16T08:57:10.741 INFO:tasks.workunit.client.0.smithi031.stdout:5/107: creat d2/dc/f1f x:0 0 0 2024-09-16T08:57:10.741 INFO:tasks.workunit.client.0.smithi031.stdout:5/108: creat d2/dc/f20 x:0 0 0 2024-09-16T08:57:10.743 INFO:tasks.workunit.client.0.smithi031.stdout:9/35: mkdir d3/d5/da 0 2024-09-16T08:57:10.745 INFO:tasks.workunit.client.1.smithi097.stdout:6/488: symlink d1/d3/d3e/d5a/l98 0 2024-09-16T08:57:10.746 INFO:tasks.workunit.client.0.smithi031.stdout:7/136: creat d2/d8/d11/f31 x:0 0 0 2024-09-16T08:57:10.748 INFO:tasks.workunit.client.0.smithi031.stdout:3/63: symlink d3/l15 0 2024-09-16T08:57:10.748 INFO:tasks.workunit.client.0.smithi031.stdout:3/64: write d3/fb [1128821,2760] 0 2024-09-16T08:57:10.749 INFO:tasks.workunit.client.1.smithi097.stdout:5/478: creat db/d18/d1c/d42/d46/d70/d90/f99 x:0 0 0 2024-09-16T08:57:10.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/432: creat d4/d9/dc/d8e/f92 x:0 0 0 2024-09-16T08:57:10.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/433: write d4/d7/d30/f5c [2193429,62872] 0 2024-09-16T08:57:10.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/434: creat d4/d9/dc/f93 x:0 0 0 2024-09-16T08:57:10.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/435: dread - d4/d1b/d41/f75 zero size 2024-09-16T08:57:10.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/436: write d4/d7/d19/f71 [4872797,82912] 0 2024-09-16T08:57:10.753 INFO:tasks.workunit.client.1.smithi097.stdout:1/500: rename d5/db/dc/d19/l1f to d5/d2c/d3b/la2 0 2024-09-16T08:57:10.753 INFO:tasks.workunit.client.1.smithi097.stdout:1/501: creat d5/d70/fa3 x:0 0 0 2024-09-16T08:57:10.753 INFO:tasks.workunit.client.1.smithi097.stdout:9/408: dwrite d2/d3f/f4b [0,4194304] 0 2024-09-16T08:57:10.753 INFO:tasks.workunit.client.0.smithi031.stdout:9/36: mkdir d3/d5/da/db 0 2024-09-16T08:57:10.754 INFO:tasks.workunit.client.1.smithi097.stdout:6/489: creat d1/d3/d16/d6a/f99 x:0 0 0 2024-09-16T08:57:10.758 INFO:tasks.workunit.client.0.smithi031.stdout:3/65: link d3/df/f10 d3/dc/f16 0 2024-09-16T08:57:10.758 INFO:tasks.workunit.client.1.smithi097.stdout:7/437: truncate d4/d9/d1c/f2c 4289697 0 2024-09-16T08:57:10.759 INFO:tasks.workunit.client.1.smithi097.stdout:7/438: chown d4/d9/d51/d88/l8a 5475 1 2024-09-16T08:57:10.759 INFO:tasks.workunit.client.1.smithi097.stdout:7/439: readlink d4/d9/dc/l10 0 2024-09-16T08:57:10.759 INFO:tasks.workunit.client.1.smithi097.stdout:7/440: creat d4/d7/d77/f94 x:0 0 0 2024-09-16T08:57:10.760 INFO:tasks.workunit.client.1.smithi097.stdout:1/502: truncate d5/f21 2324215 0 2024-09-16T08:57:10.760 INFO:tasks.workunit.client.1.smithi097.stdout:5/479: dread db/d18/d1c/d42/d46/d7e/f53 [0,4194304] 0 2024-09-16T08:57:10.761 INFO:tasks.workunit.client.1.smithi097.stdout:6/490: truncate d1/d3/d3e/f61 2867469 0 2024-09-16T08:57:10.762 INFO:tasks.workunit.client.1.smithi097.stdout:7/441: truncate d4/d7/d7b/f46 729985 0 2024-09-16T08:57:10.763 INFO:tasks.workunit.client.0.smithi031.stdout:3/66: mknod d3/c17 0 2024-09-16T08:57:10.768 INFO:tasks.workunit.client.0.smithi031.stdout:9/37: write f2 [6666383,12212] 0 2024-09-16T08:57:10.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/107: dwrite d6/f7 [0,4194304] 0 2024-09-16T08:57:10.769 INFO:tasks.workunit.client.0.smithi031.stdout:6/94: dwrite d0/ff [0,4194304] 0 2024-09-16T08:57:10.769 INFO:tasks.workunit.client.0.smithi031.stdout:6/95: chown d0/c5 387102272 1 2024-09-16T08:57:10.769 INFO:tasks.workunit.client.0.smithi031.stdout:6/96: chown d0/c5 13063101 1 2024-09-16T08:57:10.770 INFO:tasks.workunit.client.0.smithi031.stdout:3/67: write d3/fb [621028,32730] 0 2024-09-16T08:57:10.771 INFO:tasks.workunit.client.1.smithi097.stdout:2/437: dwrite d3/d4/d9/f4f [4194304,4194304] 0 2024-09-16T08:57:10.771 INFO:tasks.workunit.client.1.smithi097.stdout:2/438: dread - d3/d4/d8/d51/f63 zero size 2024-09-16T08:57:10.771 INFO:tasks.workunit.client.1.smithi097.stdout:8/469: dwrite d8/f31 [0,4194304] 0 2024-09-16T08:57:10.772 INFO:tasks.workunit.client.0.smithi031.stdout:1/117: dwrite d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:10.772 INFO:tasks.workunit.client.0.smithi031.stdout:2/48: dwrite d6/fc [0,4194304] 0 2024-09-16T08:57:10.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/49: fdatasync d6/fb 0 2024-09-16T08:57:10.773 INFO:tasks.workunit.client.0.smithi031.stdout:6/97: creat d0/f15 x:0 0 0 2024-09-16T08:57:10.773 INFO:tasks.workunit.client.0.smithi031.stdout:0/108: truncate d6/f8 739742 0 2024-09-16T08:57:10.773 INFO:tasks.workunit.client.0.smithi031.stdout:6/98: write d0/d10/f13 [452574,10788] 0 2024-09-16T08:57:10.776 INFO:tasks.workunit.client.0.smithi031.stdout:5/109: dwrite d2/dc/fd [0,4194304] 0 2024-09-16T08:57:10.776 INFO:tasks.workunit.client.0.smithi031.stdout:5/110: truncate d2/f1c 38228 0 2024-09-16T08:57:10.777 INFO:tasks.workunit.client.0.smithi031.stdout:6/99: fsync d0/d10/f13 0 2024-09-16T08:57:10.779 INFO:tasks.workunit.client.1.smithi097.stdout:2/439: mkdir d3/d4/d9/d78 0 2024-09-16T08:57:10.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/470: mknod d8/dd/d15/d2c/c8e 0 2024-09-16T08:57:10.786 INFO:tasks.workunit.client.1.smithi097.stdout:8/471: chown d8/l2a 149484 1 2024-09-16T08:57:10.787 INFO:tasks.workunit.client.1.smithi097.stdout:8/472: dread d8/f13 [0,4194304] 0 2024-09-16T08:57:10.789 INFO:tasks.workunit.client.0.smithi031.stdout:2/50: mknod d6/c10 0 2024-09-16T08:57:10.792 INFO:tasks.workunit.client.1.smithi097.stdout:1/503: dwrite d5/d2c/d3b/f9a [0,4194304] 0 2024-09-16T08:57:10.792 INFO:tasks.workunit.client.1.smithi097.stdout:1/504: readlink d5/d2c/l73 0 2024-09-16T08:57:10.792 INFO:tasks.workunit.client.1.smithi097.stdout:5/480: dwrite db/d22/f4b [0,4194304] 0 2024-09-16T08:57:10.792 INFO:tasks.workunit.client.0.smithi031.stdout:0/109: creat d6/f27 x:0 0 0 2024-09-16T08:57:10.792 INFO:tasks.workunit.client.0.smithi031.stdout:7/137: rmdir d2/d8 39 2024-09-16T08:57:10.794 INFO:tasks.workunit.client.0.smithi031.stdout:5/111: truncate d2/fa 1217983 0 2024-09-16T08:57:10.794 INFO:tasks.workunit.client.0.smithi031.stdout:5/112: chown d2/dc/f16 2 1 2024-09-16T08:57:10.799 INFO:tasks.workunit.client.0.smithi031.stdout:6/100: symlink d0/l16 0 2024-09-16T08:57:10.799 INFO:tasks.workunit.client.0.smithi031.stdout:6/101: write d0/d10/f13 [801990,60671] 0 2024-09-16T08:57:10.800 INFO:tasks.workunit.client.0.smithi031.stdout:6/102: read d0/fc [233937,97130] 0 2024-09-16T08:57:10.800 INFO:tasks.workunit.client.0.smithi031.stdout:6/103: write d0/f8 [429406,51778] 0 2024-09-16T08:57:10.801 INFO:tasks.workunit.client.1.smithi097.stdout:6/491: dwrite d1/d3/d16/f79 [0,4194304] 0 2024-09-16T08:57:10.801 INFO:tasks.workunit.client.1.smithi097.stdout:2/440: rename d3/d4/dd/c56 to d3/d4/d9/d78/c79 0 2024-09-16T08:57:10.801 INFO:tasks.workunit.client.1.smithi097.stdout:2/441: fdatasync d3/d4/d8/f16 0 2024-09-16T08:57:10.803 INFO:tasks.workunit.client.0.smithi031.stdout:3/68: mknod d3/c18 0 2024-09-16T08:57:10.803 INFO:tasks.workunit.client.0.smithi031.stdout:3/69: chown d3/dc/f13 5128707 1 2024-09-16T08:57:10.810 INFO:tasks.workunit.client.0.smithi031.stdout:2/51: mknod d6/da/c11 0 2024-09-16T08:57:10.810 INFO:tasks.workunit.client.0.smithi031.stdout:2/52: fsync d6/da/fe 0 2024-09-16T08:57:10.811 INFO:tasks.workunit.client.0.smithi031.stdout:0/110: truncate d6/f9 3250773 0 2024-09-16T08:57:10.814 INFO:tasks.workunit.client.0.smithi031.stdout:9/38: dwrite f2 [0,4194304] 0 2024-09-16T08:57:10.816 INFO:tasks.workunit.client.0.smithi031.stdout:7/138: creat d2/d8/f32 x:0 0 0 2024-09-16T08:57:10.819 INFO:tasks.workunit.client.0.smithi031.stdout:6/104: mkdir d0/d17 0 2024-09-16T08:57:10.822 INFO:tasks.workunit.client.1.smithi097.stdout:6/492: link d1/dc/d1e/f68 d1/d3/d3e/f9a 0 2024-09-16T08:57:10.825 INFO:tasks.workunit.client.0.smithi031.stdout:0/111: dread d6/f11 [0,4194304] 0 2024-09-16T08:57:10.828 INFO:tasks.workunit.client.0.smithi031.stdout:9/39: dread f2 [0,4194304] 0 2024-09-16T08:57:10.828 INFO:tasks.workunit.client.0.smithi031.stdout:1/118: rename d2/d4/cb to d2/d4/c25 0 2024-09-16T08:57:10.828 INFO:tasks.workunit.client.0.smithi031.stdout:9/40: creat d3/fc x:0 0 0 2024-09-16T08:57:10.828 INFO:tasks.workunit.client.0.smithi031.stdout:9/41: rename d3/d5 to d3/d5/da/db/dd 22 2024-09-16T08:57:10.831 INFO:tasks.workunit.client.1.smithi097.stdout:6/493: stat d1/d3/c31 0 2024-09-16T08:57:10.831 INFO:tasks.workunit.client.1.smithi097.stdout:6/494: dread - d1/d3/d16/d86/f8b zero size 2024-09-16T08:57:10.832 INFO:tasks.workunit.client.1.smithi097.stdout:1/505: dwrite d5/db/f5d [0,4194304] 0 2024-09-16T08:57:10.833 INFO:tasks.workunit.client.1.smithi097.stdout:8/473: dwrite d8/dd/d71/f7d [0,4194304] 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:7/139: link d2/c2a d2/c33 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:3/70: symlink d3/l19 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:6/105: fdatasync d0/f8 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:6/106: fdatasync d0/fc 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:6/107: fdatasync d0/d10/f11 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/113: dwrite d2/fe [0,4194304] 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/114: truncate d2/f1c 963468 0 2024-09-16T08:57:10.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/115: truncate d2/dc/fd 5673609 0 2024-09-16T08:57:10.835 INFO:tasks.workunit.client.1.smithi097.stdout:5/481: dwrite db/d18/d1c/d42/d46/d7e/d64/f63 [0,4194304] 0 2024-09-16T08:57:10.835 INFO:tasks.workunit.client.0.smithi031.stdout:0/112: symlink d6/d12/l28 0 2024-09-16T08:57:10.836 INFO:tasks.workunit.client.0.smithi031.stdout:0/113: fsync d6/fa 0 2024-09-16T08:57:10.836 INFO:tasks.workunit.client.0.smithi031.stdout:5/116: dread d2/f13 [0,4194304] 0 2024-09-16T08:57:10.836 INFO:tasks.workunit.client.0.smithi031.stdout:2/53: dwrite f4 [0,4194304] 0 2024-09-16T08:57:10.836 INFO:tasks.workunit.client.0.smithi031.stdout:5/117: chown d2/f1b 470 1 2024-09-16T08:57:10.836 INFO:tasks.workunit.client.1.smithi097.stdout:2/442: rmdir d3/d4/d8 39 2024-09-16T08:57:10.839 INFO:tasks.workunit.client.0.smithi031.stdout:7/140: stat d2/d8/d11/l1c 0 2024-09-16T08:57:10.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/71: mkdir d3/df/d1a 0 2024-09-16T08:57:10.841 INFO:tasks.workunit.client.0.smithi031.stdout:3/72: getdents d3/df/d1a 0 2024-09-16T08:57:10.844 INFO:tasks.workunit.client.1.smithi097.stdout:1/506: symlink d5/d14/la4 0 2024-09-16T08:57:10.844 INFO:tasks.workunit.client.0.smithi031.stdout:2/54: dread f4 [0,4194304] 0 2024-09-16T08:57:10.844 INFO:tasks.workunit.client.0.smithi031.stdout:2/55: write d6/da/fe [642416,12007] 0 2024-09-16T08:57:10.847 INFO:tasks.workunit.client.1.smithi097.stdout:2/443: write d3/d4/d9/f5a [2022943,57652] 0 2024-09-16T08:57:10.848 INFO:tasks.workunit.client.0.smithi031.stdout:6/108: creat d0/d17/f18 x:0 0 0 2024-09-16T08:57:10.850 INFO:tasks.workunit.client.1.smithi097.stdout:8/474: rename d8/dd/d15/d2c/d4a/d4e/f5a to d8/dd/d15/f8f 0 2024-09-16T08:57:10.853 INFO:tasks.workunit.client.0.smithi031.stdout:2/56: read d6/fc [1317108,47403] 0 2024-09-16T08:57:10.854 INFO:tasks.workunit.client.0.smithi031.stdout:2/57: fdatasync d6/f9 0 2024-09-16T08:57:10.854 INFO:tasks.workunit.client.0.smithi031.stdout:2/58: read d6/fc [3473606,74822] 0 2024-09-16T08:57:10.854 INFO:tasks.workunit.client.1.smithi097.stdout:8/475: dread d8/dd/d15/d2c/d4a/d4e/f70 [0,4194304] 0 2024-09-16T08:57:10.857 INFO:tasks.workunit.client.1.smithi097.stdout:5/482: truncate db/d18/d20/f29 1049258 0 2024-09-16T08:57:10.859 INFO:tasks.workunit.client.1.smithi097.stdout:1/507: dwrite d5/db/d25/f8d [0,4194304] 0 2024-09-16T08:57:10.859 INFO:tasks.workunit.client.1.smithi097.stdout:1/508: getdents d5/d70/d85 0 2024-09-16T08:57:10.860 INFO:tasks.workunit.client.1.smithi097.stdout:1/509: creat d5/d2c/d3b/fa5 x:0 0 0 2024-09-16T08:57:10.860 INFO:tasks.workunit.client.1.smithi097.stdout:1/510: fsync d5/d17/d54/d6b/d71/d94/f99 0 2024-09-16T08:57:10.860 INFO:tasks.workunit.client.1.smithi097.stdout:1/511: creat d5/d2c/fa6 x:0 0 0 2024-09-16T08:57:10.860 INFO:tasks.workunit.client.1.smithi097.stdout:1/512: chown d5/d2c/d3b/d7f/c8e 20 1 2024-09-16T08:57:10.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/495: getdents d1/d3/d3e/d5a 0 2024-09-16T08:57:10.861 INFO:tasks.workunit.client.0.smithi031.stdout:5/118: mkdir d2/d18/d21 0 2024-09-16T08:57:10.863 INFO:tasks.workunit.client.1.smithi097.stdout:2/444: creat d3/d4/d8/f7a x:0 0 0 2024-09-16T08:57:10.864 INFO:tasks.workunit.client.0.smithi031.stdout:9/42: link d3/fc d3/d5/da/db/fe 0 2024-09-16T08:57:10.864 INFO:tasks.workunit.client.1.smithi097.stdout:8/476: symlink d8/dd/d42/l90 0 2024-09-16T08:57:10.866 INFO:tasks.workunit.client.0.smithi031.stdout:7/141: symlink d2/l34 0 2024-09-16T08:57:10.867 INFO:tasks.workunit.client.1.smithi097.stdout:5/483: unlink db/d1a/f1f 0 2024-09-16T08:57:10.868 INFO:tasks.workunit.client.0.smithi031.stdout:7/142: dread d2/d8/f2c [0,4194304] 0 2024-09-16T08:57:10.870 INFO:tasks.workunit.client.0.smithi031.stdout:6/109: mkdir d0/d17/d19 0 2024-09-16T08:57:10.872 INFO:tasks.workunit.client.0.smithi031.stdout:2/59: rename d6/da/c11 to d6/da/c12 0 2024-09-16T08:57:10.874 INFO:tasks.workunit.client.0.smithi031.stdout:0/114: link d6/d12/d13/c1b d6/c29 0 2024-09-16T08:57:10.875 INFO:tasks.workunit.client.0.smithi031.stdout:0/115: creat d6/d12/f2a x:0 0 0 2024-09-16T08:57:10.875 INFO:tasks.workunit.client.0.smithi031.stdout:0/116: write d6/d12/f17 [779603,112875] 0 2024-09-16T08:57:10.876 INFO:tasks.workunit.client.0.smithi031.stdout:5/119: creat d2/d18/f22 x:0 0 0 2024-09-16T08:57:10.878 INFO:tasks.workunit.client.1.smithi097.stdout:1/513: link d5/d17/d54/d6b/l75 d5/la7 0 2024-09-16T08:57:10.878 INFO:tasks.workunit.client.1.smithi097.stdout:1/514: write d5/f21 [566642,21601] 0 2024-09-16T08:57:10.879 INFO:tasks.workunit.client.0.smithi031.stdout:9/43: symlink d3/d5/lf 0 2024-09-16T08:57:10.879 INFO:tasks.workunit.client.0.smithi031.stdout:9/44: dread d3/d5/f6 [0,4194304] 0 2024-09-16T08:57:10.880 INFO:tasks.workunit.client.1.smithi097.stdout:6/496: symlink d1/d3/d81/l9b 0 2024-09-16T08:57:10.880 INFO:tasks.workunit.client.0.smithi031.stdout:3/73: getdents d3 0 2024-09-16T08:57:10.883 INFO:tasks.workunit.client.1.smithi097.stdout:2/445: rename d3/d4/l5 to d3/l7b 0 2024-09-16T08:57:10.884 INFO:tasks.workunit.client.0.smithi031.stdout:7/143: creat d2/f35 x:0 0 0 2024-09-16T08:57:10.885 INFO:tasks.workunit.client.1.smithi097.stdout:5/484: rmdir db/d18/d1c/d42 39 2024-09-16T08:57:10.886 INFO:tasks.workunit.client.0.smithi031.stdout:6/110: symlink d0/d17/d19/l1a 0 2024-09-16T08:57:10.888 INFO:tasks.workunit.client.0.smithi031.stdout:6/111: dread d0/f8 [0,4194304] 0 2024-09-16T08:57:10.898 INFO:tasks.workunit.client.1.smithi097.stdout:1/515: mkdir d5/d17/d54/d6b/da8 0 2024-09-16T08:57:10.899 INFO:tasks.workunit.client.1.smithi097.stdout:1/516: dread - d5/d70/fa3 zero size 2024-09-16T08:57:10.899 INFO:tasks.workunit.client.1.smithi097.stdout:1/517: truncate d5/db/d25/f45 1089577 0 2024-09-16T08:57:10.899 INFO:tasks.workunit.client.1.smithi097.stdout:1/518: readlink d5/d17/d54/l56 0 2024-09-16T08:57:10.899 INFO:tasks.workunit.client.1.smithi097.stdout:2/446: read d3/d4/d9/f4f [8113595,4262] 0 2024-09-16T08:57:10.899 INFO:tasks.workunit.client.1.smithi097.stdout:2/447: read d3/d4/d9/f5a [395423,1739] 0 2024-09-16T08:57:10.901 INFO:tasks.workunit.client.1.smithi097.stdout:5/485: truncate db/d18/d1c/d42/d46/d7e/f78 187364 0 2024-09-16T08:57:10.903 INFO:tasks.workunit.client.1.smithi097.stdout:5/486: mknod db/d22/d58/c9a 0 2024-09-16T08:57:10.905 INFO:tasks.workunit.client.0.smithi031.stdout:5/120: symlink d2/dc/d1d/l23 0 2024-09-16T08:57:10.905 INFO:tasks.workunit.client.0.smithi031.stdout:5/121: readlink d2/dc/d1d/l23 0 2024-09-16T08:57:10.906 INFO:tasks.workunit.client.1.smithi097.stdout:2/448: mkdir d3/d4/d77/d48/d7c 0 2024-09-16T08:57:10.907 INFO:tasks.workunit.client.0.smithi031.stdout:0/117: dwrite d6/f7 [0,4194304] 0 2024-09-16T08:57:10.911 INFO:tasks.workunit.client.1.smithi097.stdout:5/487: creat db/d18/d1c/d42/d46/f9b x:0 0 0 2024-09-16T08:57:10.911 INFO:tasks.workunit.client.1.smithi097.stdout:5/488: write db/f56 [804769,104584] 0 2024-09-16T08:57:10.911 INFO:tasks.workunit.client.1.smithi097.stdout:5/489: chown db/d1a/f3a 202587 1 2024-09-16T08:57:10.912 INFO:tasks.workunit.client.1.smithi097.stdout:2/449: write d3/d4/dd/f1f [395464,116794] 0 2024-09-16T08:57:10.912 INFO:tasks.workunit.client.1.smithi097.stdout:1/519: read d5/d14/d95/fa0 [1289698,64445] 0 2024-09-16T08:57:10.912 INFO:tasks.workunit.client.0.smithi031.stdout:3/74: truncate d3/dc/f16 5686104 0 2024-09-16T08:57:10.912 INFO:tasks.workunit.client.0.smithi031.stdout:3/75: dread - d3/f5 zero size 2024-09-16T08:57:10.913 INFO:tasks.workunit.client.0.smithi031.stdout:3/76: fsync d3/f5 0 2024-09-16T08:57:10.913 INFO:tasks.workunit.client.0.smithi031.stdout:7/144: mknod d2/d2d/c36 0 2024-09-16T08:57:10.913 INFO:tasks.workunit.client.0.smithi031.stdout:7/145: read - d2/d8/f32 zero size 2024-09-16T08:57:10.913 INFO:tasks.workunit.client.1.smithi097.stdout:5/490: link db/d22/f4b db/d18/d1c/d42/d46/d70/f9c 0 2024-09-16T08:57:10.913 INFO:tasks.workunit.client.1.smithi097.stdout:5/491: fsync db/d1a/f92 0 2024-09-16T08:57:10.914 INFO:tasks.workunit.client.0.smithi031.stdout:2/60: rmdir d6/da 39 2024-09-16T08:57:10.914 INFO:tasks.workunit.client.0.smithi031.stdout:2/61: readlink - no filename 2024-09-16T08:57:10.916 INFO:tasks.workunit.client.1.smithi097.stdout:6/497: dwrite d1/dc/f72 [0,4194304] 0 2024-09-16T08:57:10.917 INFO:tasks.workunit.client.0.smithi031.stdout:6/112: dwrite d0/d10/f13 [0,4194304] 0 2024-09-16T08:57:10.917 INFO:tasks.workunit.client.0.smithi031.stdout:5/122: mknod d2/dc/d1d/c24 0 2024-09-16T08:57:10.917 INFO:tasks.workunit.client.0.smithi031.stdout:5/123: truncate d2/dc/f20 481308 0 2024-09-16T08:57:10.917 INFO:tasks.workunit.client.1.smithi097.stdout:1/520: mknod d5/d17/d54/ca9 0 2024-09-16T08:57:10.920 INFO:tasks.workunit.client.0.smithi031.stdout:5/124: dread d2/f5 [0,4194304] 0 2024-09-16T08:57:10.921 INFO:tasks.workunit.client.1.smithi097.stdout:5/492: symlink db/d18/d1c/d42/d46/d7e/l9d 0 2024-09-16T08:57:10.921 INFO:tasks.workunit.client.0.smithi031.stdout:9/45: rmdir d3 39 2024-09-16T08:57:10.923 INFO:tasks.workunit.client.0.smithi031.stdout:3/77: symlink d3/d14/l1b 0 2024-09-16T08:57:10.926 INFO:tasks.workunit.client.0.smithi031.stdout:7/146: rename d2/c5 to d2/d8/d11/c37 0 2024-09-16T08:57:10.926 INFO:tasks.workunit.client.0.smithi031.stdout:7/147: readlink d2/d8/d11/l20 0 2024-09-16T08:57:10.928 INFO:tasks.workunit.client.0.smithi031.stdout:2/62: creat d6/f13 x:0 0 0 2024-09-16T08:57:10.931 INFO:tasks.workunit.client.1.smithi097.stdout:6/498: dwrite d1/d3/d16/f89 [0,4194304] 0 2024-09-16T08:57:10.933 INFO:tasks.workunit.client.0.smithi031.stdout:0/118: getdents d6 0 2024-09-16T08:57:10.933 INFO:tasks.workunit.client.0.smithi031.stdout:0/119: chown d6/d12/d13 1148 1 2024-09-16T08:57:10.933 INFO:tasks.workunit.client.0.smithi031.stdout:0/120: fdatasync d6/f1e 0 2024-09-16T08:57:10.935 INFO:tasks.workunit.client.1.smithi097.stdout:1/521: symlink d5/d17/d54/d6b/d71/laa 0 2024-09-16T08:57:10.936 INFO:tasks.workunit.client.0.smithi031.stdout:9/46: symlink d3/d5/da/db/l10 0 2024-09-16T08:57:10.936 INFO:tasks.workunit.client.0.smithi031.stdout:9/47: truncate d3/d5/da/db/fe 595851 0 2024-09-16T08:57:10.937 INFO:tasks.workunit.client.1.smithi097.stdout:2/450: rename d3/d4/dd to d3/d4/d77/d48/d7c/d7d 0 2024-09-16T08:57:10.939 INFO:tasks.workunit.client.0.smithi031.stdout:6/113: rename d0/d17/d19/l1a to d0/d10/d14/l1b 0 2024-09-16T08:57:10.939 INFO:tasks.workunit.client.0.smithi031.stdout:3/78: rename d3 to d3/dc/d1c 22 2024-09-16T08:57:10.939 INFO:tasks.workunit.client.0.smithi031.stdout:9/48: rename d3 to d3/d5/d11 22 2024-09-16T08:57:10.940 INFO:tasks.workunit.client.0.smithi031.stdout:2/63: symlink d6/da/l14 0 2024-09-16T08:57:10.942 INFO:tasks.workunit.client.0.smithi031.stdout:0/121: symlink d6/d12/l2b 0 2024-09-16T08:57:10.943 INFO:tasks.workunit.client.1.smithi097.stdout:6/499: mknod d1/d3/c9c 0 2024-09-16T08:57:10.944 INFO:tasks.workunit.client.0.smithi031.stdout:5/125: dwrite d2/f1b [0,4194304] 0 2024-09-16T08:57:10.944 INFO:tasks.workunit.client.0.smithi031.stdout:5/126: truncate d2/dc/f1f 906884 0 2024-09-16T08:57:10.944 INFO:tasks.workunit.client.0.smithi031.stdout:5/127: chown d2/f1b 103013 1 2024-09-16T08:57:10.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/64: write f4 [2188226,87322] 0 2024-09-16T08:57:10.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/65: write d6/fc [152402,128516] 0 2024-09-16T08:57:10.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/66: read - d6/da/ff zero size 2024-09-16T08:57:10.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/67: write f4 [1115016,129639] 0 2024-09-16T08:57:10.945 INFO:tasks.workunit.client.1.smithi097.stdout:1/522: mknod d5/d70/d9d/cab 0 2024-09-16T08:57:10.947 INFO:tasks.workunit.client.0.smithi031.stdout:6/114: creat d0/d17/d19/f1c x:0 0 0 2024-09-16T08:57:10.948 INFO:tasks.workunit.client.0.smithi031.stdout:7/148: dwrite d2/d8/f2c [0,4194304] 0 2024-09-16T08:57:10.949 INFO:tasks.workunit.client.1.smithi097.stdout:5/493: rename db/d18/d1c/d42/d46/d7e/d64/f63 to db/d18/d1c/d42/d46/d7e/f9e 0 2024-09-16T08:57:10.949 INFO:tasks.workunit.client.1.smithi097.stdout:5/494: dread - db/d18/d20/d2b/f6f zero size 2024-09-16T08:57:10.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/79: mkdir d3/df/d1a/d1d 0 2024-09-16T08:57:10.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/80: dread - d3/f5 zero size 2024-09-16T08:57:10.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/81: chown d3/df/f10 49283599 1 2024-09-16T08:57:10.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/82: fsync d3/df/f10 0 2024-09-16T08:57:10.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/83: fdatasync d3/dc/f13 0 2024-09-16T08:57:10.951 INFO:tasks.workunit.client.1.smithi097.stdout:2/451: link d3/f71 d3/d4/f7e 0 2024-09-16T08:57:10.951 INFO:tasks.workunit.client.1.smithi097.stdout:2/452: write d3/d4/d8/f2b [4194668,86201] 0 2024-09-16T08:57:10.951 INFO:tasks.workunit.client.1.smithi097.stdout:2/453: readlink d3/d4/d77/d48/d7c/d7d/l28 0 2024-09-16T08:57:10.952 INFO:tasks.workunit.client.0.smithi031.stdout:9/49: symlink d3/d5/l12 0 2024-09-16T08:57:10.952 INFO:tasks.workunit.client.0.smithi031.stdout:9/50: read d3/d5/da/db/fe [281215,3009] 0 2024-09-16T08:57:10.952 INFO:tasks.workunit.client.0.smithi031.stdout:9/51: dread - d3/f9 zero size 2024-09-16T08:57:10.954 INFO:tasks.workunit.client.1.smithi097.stdout:6/500: unlink d1/dc/d3c/c90 0 2024-09-16T08:57:10.955 INFO:tasks.workunit.client.1.smithi097.stdout:6/501: read d1/d3/d16/f7b [370416,34453] 0 2024-09-16T08:57:10.956 INFO:tasks.workunit.client.0.smithi031.stdout:0/122: dread d6/f7 [0,4194304] 0 2024-09-16T08:57:10.956 INFO:tasks.workunit.client.0.smithi031.stdout:0/123: creat d6/d12/f2c x:0 0 0 2024-09-16T08:57:10.957 INFO:tasks.workunit.client.1.smithi097.stdout:1/523: creat d5/d14/d95/fac x:0 0 0 2024-09-16T08:57:10.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/115: write d0/ff [1004776,106254] 0 2024-09-16T08:57:10.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/116: stat d0/d10 0 2024-09-16T08:57:10.960 INFO:tasks.workunit.client.1.smithi097.stdout:5/495: mkdir db/d18/d1c/d42/d46/d70/d9f 0 2024-09-16T08:57:10.960 INFO:tasks.workunit.client.1.smithi097.stdout:5/496: truncate db/d18/d20/d2b/f7c 196051 0 2024-09-16T08:57:10.964 INFO:tasks.workunit.client.1.smithi097.stdout:4/461: sync 2024-09-16T08:57:10.964 INFO:tasks.workunit.client.1.smithi097.stdout:7/442: sync 2024-09-16T08:57:10.964 INFO:tasks.workunit.client.1.smithi097.stdout:3/475: sync 2024-09-16T08:57:10.964 INFO:tasks.workunit.client.1.smithi097.stdout:0/504: sync 2024-09-16T08:57:10.964 INFO:tasks.workunit.client.1.smithi097.stdout:7/443: creat d4/d1b/d41/f95 x:0 0 0 2024-09-16T08:57:10.964 INFO:tasks.workunit.client.1.smithi097.stdout:3/476: creat d7/de/d3c/d3f/d60/d6b/fa3 x:0 0 0 2024-09-16T08:57:10.964 INFO:tasks.workunit.client.1.smithi097.stdout:9/409: sync 2024-09-16T08:57:10.965 INFO:tasks.workunit.client.1.smithi097.stdout:2/454: mkdir d3/d4/d68/d7f 0 2024-09-16T08:57:10.966 INFO:tasks.workunit.client.1.smithi097.stdout:6/502: symlink d1/dc/l9d 0 2024-09-16T08:57:10.968 INFO:tasks.workunit.client.0.smithi031.stdout:6/117: fdatasync d0/ff 0 2024-09-16T08:57:10.968 INFO:tasks.workunit.client.0.smithi031.stdout:6/118: write d0/f15 [910528,106665] 0 2024-09-16T08:57:10.968 INFO:tasks.workunit.client.0.smithi031.stdout:9/52: dwrite d3/fc [0,4194304] 0 2024-09-16T08:57:10.969 INFO:tasks.workunit.client.0.smithi031.stdout:2/68: creat d6/da/f15 x:0 0 0 2024-09-16T08:57:10.970 INFO:tasks.workunit.client.1.smithi097.stdout:5/497: mknod db/d18/d1c/d42/d46/d70/d90/ca0 0 2024-09-16T08:57:10.976 INFO:tasks.workunit.client.1.smithi097.stdout:4/462: link d0/d7/c7f d0/d7/d52/d26/d3d/ca2 0 2024-09-16T08:57:10.977 INFO:tasks.workunit.client.0.smithi031.stdout:7/149: symlink d2/d2d/l38 0 2024-09-16T08:57:10.978 INFO:tasks.workunit.client.0.smithi031.stdout:6/119: write d0/fb [3751048,93004] 0 2024-09-16T08:57:10.978 INFO:tasks.workunit.client.1.smithi097.stdout:5/498: read db/d18/d1c/d42/d46/d7e/f53 [1271533,92257] 0 2024-09-16T08:57:10.978 INFO:tasks.workunit.client.1.smithi097.stdout:5/499: readlink db/l4f 0 2024-09-16T08:57:10.978 INFO:tasks.workunit.client.0.smithi031.stdout:9/53: dread d3/d5/f6 [0,4194304] 0 2024-09-16T08:57:10.979 INFO:tasks.workunit.client.0.smithi031.stdout:3/84: truncate d3/dc/f16 3934542 0 2024-09-16T08:57:10.982 INFO:tasks.workunit.client.1.smithi097.stdout:0/505: dwrite da/d24/d4b/f52 [0,4194304] 0 2024-09-16T08:57:10.982 INFO:tasks.workunit.client.1.smithi097.stdout:7/444: symlink d4/d9/d51/l96 0 2024-09-16T08:57:10.983 INFO:tasks.workunit.client.0.smithi031.stdout:0/124: rmdir d6 39 2024-09-16T08:57:10.984 INFO:tasks.workunit.client.1.smithi097.stdout:3/477: mknod d7/de/d3c/d4f/ca4 0 2024-09-16T08:57:10.984 INFO:tasks.workunit.client.1.smithi097.stdout:3/478: write d7/de/d6c/f6f [232791,101714] 0 2024-09-16T08:57:10.985 INFO:tasks.workunit.client.1.smithi097.stdout:9/410: symlink d2/d3f/d62/l80 0 2024-09-16T08:57:10.988 INFO:tasks.workunit.client.0.smithi031.stdout:9/54: write d3/d5/da/db/fe [2996046,74587] 0 2024-09-16T08:57:10.988 INFO:tasks.workunit.client.0.smithi031.stdout:9/55: chown d3/d5/da/db 146149 1 2024-09-16T08:57:10.988 INFO:tasks.workunit.client.0.smithi031.stdout:9/56: readlink d3/d5/da/db/l10 0 2024-09-16T08:57:10.989 INFO:tasks.workunit.client.1.smithi097.stdout:2/455: unlink d3/f27 0 2024-09-16T08:57:10.990 INFO:tasks.workunit.client.1.smithi097.stdout:6/503: mkdir d1/dc/d1e/d9e 0 2024-09-16T08:57:10.995 INFO:tasks.workunit.client.0.smithi031.stdout:9/57: dread d3/d5/da/db/fe [0,4194304] 0 2024-09-16T08:57:10.995 INFO:tasks.workunit.client.0.smithi031.stdout:9/58: read d3/fc [1464114,24503] 0 2024-09-16T08:57:10.998 INFO:tasks.workunit.client.1.smithi097.stdout:6/504: dread d1/dc/d3c/f76 [0,4194304] 0 2024-09-16T08:57:10.998 INFO:tasks.workunit.client.1.smithi097.stdout:4/463: creat d0/d39/fa3 x:0 0 0 2024-09-16T08:57:10.999 INFO:tasks.workunit.client.0.smithi031.stdout:5/128: rename d2/d18 to d2/dc/d25 0 2024-09-16T08:57:11.000 INFO:tasks.workunit.client.1.smithi097.stdout:5/500: mknod db/d18/d1c/ca1 0 2024-09-16T08:57:11.000 INFO:tasks.workunit.client.1.smithi097.stdout:5/501: chown db/d18 5481 1 2024-09-16T08:57:11.004 INFO:tasks.workunit.client.0.smithi031.stdout:2/69: rmdir d6 39 2024-09-16T08:57:11.005 INFO:tasks.workunit.client.0.smithi031.stdout:3/85: read d3/dc/f16 [3710349,85217] 0 2024-09-16T08:57:11.005 INFO:tasks.workunit.client.0.smithi031.stdout:3/86: read d3/fa [3074328,30246] 0 2024-09-16T08:57:11.005 INFO:tasks.workunit.client.0.smithi031.stdout:3/87: chown d3/f4 7091 1 2024-09-16T08:57:11.007 INFO:tasks.workunit.client.1.smithi097.stdout:7/445: rename d4/d9/d1c/d5f/l7d to d4/d9/d51/l97 0 2024-09-16T08:57:11.010 INFO:tasks.workunit.client.1.smithi097.stdout:3/479: creat d7/d28/d30/d88/fa5 x:0 0 0 2024-09-16T08:57:11.012 INFO:tasks.workunit.client.1.smithi097.stdout:4/464: dwrite d0/f4 [4194304,4194304] 0 2024-09-16T08:57:11.015 INFO:tasks.workunit.client.0.smithi031.stdout:7/150: rmdir d2 39 2024-09-16T08:57:11.015 INFO:tasks.workunit.client.0.smithi031.stdout:7/151: dread - d2/f1a zero size 2024-09-16T08:57:11.015 INFO:tasks.workunit.client.0.smithi031.stdout:7/152: write d2/f35 [42674,3082] 0 2024-09-16T08:57:11.016 INFO:tasks.workunit.client.1.smithi097.stdout:2/456: creat d3/d4/d9/d67/f80 x:0 0 0 2024-09-16T08:57:11.018 INFO:tasks.workunit.client.0.smithi031.stdout:9/59: creat d3/d5/f13 x:0 0 0 2024-09-16T08:57:11.018 INFO:tasks.workunit.client.0.smithi031.stdout:9/60: write d3/d5/f13 [1029473,128255] 0 2024-09-16T08:57:11.019 INFO:tasks.workunit.client.0.smithi031.stdout:0/125: rmdir d6/d12/d13 39 2024-09-16T08:57:11.020 INFO:tasks.workunit.client.1.smithi097.stdout:4/465: dread d0/d7/df/f13 [0,4194304] 0 2024-09-16T08:57:11.020 INFO:tasks.workunit.client.1.smithi097.stdout:4/466: write d0/d36/d4c/f68 [1213600,99862] 0 2024-09-16T08:57:11.020 INFO:tasks.workunit.client.1.smithi097.stdout:4/467: write d0/d7/df/d15/d1f/f81 [4205292,122222] 0 2024-09-16T08:57:11.020 INFO:tasks.workunit.client.1.smithi097.stdout:4/468: write d0/d7/d52/f78 [3063455,51595] 0 2024-09-16T08:57:11.021 INFO:tasks.workunit.client.0.smithi031.stdout:6/120: rename d0/d10/f13 to d0/d17/d19/f1d 0 2024-09-16T08:57:11.022 INFO:tasks.workunit.client.1.smithi097.stdout:5/502: creat db/d18/d1c/d42/fa2 x:0 0 0 2024-09-16T08:57:11.023 INFO:tasks.workunit.client.1.smithi097.stdout:5/503: chown db/d1a/f2c 8 1 2024-09-16T08:57:11.023 INFO:tasks.workunit.client.0.smithi031.stdout:5/129: truncate d2/f1c 627945 0 2024-09-16T08:57:11.025 INFO:tasks.workunit.client.1.smithi097.stdout:3/480: write d7/de/d3c/d3f/f7b [2617556,96074] 0 2024-09-16T08:57:11.025 INFO:tasks.workunit.client.1.smithi097.stdout:3/481: chown d7/de/d29/d35/d3d/d57 427104 1 2024-09-16T08:57:11.025 INFO:tasks.workunit.client.1.smithi097.stdout:3/482: truncate d7/de/d29/d35/d3d/f87 592959 0 2024-09-16T08:57:11.025 INFO:tasks.workunit.client.1.smithi097.stdout:9/411: rename d2/d5/d3a/f6f to d2/d3f/d62/f81 0 2024-09-16T08:57:11.026 INFO:tasks.workunit.client.0.smithi031.stdout:2/70: creat d6/f16 x:0 0 0 2024-09-16T08:57:11.028 INFO:tasks.workunit.client.1.smithi097.stdout:7/446: dwrite d4/f64 [0,4194304] 0 2024-09-16T08:57:11.030 INFO:tasks.workunit.client.0.smithi031.stdout:3/88: mkdir d3/df/d1a/d1d/d1e 0 2024-09-16T08:57:11.031 INFO:tasks.workunit.client.1.smithi097.stdout:2/457: creat d3/d4/d77/d48/d7c/d7d/f81 x:0 0 0 2024-09-16T08:57:11.032 INFO:tasks.workunit.client.1.smithi097.stdout:2/458: stat d3/d4/d77/d48/d7c/d7d/f81 0 2024-09-16T08:57:11.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/412: write d2/d30/f5a [2045043,30431] 0 2024-09-16T08:57:11.032 INFO:tasks.workunit.client.0.smithi031.stdout:7/153: rmdir d2/d8/d11 39 2024-09-16T08:57:11.033 INFO:tasks.workunit.client.1.smithi097.stdout:5/504: dread db/d18/d1c/d42/d46/d7e/f9e [0,4194304] 0 2024-09-16T08:57:11.033 INFO:tasks.workunit.client.1.smithi097.stdout:5/505: fsync db/f40 0 2024-09-16T08:57:11.034 INFO:tasks.workunit.client.1.smithi097.stdout:6/505: truncate d1/d3/d16/f2e 10366 0 2024-09-16T08:57:11.035 INFO:tasks.workunit.client.0.smithi031.stdout:9/61: creat d3/d5/da/f14 x:0 0 0 2024-09-16T08:57:11.035 INFO:tasks.workunit.client.0.smithi031.stdout:9/62: truncate d3/d5/da/db/fe 4305678 0 2024-09-16T08:57:11.035 INFO:tasks.workunit.client.0.smithi031.stdout:9/63: chown d3/d5/da/db/l10 394397 1 2024-09-16T08:57:11.039 INFO:tasks.workunit.client.1.smithi097.stdout:4/469: mkdir d0/d35/da4 0 2024-09-16T08:57:11.040 INFO:tasks.workunit.client.1.smithi097.stdout:4/470: creat d0/d7/df/d83/fa5 x:0 0 0 2024-09-16T08:57:11.042 INFO:tasks.workunit.client.0.smithi031.stdout:5/130: creat d2/dc/d25/d21/f26 x:0 0 0 2024-09-16T08:57:11.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/71: creat d6/da/f17 x:0 0 0 2024-09-16T08:57:11.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/72: write d6/da/fe [656842,108371] 0 2024-09-16T08:57:11.046 INFO:tasks.workunit.client.0.smithi031.stdout:2/73: dread - d6/da/f17 zero size 2024-09-16T08:57:11.048 INFO:tasks.workunit.client.0.smithi031.stdout:3/89: mknod d3/df/d1a/d1d/d1e/c1f 0 2024-09-16T08:57:11.048 INFO:tasks.workunit.client.0.smithi031.stdout:3/90: write d3/f5 [778701,128158] 0 2024-09-16T08:57:11.052 INFO:tasks.workunit.client.1.smithi097.stdout:2/459: dwrite d3/d4/d77/d48/d7c/d7d/d53/f6c [0,4194304] 0 2024-09-16T08:57:11.052 INFO:tasks.workunit.client.1.smithi097.stdout:2/460: write d3/d4/d8/d51/f63 [78803,103719] 0 2024-09-16T08:57:11.053 INFO:tasks.workunit.client.1.smithi097.stdout:7/447: mknod d4/d9/d1c/c98 0 2024-09-16T08:57:11.054 INFO:tasks.workunit.client.0.smithi031.stdout:9/64: symlink d3/d5/da/db/l15 0 2024-09-16T08:57:11.057 INFO:tasks.workunit.client.0.smithi031.stdout:5/131: dread d2/fe [0,4194304] 0 2024-09-16T08:57:11.058 INFO:tasks.workunit.client.0.smithi031.stdout:0/126: getdents d6/d12 0 2024-09-16T08:57:11.059 INFO:tasks.workunit.client.1.smithi097.stdout:9/413: symlink d2/d5/d72/l82 0 2024-09-16T08:57:11.059 INFO:tasks.workunit.client.0.smithi031.stdout:9/65: dread d3/d5/da/db/fe [0,4194304] 0 2024-09-16T08:57:11.060 INFO:tasks.workunit.client.0.smithi031.stdout:6/121: getdents d0 0 2024-09-16T08:57:11.060 INFO:tasks.workunit.client.0.smithi031.stdout:6/122: creat d0/d17/f1e x:0 0 0 2024-09-16T08:57:11.061 INFO:tasks.workunit.client.0.smithi031.stdout:6/123: write d0/f8 [939926,9258] 0 2024-09-16T08:57:11.063 INFO:tasks.workunit.client.0.smithi031.stdout:3/91: mkdir d3/d14/d20 0 2024-09-16T08:57:11.067 INFO:tasks.workunit.client.1.smithi097.stdout:3/483: rmdir d7/de/d3c/d3f/d60/d6b 39 2024-09-16T08:57:11.067 INFO:tasks.workunit.client.0.smithi031.stdout:7/154: getdents d2/d8/d11 0 2024-09-16T08:57:11.067 INFO:tasks.workunit.client.0.smithi031.stdout:7/155: truncate d2/ff 5077981 0 2024-09-16T08:57:11.068 INFO:tasks.workunit.client.1.smithi097.stdout:6/506: dwrite d1/dc/f7d [0,4194304] 0 2024-09-16T08:57:11.068 INFO:tasks.workunit.client.0.smithi031.stdout:5/132: link d2/dc/f16 d2/dc/f27 0 2024-09-16T08:57:11.070 INFO:tasks.workunit.client.0.smithi031.stdout:0/127: creat d6/d12/f2d x:0 0 0 2024-09-16T08:57:11.071 INFO:tasks.workunit.client.0.smithi031.stdout:9/66: link d3/d5/lf d3/d5/l16 0 2024-09-16T08:57:11.072 INFO:tasks.workunit.client.0.smithi031.stdout:9/67: write d3/fc [1103948,11620] 0 2024-09-16T08:57:11.072 INFO:tasks.workunit.client.1.smithi097.stdout:2/461: mknod d3/d1b/c82 0 2024-09-16T08:57:11.073 INFO:tasks.workunit.client.1.smithi097.stdout:2/462: read d3/d4/d77/d48/d7c/d7d/f1f [497959,37703] 0 2024-09-16T08:57:11.073 INFO:tasks.workunit.client.1.smithi097.stdout:2/463: write d3/d4/d68/f72 [694455,72037] 0 2024-09-16T08:57:11.074 INFO:tasks.workunit.client.1.smithi097.stdout:7/448: link d4/d12/l67 d4/d9/d51/d8b/l99 0 2024-09-16T08:57:11.074 INFO:tasks.workunit.client.1.smithi097.stdout:7/449: chown d4/f23 513560 1 2024-09-16T08:57:11.076 INFO:tasks.workunit.client.0.smithi031.stdout:7/156: symlink d2/l39 0 2024-09-16T08:57:11.077 INFO:tasks.workunit.client.0.smithi031.stdout:5/133: mknod d2/dc/d25/c28 0 2024-09-16T08:57:11.078 INFO:tasks.workunit.client.0.smithi031.stdout:9/68: getdents d3/d5/da/db 0 2024-09-16T08:57:11.078 INFO:tasks.workunit.client.0.smithi031.stdout:9/69: creat d3/d5/f17 x:0 0 0 2024-09-16T08:57:11.079 INFO:tasks.workunit.client.0.smithi031.stdout:9/70: write d3/d5/da/db/fe [4865104,58774] 0 2024-09-16T08:57:11.079 INFO:tasks.workunit.client.0.smithi031.stdout:9/71: truncate d3/d5/da/f14 994320 0 2024-09-16T08:57:11.079 INFO:tasks.workunit.client.0.smithi031.stdout:2/74: dwrite d6/f13 [0,4194304] 0 2024-09-16T08:57:11.080 INFO:tasks.workunit.client.1.smithi097.stdout:9/414: dwrite d2/f38 [0,4194304] 0 2024-09-16T08:57:11.081 INFO:tasks.workunit.client.1.smithi097.stdout:7/450: dread d4/d12/d1f/f21 [0,4194304] 0 2024-09-16T08:57:11.081 INFO:tasks.workunit.client.1.smithi097.stdout:4/471: dwrite d0/f30 [0,4194304] 0 2024-09-16T08:57:11.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/134: rmdir d2/dc/d1d 39 2024-09-16T08:57:11.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/135: write d2/dc/f20 [854997,32582] 0 2024-09-16T08:57:11.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/136: fdatasync d2/f1c 0 2024-09-16T08:57:11.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/137: creat d2/dc/d25/d21/f29 x:0 0 0 2024-09-16T08:57:11.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/138: chown d2 1983 1 2024-09-16T08:57:11.082 INFO:tasks.workunit.client.0.smithi031.stdout:3/92: dread d3/f5 [0,4194304] 0 2024-09-16T08:57:11.083 INFO:tasks.workunit.client.0.smithi031.stdout:0/128: read d6/f1e [3333194,58673] 0 2024-09-16T08:57:11.085 INFO:tasks.workunit.client.0.smithi031.stdout:9/72: creat d3/d5/da/db/f18 x:0 0 0 2024-09-16T08:57:11.085 INFO:tasks.workunit.client.0.smithi031.stdout:9/73: readlink d3/d5/l12 0 2024-09-16T08:57:11.090 INFO:tasks.workunit.client.0.smithi031.stdout:6/124: getdents d0/d17 0 2024-09-16T08:57:11.090 INFO:tasks.workunit.client.0.smithi031.stdout:6/125: read - d0/d17/f1e zero size 2024-09-16T08:57:11.092 INFO:tasks.workunit.client.1.smithi097.stdout:9/415: write d2/d5/fb [1886505,130361] 0 2024-09-16T08:57:11.094 INFO:tasks.workunit.client.0.smithi031.stdout:0/129: creat d6/d12/d13/f2e x:0 0 0 2024-09-16T08:57:11.097 INFO:tasks.workunit.client.0.smithi031.stdout:6/126: symlink d0/l1f 0 2024-09-16T08:57:11.097 INFO:tasks.workunit.client.0.smithi031.stdout:6/127: truncate d0/fc 1052552 0 2024-09-16T08:57:11.097 INFO:tasks.workunit.client.0.smithi031.stdout:6/128: creat d0/d10/f20 x:0 0 0 2024-09-16T08:57:11.097 INFO:tasks.workunit.client.0.smithi031.stdout:6/129: write d0/f4 [3949377,127332] 0 2024-09-16T08:57:11.098 INFO:tasks.workunit.client.0.smithi031.stdout:9/74: mknod d3/d5/da/db/c19 0 2024-09-16T08:57:11.101 INFO:tasks.workunit.client.1.smithi097.stdout:5/506: rmdir db 39 2024-09-16T08:57:11.101 INFO:tasks.workunit.client.1.smithi097.stdout:5/507: getdents db/d18/d1c/d42/d46/d7e/d8f 0 2024-09-16T08:57:11.101 INFO:tasks.workunit.client.1.smithi097.stdout:5/508: dread - db/d18/d1c/d42/d46/d7e/d76/f8e zero size 2024-09-16T08:57:11.101 INFO:tasks.workunit.client.1.smithi097.stdout:3/484: truncate d7/de/d6c/f37 2778229 0 2024-09-16T08:57:11.101 INFO:tasks.workunit.client.0.smithi031.stdout:5/139: dwrite d2/dc/d25/d21/f29 [0,4194304] 0 2024-09-16T08:57:11.105 INFO:tasks.workunit.client.0.smithi031.stdout:2/75: write f1 [2328536,76544] 0 2024-09-16T08:57:11.105 INFO:tasks.workunit.client.0.smithi031.stdout:7/157: dwrite d2/d8/d11/f24 [0,4194304] 0 2024-09-16T08:57:11.110 INFO:tasks.workunit.client.0.smithi031.stdout:9/75: symlink d3/d5/da/db/l1a 0 2024-09-16T08:57:11.110 INFO:tasks.workunit.client.1.smithi097.stdout:6/507: write d1/d3/f1c [2324817,43245] 0 2024-09-16T08:57:11.110 INFO:tasks.workunit.client.1.smithi097.stdout:6/508: stat d1/d3/d16/d4f/f78 0 2024-09-16T08:57:11.113 INFO:tasks.workunit.client.0.smithi031.stdout:7/158: mknod d2/c3a 0 2024-09-16T08:57:11.114 INFO:tasks.workunit.client.1.smithi097.stdout:2/464: unlink d3/d4/d9/l3b 0 2024-09-16T08:57:11.114 INFO:tasks.workunit.client.1.smithi097.stdout:2/465: creat d3/d1b/f83 x:0 0 0 2024-09-16T08:57:11.114 INFO:tasks.workunit.client.0.smithi031.stdout:9/76: symlink d3/d5/l1b 0 2024-09-16T08:57:11.114 INFO:tasks.workunit.client.0.smithi031.stdout:9/77: dread - d3/f9 zero size 2024-09-16T08:57:11.114 INFO:tasks.workunit.client.0.smithi031.stdout:9/78: fsync d3/d5/f17 0 2024-09-16T08:57:11.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/93: dread f1 [0,4194304] 0 2024-09-16T08:57:11.117 INFO:tasks.workunit.client.0.smithi031.stdout:2/76: fdatasync d6/da/fe 0 2024-09-16T08:57:11.117 INFO:tasks.workunit.client.0.smithi031.stdout:2/77: write d6/f16 [578499,9080] 0 2024-09-16T08:57:11.121 INFO:tasks.workunit.client.0.smithi031.stdout:3/94: dread d3/fb [0,4194304] 0 2024-09-16T08:57:11.125 INFO:tasks.workunit.client.0.smithi031.stdout:0/130: dwrite d6/f1e [0,4194304] 0 2024-09-16T08:57:11.131 INFO:tasks.workunit.client.1.smithi097.stdout:3/485: dwrite d7/fa [0,4194304] 0 2024-09-16T08:57:11.133 INFO:tasks.workunit.client.0.smithi031.stdout:6/130: dwrite d0/d17/f18 [0,4194304] 0 2024-09-16T08:57:11.133 INFO:tasks.workunit.client.1.smithi097.stdout:3/486: dread d7/de/d29/f76 [0,4194304] 0 2024-09-16T08:57:11.136 INFO:tasks.workunit.client.0.smithi031.stdout:7/159: symlink d2/l3b 0 2024-09-16T08:57:11.136 INFO:tasks.workunit.client.0.smithi031.stdout:7/160: write d2/d8/f30 [938563,32449] 0 2024-09-16T08:57:11.139 INFO:tasks.workunit.client.0.smithi031.stdout:9/79: symlink d3/d5/da/l1c 0 2024-09-16T08:57:11.140 INFO:tasks.workunit.client.0.smithi031.stdout:2/78: symlink d6/l18 0 2024-09-16T08:57:11.142 INFO:tasks.workunit.client.0.smithi031.stdout:5/140: dwrite d2/fa [0,4194304] 0 2024-09-16T08:57:11.142 INFO:tasks.workunit.client.0.smithi031.stdout:5/141: chown d2/dc/fd 634982975 1 2024-09-16T08:57:11.143 INFO:tasks.workunit.client.0.smithi031.stdout:3/95: dwrite d3/f5 [0,4194304] 0 2024-09-16T08:57:11.148 INFO:tasks.workunit.client.0.smithi031.stdout:5/142: dread d2/dc/f19 [0,4194304] 0 2024-09-16T08:57:11.153 INFO:tasks.workunit.client.0.smithi031.stdout:6/131: mknod d0/dd/c21 0 2024-09-16T08:57:11.154 INFO:tasks.workunit.client.0.smithi031.stdout:7/161: rmdir d2/d2d 39 2024-09-16T08:57:11.155 INFO:tasks.workunit.client.1.smithi097.stdout:7/451: rmdir d4/d7 39 2024-09-16T08:57:11.156 INFO:tasks.workunit.client.0.smithi031.stdout:2/79: mknod d6/da/c19 0 2024-09-16T08:57:11.156 INFO:tasks.workunit.client.0.smithi031.stdout:2/80: write d6/da/fd [557094,122472] 0 2024-09-16T08:57:11.158 INFO:tasks.workunit.client.1.smithi097.stdout:4/472: unlink d0/d36/d4c/f68 0 2024-09-16T08:57:11.158 INFO:tasks.workunit.client.1.smithi097.stdout:4/473: creat d0/d39/d71/fa6 x:0 0 0 2024-09-16T08:57:11.159 INFO:tasks.workunit.client.1.smithi097.stdout:9/416: symlink d2/d5/l83 0 2024-09-16T08:57:11.159 INFO:tasks.workunit.client.1.smithi097.stdout:9/417: stat d2/d3f/f55 0 2024-09-16T08:57:11.160 INFO:tasks.workunit.client.1.smithi097.stdout:5/509: creat db/d6e/d95/fa3 x:0 0 0 2024-09-16T08:57:11.162 INFO:tasks.workunit.client.1.smithi097.stdout:6/509: chown d1/d3/d81/f88 623473 1 2024-09-16T08:57:11.164 INFO:tasks.workunit.client.0.smithi031.stdout:3/96: dwrite d3/fa [4194304,4194304] 0 2024-09-16T08:57:11.165 INFO:tasks.workunit.client.0.smithi031.stdout:9/80: dwrite d3/f9 [0,4194304] 0 2024-09-16T08:57:11.165 INFO:tasks.workunit.client.0.smithi031.stdout:9/81: write d3/d5/da/db/f18 [16072,38290] 0 2024-09-16T08:57:11.167 INFO:tasks.workunit.client.0.smithi031.stdout:6/132: dread d0/d17/f18 [0,4194304] 0 2024-09-16T08:57:11.168 INFO:tasks.workunit.client.1.smithi097.stdout:2/466: unlink d3/d4/d8/f39 0 2024-09-16T08:57:11.171 INFO:tasks.workunit.client.0.smithi031.stdout:9/82: read f2 [2140573,6803] 0 2024-09-16T08:57:11.172 INFO:tasks.workunit.client.1.smithi097.stdout:7/452: dwrite d4/d1b/d41/f52 [0,4194304] 0 2024-09-16T08:57:11.173 INFO:tasks.workunit.client.1.smithi097.stdout:7/453: stat d4/d1b/c2f 0 2024-09-16T08:57:11.173 INFO:tasks.workunit.client.1.smithi097.stdout:7/454: creat d4/d9/d1c/d5f/f9a x:0 0 0 2024-09-16T08:57:11.175 INFO:tasks.workunit.client.1.smithi097.stdout:3/487: link d7/de/d3c/d3f/f6d d7/d28/d63/fa6 0 2024-09-16T08:57:11.175 INFO:tasks.workunit.client.0.smithi031.stdout:3/97: write d3/df/f10 [4850594,64453] 0 2024-09-16T08:57:11.177 INFO:tasks.workunit.client.1.smithi097.stdout:2/467: write d3/f32 [1687174,103176] 0 2024-09-16T08:57:11.177 INFO:tasks.workunit.client.1.smithi097.stdout:2/468: chown d3/d4/d8/c3f 19536 1 2024-09-16T08:57:11.177 INFO:tasks.workunit.client.1.smithi097.stdout:1/524: sync 2024-09-16T08:57:11.178 INFO:tasks.workunit.client.1.smithi097.stdout:8/477: sync 2024-09-16T08:57:11.178 INFO:tasks.workunit.client.1.smithi097.stdout:4/474: mkdir d0/d36/da7 0 2024-09-16T08:57:11.180 INFO:tasks.workunit.client.1.smithi097.stdout:5/510: creat db/d18/d1c/d42/d46/d70/d9f/fa4 x:0 0 0 2024-09-16T08:57:11.181 INFO:tasks.workunit.client.1.smithi097.stdout:5/511: write db/d22/f96 [314377,78752] 0 2024-09-16T08:57:11.183 INFO:tasks.workunit.client.1.smithi097.stdout:6/510: mkdir d1/d3/d16/d4f/d9f 0 2024-09-16T08:57:11.184 INFO:tasks.workunit.client.1.smithi097.stdout:0/506: sync 2024-09-16T08:57:11.188 INFO:tasks.workunit.client.1.smithi097.stdout:9/418: dwrite d2/d3f/f43 [0,4194304] 0 2024-09-16T08:57:11.192 INFO:tasks.workunit.client.1.smithi097.stdout:7/455: link d4/d9/dc/f72 d4/d9/d51/d88/f9b 0 2024-09-16T08:57:11.192 INFO:tasks.workunit.client.1.smithi097.stdout:4/475: dread d0/f3e [0,4194304] 0 2024-09-16T08:57:11.194 INFO:tasks.workunit.client.1.smithi097.stdout:1/525: rmdir d5/d70/d9d 39 2024-09-16T08:57:11.194 INFO:tasks.workunit.client.1.smithi097.stdout:1/526: truncate d5/f77 926814 0 2024-09-16T08:57:11.196 INFO:tasks.workunit.client.1.smithi097.stdout:8/478: unlink c0 0 2024-09-16T08:57:11.197 INFO:tasks.workunit.client.1.smithi097.stdout:5/512: rmdir db/d18/d20 39 2024-09-16T08:57:11.199 INFO:tasks.workunit.client.1.smithi097.stdout:0/507: symlink da/df/d13/db0/lb6 0 2024-09-16T08:57:11.201 INFO:tasks.workunit.client.0.smithi031.stdout:5/143: creat d2/dc/d1d/f2a x:0 0 0 2024-09-16T08:57:11.201 INFO:tasks.workunit.client.0.smithi031.stdout:5/144: write d2/dc/f27 [845751,13287] 0 2024-09-16T08:57:11.202 INFO:tasks.workunit.client.1.smithi097.stdout:9/419: mkdir d2/d30/d84 0 2024-09-16T08:57:11.203 INFO:tasks.workunit.client.0.smithi031.stdout:0/131: rename d6/d12/d13/c1b to d6/d12/d13/c2f 0 2024-09-16T08:57:11.203 INFO:tasks.workunit.client.0.smithi031.stdout:0/132: readlink d6/d12/l1c 0 2024-09-16T08:57:11.203 INFO:tasks.workunit.client.0.smithi031.stdout:0/133: stat d6/f1a 0 2024-09-16T08:57:11.204 INFO:tasks.workunit.client.1.smithi097.stdout:2/469: dread d3/f32 [0,4194304] 0 2024-09-16T08:57:11.204 INFO:tasks.workunit.client.1.smithi097.stdout:3/488: dwrite d7/de/d3c/f79 [0,4194304] 0 2024-09-16T08:57:11.204 INFO:tasks.workunit.client.1.smithi097.stdout:3/489: chown d7/de/d29/d35/d3d/c4b 1151152 1 2024-09-16T08:57:11.204 INFO:tasks.workunit.client.1.smithi097.stdout:0/508: dread da/d1a/d41/f50 [0,4194304] 0 2024-09-16T08:57:11.204 INFO:tasks.workunit.client.1.smithi097.stdout:3/490: fsync d7/de/d29/d35/d3d/f59 0 2024-09-16T08:57:11.204 INFO:tasks.workunit.client.1.smithi097.stdout:2/470: creat d3/d4/d9/d67/f84 x:0 0 0 2024-09-16T08:57:11.204 INFO:tasks.workunit.client.1.smithi097.stdout:7/456: getdents d4/d9/d51 0 2024-09-16T08:57:11.205 INFO:tasks.workunit.client.1.smithi097.stdout:3/491: read d7/de/d6c/f6f [85363,108232] 0 2024-09-16T08:57:11.205 INFO:tasks.workunit.client.0.smithi031.stdout:3/98: dwrite d3/df/f10 [4194304,4194304] 0 2024-09-16T08:57:11.205 INFO:tasks.workunit.client.0.smithi031.stdout:7/162: mknod d2/c3c 0 2024-09-16T08:57:11.206 INFO:tasks.workunit.client.0.smithi031.stdout:7/163: chown d2/d8/f1d 147 1 2024-09-16T08:57:11.206 INFO:tasks.workunit.client.1.smithi097.stdout:1/527: getdents d5/d2c/d3b/d7f 0 2024-09-16T08:57:11.206 INFO:tasks.workunit.client.1.smithi097.stdout:3/492: write d7/de/d6c/f66 [6394738,46104] 0 2024-09-16T08:57:11.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/81: mknod d6/c1a 0 2024-09-16T08:57:11.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/82: write d6/da/fe [1615373,57187] 0 2024-09-16T08:57:11.207 INFO:tasks.workunit.client.0.smithi031.stdout:0/134: dread d6/d12/f17 [0,4194304] 0 2024-09-16T08:57:11.212 INFO:tasks.workunit.client.1.smithi097.stdout:8/479: link d8/dd/d15/d2c/d4a/d4e/c67 d8/dd/d15/d2c/d48/d56/c91 0 2024-09-16T08:57:11.214 INFO:tasks.workunit.client.1.smithi097.stdout:5/513: truncate db/fe 5998474 0 2024-09-16T08:57:11.214 INFO:tasks.workunit.client.1.smithi097.stdout:5/514: fsync db/f40 0 2024-09-16T08:57:11.214 INFO:tasks.workunit.client.1.smithi097.stdout:5/515: write db/d18/d20/d2b/f75 [1251376,17207] 0 2024-09-16T08:57:11.214 INFO:tasks.workunit.client.1.smithi097.stdout:5/516: write db/d18/d1c/d42/d46/d7e/d64/d7f/f85 [291571,119380] 0 2024-09-16T08:57:11.215 INFO:tasks.workunit.client.1.smithi097.stdout:5/517: chown db/d18/d20/f29 3236889 1 2024-09-16T08:57:11.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/83: mknod d3/d5/c1d 0 2024-09-16T08:57:11.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/84: write d3/d5/f13 [1224891,73105] 0 2024-09-16T08:57:11.219 INFO:tasks.workunit.client.1.smithi097.stdout:9/420: creat d2/d5/d3a/d3c/f85 x:0 0 0 2024-09-16T08:57:11.220 INFO:tasks.workunit.client.0.smithi031.stdout:5/145: write d2/f11 [819241,128839] 0 2024-09-16T08:57:11.220 INFO:tasks.workunit.client.0.smithi031.stdout:9/85: dread d3/d5/da/f14 [0,4194304] 0 2024-09-16T08:57:11.221 INFO:tasks.workunit.client.0.smithi031.stdout:3/99: dread d3/dc/f16 [0,4194304] 0 2024-09-16T08:57:11.224 INFO:tasks.workunit.client.1.smithi097.stdout:2/471: rename d3/d4/d77/d48/d7c/d7d/c10 to d3/d4/d77/d48/d7c/d7d/d53/d47/c85 0 2024-09-16T08:57:11.224 INFO:tasks.workunit.client.1.smithi097.stdout:2/472: write d3/d1b/f5f [1373186,14271] 0 2024-09-16T08:57:11.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/164: dread d2/d8/fd [0,4194304] 0 2024-09-16T08:57:11.230 INFO:tasks.workunit.client.0.smithi031.stdout:3/100: dread d3/fa [4194304,4194304] 0 2024-09-16T08:57:11.234 INFO:tasks.workunit.client.1.smithi097.stdout:4/476: dwrite d0/f30 [0,4194304] 0 2024-09-16T08:57:11.234 INFO:tasks.workunit.client.1.smithi097.stdout:3/493: creat d7/de/d29/fa7 x:0 0 0 2024-09-16T08:57:11.235 INFO:tasks.workunit.client.0.smithi031.stdout:2/83: mknod d6/da/c1b 0 2024-09-16T08:57:11.235 INFO:tasks.workunit.client.1.smithi097.stdout:1/528: dwrite d5/db/dc/f7d [0,4194304] 0 2024-09-16T08:57:11.236 INFO:tasks.workunit.client.1.smithi097.stdout:8/480: mknod d8/dd/d15/d16/d7c/c92 0 2024-09-16T08:57:11.237 INFO:tasks.workunit.client.0.smithi031.stdout:0/135: creat d6/d12/d13/f30 x:0 0 0 2024-09-16T08:57:11.238 INFO:tasks.workunit.client.1.smithi097.stdout:5/518: getdents db/d18/d1c/d42/d46 0 2024-09-16T08:57:11.239 INFO:tasks.workunit.client.0.smithi031.stdout:6/133: truncate d0/f8 373821 0 2024-09-16T08:57:11.242 INFO:tasks.workunit.client.1.smithi097.stdout:9/421: unlink d2/d5/d3a/f3d 0 2024-09-16T08:57:11.242 INFO:tasks.workunit.client.1.smithi097.stdout:9/422: write d2/d30/d63/f77 [687163,21780] 0 2024-09-16T08:57:11.243 INFO:tasks.workunit.client.1.smithi097.stdout:1/529: dread d5/d14/d95/fa0 [0,4194304] 0 2024-09-16T08:57:11.243 INFO:tasks.workunit.client.1.smithi097.stdout:1/530: dread - d5/db/dc/d19/f8c zero size 2024-09-16T08:57:11.244 INFO:tasks.workunit.client.1.smithi097.stdout:2/473: rename d3/l45 to d3/d4/d77/d48/d7c/d7d/l86 0 2024-09-16T08:57:11.245 INFO:tasks.workunit.client.0.smithi031.stdout:4/47: sync 2024-09-16T08:57:11.246 INFO:tasks.workunit.client.1.smithi097.stdout:0/509: dwrite da/d1a/d41/f71 [0,4194304] 0 2024-09-16T08:57:11.246 INFO:tasks.workunit.client.1.smithi097.stdout:8/481: read d8/dd/d15/d2c/f74 [3037588,115321] 0 2024-09-16T08:57:11.246 INFO:tasks.workunit.client.0.smithi031.stdout:7/165: rename d2/l23 to d2/l3d 0 2024-09-16T08:57:11.251 INFO:tasks.workunit.client.1.smithi097.stdout:7/457: rmdir d4/d7/d79 39 2024-09-16T08:57:11.251 INFO:tasks.workunit.client.0.smithi031.stdout:3/101: symlink d3/df/d1a/d1d/d1e/l21 0 2024-09-16T08:57:11.251 INFO:tasks.workunit.client.0.smithi031.stdout:3/102: write d3/fb [1390247,47161] 0 2024-09-16T08:57:11.255 INFO:tasks.workunit.client.0.smithi031.stdout:6/134: rename d0/d10/f20 to d0/dd/f22 0 2024-09-16T08:57:11.256 INFO:tasks.workunit.client.1.smithi097.stdout:4/477: link d0/d7/d52/d26/d3d/ca2 d0/d35/da4/ca8 0 2024-09-16T08:57:11.257 INFO:tasks.workunit.client.0.smithi031.stdout:7/166: rmdir d2/d8/d11 39 2024-09-16T08:57:11.257 INFO:tasks.workunit.client.0.smithi031.stdout:6/135: write d0/f15 [425261,47026] 0 2024-09-16T08:57:11.259 INFO:tasks.workunit.client.1.smithi097.stdout:3/494: mkdir d7/de/d8d/d90/d92/da8 0 2024-09-16T08:57:11.259 INFO:tasks.workunit.client.1.smithi097.stdout:3/495: write d7/de/d29/d35/f67 [650209,7379] 0 2024-09-16T08:57:11.261 INFO:tasks.workunit.client.0.smithi031.stdout:3/103: mknod d3/d14/c22 0 2024-09-16T08:57:11.261 INFO:tasks.workunit.client.0.smithi031.stdout:3/104: fsync d3/f4 0 2024-09-16T08:57:11.261 INFO:tasks.workunit.client.0.smithi031.stdout:3/105: stat d3/d14 0 2024-09-16T08:57:11.261 INFO:tasks.workunit.client.0.smithi031.stdout:3/106: chown d3/le 0 1 2024-09-16T08:57:11.261 INFO:tasks.workunit.client.0.smithi031.stdout:3/107: rename d3/d14/d20 to d3/d14/d20/d23 22 2024-09-16T08:57:11.261 INFO:tasks.workunit.client.0.smithi031.stdout:3/108: read f1 [2993326,5662] 0 2024-09-16T08:57:11.262 INFO:tasks.workunit.client.1.smithi097.stdout:3/496: read d7/de/d3c/d3f/d60/d6b/f86 [517390,55134] 0 2024-09-16T08:57:11.262 INFO:tasks.workunit.client.1.smithi097.stdout:3/497: readlink d7/de/d6c/l39 0 2024-09-16T08:57:11.262 INFO:tasks.workunit.client.1.smithi097.stdout:3/498: chown d7/de/d29/d35/d3d/f59 29163554 1 2024-09-16T08:57:11.263 INFO:tasks.workunit.client.1.smithi097.stdout:5/519: truncate db/d18/d20/f29 181954 0 2024-09-16T08:57:11.265 INFO:tasks.workunit.client.0.smithi031.stdout:7/167: dread d2/d8/d11/f24 [0,4194304] 0 2024-09-16T08:57:11.265 INFO:tasks.workunit.client.1.smithi097.stdout:9/423: mknod d2/d5/d3a/d3c/c86 0 2024-09-16T08:57:11.265 INFO:tasks.workunit.client.1.smithi097.stdout:9/424: read - d2/d5/d3a/d3c/d73/f7b zero size 2024-09-16T08:57:11.265 INFO:tasks.workunit.client.1.smithi097.stdout:9/425: fsync d2/f1a 0 2024-09-16T08:57:11.266 INFO:tasks.workunit.client.0.smithi031.stdout:7/168: dread d2/d8/d11/f1b [0,4194304] 0 2024-09-16T08:57:11.266 INFO:tasks.workunit.client.0.smithi031.stdout:3/109: mkdir d3/df/d1a/d24 0 2024-09-16T08:57:11.267 INFO:tasks.workunit.client.1.smithi097.stdout:1/531: rmdir d5/db/d25 39 2024-09-16T08:57:11.268 INFO:tasks.workunit.client.1.smithi097.stdout:2/474: mknod d3/d4/d68/c87 0 2024-09-16T08:57:11.273 INFO:tasks.workunit.client.1.smithi097.stdout:8/482: symlink d8/dd/d42/l93 0 2024-09-16T08:57:11.273 INFO:tasks.workunit.client.1.smithi097.stdout:3/499: dread d7/de/d3c/f79 [0,4194304] 0 2024-09-16T08:57:11.273 INFO:tasks.workunit.client.1.smithi097.stdout:8/483: write d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f84 [675183,129025] 0 2024-09-16T08:57:11.273 INFO:tasks.workunit.client.1.smithi097.stdout:8/484: dread - d8/dd/d15/d2c/d48/d56/f6d zero size 2024-09-16T08:57:11.275 INFO:tasks.workunit.client.1.smithi097.stdout:4/478: rename d0/d7/d52/d26/d8c/l9e to d0/d7/df/d15/d1f/la9 0 2024-09-16T08:57:11.276 INFO:tasks.workunit.client.1.smithi097.stdout:5/520: mknod db/d18/d1c/d42/d46/d7e/d8f/ca5 0 2024-09-16T08:57:11.276 INFO:tasks.workunit.client.1.smithi097.stdout:5/521: chown db/d18/d1c/d42/d46/d7e/d8f/ca5 32642 1 2024-09-16T08:57:11.277 INFO:tasks.workunit.client.0.smithi031.stdout:7/169: write d2/f22 [248137,55172] 0 2024-09-16T08:57:11.280 INFO:tasks.workunit.client.0.smithi031.stdout:7/170: fsync d2/d2d/f2e 0 2024-09-16T08:57:11.281 INFO:tasks.workunit.client.0.smithi031.stdout:1/119: sync 2024-09-16T08:57:11.281 INFO:tasks.workunit.client.0.smithi031.stdout:8/84: sync 2024-09-16T08:57:11.281 INFO:tasks.workunit.client.1.smithi097.stdout:1/532: creat d5/db/dc/fad x:0 0 0 2024-09-16T08:57:11.281 INFO:tasks.workunit.client.1.smithi097.stdout:5/522: dread db/d18/d1c/d42/d46/f6c [0,4194304] 0 2024-09-16T08:57:11.281 INFO:tasks.workunit.client.1.smithi097.stdout:5/523: chown db/fe 9040243 1 2024-09-16T08:57:11.281 INFO:tasks.workunit.client.1.smithi097.stdout:5/524: chown db/f40 505 1 2024-09-16T08:57:11.282 INFO:tasks.workunit.client.1.smithi097.stdout:2/475: creat d3/f88 x:0 0 0 2024-09-16T08:57:11.284 INFO:tasks.workunit.client.0.smithi031.stdout:7/171: symlink d2/l3e 0 2024-09-16T08:57:11.284 INFO:tasks.workunit.client.0.smithi031.stdout:7/172: write d2/d2d/f2e [803237,123225] 0 2024-09-16T08:57:11.285 INFO:tasks.workunit.client.0.smithi031.stdout:2/84: dwrite d6/f16 [0,4194304] 0 2024-09-16T08:57:11.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/110: dread d3/df/f10 [0,4194304] 0 2024-09-16T08:57:11.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/111: chown d3/d14/l1b 56060 1 2024-09-16T08:57:11.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/173: dread d2/ff [4194304,4194304] 0 2024-09-16T08:57:11.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/174: write d2/f1a [339209,11803] 0 2024-09-16T08:57:11.285 INFO:tasks.workunit.client.0.smithi031.stdout:7/175: dread - d2/d8/f32 zero size 2024-09-16T08:57:11.286 INFO:tasks.workunit.client.1.smithi097.stdout:0/510: rmdir da/df/d8a 39 2024-09-16T08:57:11.286 INFO:tasks.workunit.client.0.smithi031.stdout:3/112: read d3/fb [883845,31714] 0 2024-09-16T08:57:11.290 INFO:tasks.workunit.client.0.smithi031.stdout:6/136: fsync d0/dd/f22 0 2024-09-16T08:57:11.290 INFO:tasks.workunit.client.0.smithi031.stdout:6/137: read - d0/d17/d19/f1c zero size 2024-09-16T08:57:11.290 INFO:tasks.workunit.client.0.smithi031.stdout:6/138: fsync d0/d10/f11 0 2024-09-16T08:57:11.291 INFO:tasks.workunit.client.0.smithi031.stdout:6/139: write d0/dd/f22 [780548,25841] 0 2024-09-16T08:57:11.291 INFO:tasks.workunit.client.1.smithi097.stdout:4/479: creat d0/d36/da7/faa x:0 0 0 2024-09-16T08:57:11.294 INFO:tasks.workunit.client.0.smithi031.stdout:7/176: creat d2/d8/f3f x:0 0 0 2024-09-16T08:57:11.295 INFO:tasks.workunit.client.0.smithi031.stdout:8/85: getdents d4/d7 0 2024-09-16T08:57:11.295 INFO:tasks.workunit.client.0.smithi031.stdout:7/177: dread d2/d8/d11/f1b [0,4194304] 0 2024-09-16T08:57:11.295 INFO:tasks.workunit.client.0.smithi031.stdout:7/178: write d2/f1a [1391647,81728] 0 2024-09-16T08:57:11.299 INFO:tasks.workunit.client.0.smithi031.stdout:3/113: rename d3/f11 to d3/df/d1a/d24/f25 0 2024-09-16T08:57:11.300 INFO:tasks.workunit.client.0.smithi031.stdout:0/136: dwrite d6/d12/d13/f2e [0,4194304] 0 2024-09-16T08:57:11.300 INFO:tasks.workunit.client.0.smithi031.stdout:0/137: dread - d6/d12/f2d zero size 2024-09-16T08:57:11.300 INFO:tasks.workunit.client.0.smithi031.stdout:6/140: creat d0/dd/f23 x:0 0 0 2024-09-16T08:57:11.300 INFO:tasks.workunit.client.0.smithi031.stdout:6/141: truncate d0/dd/f22 865279 0 2024-09-16T08:57:11.300 INFO:tasks.workunit.client.0.smithi031.stdout:6/142: write d0/d10/f11 [435149,10589] 0 2024-09-16T08:57:11.302 INFO:tasks.workunit.client.0.smithi031.stdout:8/86: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:11.302 INFO:tasks.workunit.client.0.smithi031.stdout:8/87: dread - d4/f17 zero size 2024-09-16T08:57:11.304 INFO:tasks.workunit.client.0.smithi031.stdout:3/114: rmdir d3/df/d1a/d1d/d1e 39 2024-09-16T08:57:11.304 INFO:tasks.workunit.client.1.smithi097.stdout:9/426: rmdir d2 39 2024-09-16T08:57:11.306 INFO:tasks.workunit.client.1.smithi097.stdout:7/458: dwrite d4/d7/d7b/f40 [0,4194304] 0 2024-09-16T08:57:11.306 INFO:tasks.workunit.client.1.smithi097.stdout:1/533: symlink d5/d14/lae 0 2024-09-16T08:57:11.309 INFO:tasks.workunit.client.1.smithi097.stdout:5/525: creat db/d18/d1c/d42/d46/fa6 x:0 0 0 2024-09-16T08:57:11.311 INFO:tasks.workunit.client.0.smithi031.stdout:0/138: creat d6/f31 x:0 0 0 2024-09-16T08:57:11.312 INFO:tasks.workunit.client.1.smithi097.stdout:2/476: unlink d3/d4/d9/d67/f80 0 2024-09-16T08:57:11.313 INFO:tasks.workunit.client.0.smithi031.stdout:7/179: fsync d2/d8/f3f 0 2024-09-16T08:57:11.314 INFO:tasks.workunit.client.1.smithi097.stdout:5/526: read db/d18/d1c/d42/d46/d7e/f9e [2923901,114295] 0 2024-09-16T08:57:11.315 INFO:tasks.workunit.client.0.smithi031.stdout:8/88: creat d4/d7/f1a x:0 0 0 2024-09-16T08:57:11.315 INFO:tasks.workunit.client.0.smithi031.stdout:8/89: readlink d4/l16 0 2024-09-16T08:57:11.315 INFO:tasks.workunit.client.0.smithi031.stdout:9/86: dwrite d3/d5/da/db/fe [0,4194304] 0 2024-09-16T08:57:11.316 INFO:tasks.workunit.client.1.smithi097.stdout:8/485: getdents d8/dd/d71 0 2024-09-16T08:57:11.316 INFO:tasks.workunit.client.1.smithi097.stdout:8/486: fsync d8/f31 0 2024-09-16T08:57:11.318 INFO:tasks.workunit.client.0.smithi031.stdout:0/139: mknod d6/c32 0 2024-09-16T08:57:11.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/140: fdatasync d6/f9 0 2024-09-16T08:57:11.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/141: write d6/d12/d13/f24 [976891,73003] 0 2024-09-16T08:57:11.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/142: write d6/d12/f2c [951459,47790] 0 2024-09-16T08:57:11.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/143: stat d6/d12 0 2024-09-16T08:57:11.319 INFO:tasks.workunit.client.1.smithi097.stdout:9/427: creat d2/d3f/d62/f87 x:0 0 0 2024-09-16T08:57:11.320 INFO:tasks.workunit.client.0.smithi031.stdout:7/180: creat d2/d8/d11/f40 x:0 0 0 2024-09-16T08:57:11.321 INFO:tasks.workunit.client.0.smithi031.stdout:6/143: getdents d0/dd 0 2024-09-16T08:57:11.323 INFO:tasks.workunit.client.1.smithi097.stdout:7/459: dread d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:11.323 INFO:tasks.workunit.client.1.smithi097.stdout:7/460: dread - d4/d9/dc/f93 zero size 2024-09-16T08:57:11.324 INFO:tasks.workunit.client.0.smithi031.stdout:0/144: link d6/d12/d13/f23 d6/f33 0 2024-09-16T08:57:11.324 INFO:tasks.workunit.client.0.smithi031.stdout:0/145: fsync d6/d12/f17 0 2024-09-16T08:57:11.325 INFO:tasks.workunit.client.1.smithi097.stdout:1/534: unlink d5/db/dc/f48 0 2024-09-16T08:57:11.326 INFO:tasks.workunit.client.1.smithi097.stdout:1/535: write d5/db/f6f [554888,82203] 0 2024-09-16T08:57:11.327 INFO:tasks.workunit.client.0.smithi031.stdout:9/87: write d3/d5/da/db/fe [825312,91412] 0 2024-09-16T08:57:11.327 INFO:tasks.workunit.client.1.smithi097.stdout:2/477: symlink d3/d4/d8/l89 0 2024-09-16T08:57:11.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/144: creat d0/d10/d14/f24 x:0 0 0 2024-09-16T08:57:11.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/145: getdents d0/dd 0 2024-09-16T08:57:11.329 INFO:tasks.workunit.client.0.smithi031.stdout:6/146: write d0/fc [1572473,60558] 0 2024-09-16T08:57:11.330 INFO:tasks.workunit.client.0.smithi031.stdout:0/146: write d6/f7 [461419,40408] 0 2024-09-16T08:57:11.330 INFO:tasks.workunit.client.0.smithi031.stdout:0/147: fdatasync d6/f8 0 2024-09-16T08:57:11.330 INFO:tasks.workunit.client.1.smithi097.stdout:0/511: getdents da/d24/d76 0 2024-09-16T08:57:11.333 INFO:tasks.workunit.client.0.smithi031.stdout:7/181: symlink d2/d8/l41 0 2024-09-16T08:57:11.333 INFO:tasks.workunit.client.0.smithi031.stdout:7/182: chown d2/f18 139924789 1 2024-09-16T08:57:11.335 INFO:tasks.workunit.client.1.smithi097.stdout:8/487: truncate d8/dd/d15/f28 920915 0 2024-09-16T08:57:11.336 INFO:tasks.workunit.client.0.smithi031.stdout:5/146: dwrite d2/f11 [4194304,4194304] 0 2024-09-16T08:57:11.336 INFO:tasks.workunit.client.1.smithi097.stdout:3/500: dwrite d7/de/d29/d35/d3d/d57/f91 [0,4194304] 0 2024-09-16T08:57:11.337 INFO:tasks.workunit.client.0.smithi031.stdout:5/147: dread d2/f1c [0,4194304] 0 2024-09-16T08:57:11.337 INFO:tasks.workunit.client.0.smithi031.stdout:5/148: creat d2/f2b x:0 0 0 2024-09-16T08:57:11.337 INFO:tasks.workunit.client.0.smithi031.stdout:5/149: readlink d2/l9 0 2024-09-16T08:57:11.337 INFO:tasks.workunit.client.1.smithi097.stdout:9/428: unlink d2/d30/c5b 0 2024-09-16T08:57:11.338 INFO:tasks.workunit.client.0.smithi031.stdout:1/120: dwrite d2/de/f13 [0,4194304] 0 2024-09-16T08:57:11.338 INFO:tasks.workunit.client.0.smithi031.stdout:2/85: dwrite d6/f8 [0,4194304] 0 2024-09-16T08:57:11.339 INFO:tasks.workunit.client.0.smithi031.stdout:7/183: truncate d2/fb 2292209 0 2024-09-16T08:57:11.340 INFO:tasks.workunit.client.1.smithi097.stdout:7/461: symlink d4/d1b/l9c 0 2024-09-16T08:57:11.340 INFO:tasks.workunit.client.1.smithi097.stdout:7/462: chown d4/d1b/d41/l62 874 1 2024-09-16T08:57:11.341 INFO:tasks.workunit.client.1.smithi097.stdout:4/480: dwrite d0/d23/f3a [0,4194304] 0 2024-09-16T08:57:11.342 INFO:tasks.workunit.client.0.smithi031.stdout:9/88: link d3/fc d3/d5/da/db/f1e 0 2024-09-16T08:57:11.342 INFO:tasks.workunit.client.0.smithi031.stdout:3/115: dwrite d3/df/d1a/d24/f25 [4194304,4194304] 0 2024-09-16T08:57:11.346 INFO:tasks.workunit.client.1.smithi097.stdout:0/512: link da/d1a/d26/f59 da/d1a/d26/d47/db3/d60/d73/d85/dad/fb7 0 2024-09-16T08:57:11.348 INFO:tasks.workunit.client.1.smithi097.stdout:5/527: dwrite db/d1a/f38 [4194304,4194304] 0 2024-09-16T08:57:11.349 INFO:tasks.workunit.client.1.smithi097.stdout:1/536: dwrite d5/db/d25/f93 [0,4194304] 0 2024-09-16T08:57:11.349 INFO:tasks.workunit.client.1.smithi097.stdout:1/537: write d5/d17/d54/d6b/f9c [565248,32295] 0 2024-09-16T08:57:11.352 INFO:tasks.workunit.client.0.smithi031.stdout:3/116: dread d3/df/f10 [0,4194304] 0 2024-09-16T08:57:11.353 INFO:tasks.workunit.client.0.smithi031.stdout:5/150: mknod d2/dc/d25/c2c 0 2024-09-16T08:57:11.354 INFO:tasks.workunit.client.0.smithi031.stdout:5/151: chown d2/dc/d1d/f2a 4 1 2024-09-16T08:57:11.354 INFO:tasks.workunit.client.0.smithi031.stdout:5/152: fdatasync d2/f1c 0 2024-09-16T08:57:11.360 INFO:tasks.workunit.client.0.smithi031.stdout:2/86: mknod d6/da/c1c 0 2024-09-16T08:57:11.363 INFO:tasks.workunit.client.1.smithi097.stdout:8/488: dwrite d8/dd/d15/d2c/d48/d56/f6d [0,4194304] 0 2024-09-16T08:57:11.363 INFO:tasks.workunit.client.1.smithi097.stdout:8/489: fsync d8/dd/d15/d2c/f74 0 2024-09-16T08:57:11.364 INFO:tasks.workunit.client.0.smithi031.stdout:9/89: readlink d3/d5/da/db/l15 0 2024-09-16T08:57:11.365 INFO:tasks.workunit.client.1.smithi097.stdout:3/501: symlink d7/de/la9 0 2024-09-16T08:57:11.365 INFO:tasks.workunit.client.1.smithi097.stdout:7/463: creat d4/f9d x:0 0 0 2024-09-16T08:57:11.365 INFO:tasks.workunit.client.1.smithi097.stdout:7/464: readlink d4/d9/d51/l97 0 2024-09-16T08:57:11.371 INFO:tasks.workunit.client.0.smithi031.stdout:1/121: dwrite f0 [0,4194304] 0 2024-09-16T08:57:11.372 INFO:tasks.workunit.client.1.smithi097.stdout:0/513: symlink da/d24/d4c/d80/lb8 0 2024-09-16T08:57:11.372 INFO:tasks.workunit.client.0.smithi031.stdout:2/87: dread d6/f16 [0,4194304] 0 2024-09-16T08:57:11.373 INFO:tasks.workunit.client.0.smithi031.stdout:2/88: dread - d6/da/ff zero size 2024-09-16T08:57:11.375 INFO:tasks.workunit.client.0.smithi031.stdout:5/153: dwrite d2/dc/d25/f22 [0,4194304] 0 2024-09-16T08:57:11.375 INFO:tasks.workunit.client.0.smithi031.stdout:5/154: chown d2/fe 12 1 2024-09-16T08:57:11.375 INFO:tasks.workunit.client.0.smithi031.stdout:6/147: dwrite d0/dd/f22 [0,4194304] 0 2024-09-16T08:57:11.376 INFO:tasks.workunit.client.0.smithi031.stdout:6/148: chown d0/l16 22793606 1 2024-09-16T08:57:11.376 INFO:tasks.workunit.client.1.smithi097.stdout:3/502: write d7/de/d3c/f45 [981128,61424] 0 2024-09-16T08:57:11.376 INFO:tasks.workunit.client.1.smithi097.stdout:5/528: creat db/fa7 x:0 0 0 2024-09-16T08:57:11.377 INFO:tasks.workunit.client.1.smithi097.stdout:5/529: fdatasync db/f56 0 2024-09-16T08:57:11.383 INFO:tasks.workunit.client.1.smithi097.stdout:7/465: dread d4/f64 [0,4194304] 0 2024-09-16T08:57:11.383 INFO:tasks.workunit.client.1.smithi097.stdout:7/466: dread - d4/d7/d79/f82 zero size 2024-09-16T08:57:11.384 INFO:tasks.workunit.client.1.smithi097.stdout:1/538: symlink d5/d2c/d3b/d7f/laf 0 2024-09-16T08:57:11.386 INFO:tasks.workunit.client.0.smithi031.stdout:3/117: creat d3/df/d1a/f26 x:0 0 0 2024-09-16T08:57:11.387 INFO:tasks.workunit.client.1.smithi097.stdout:6/511: sync 2024-09-16T08:57:11.387 INFO:tasks.workunit.client.1.smithi097.stdout:0/514: write da/d24/d4c/d80/d3b/f91 [3726226,78392] 0 2024-09-16T08:57:11.387 INFO:tasks.workunit.client.1.smithi097.stdout:0/515: write da/d1a/d41/f50 [20849,113549] 0 2024-09-16T08:57:11.387 INFO:tasks.workunit.client.1.smithi097.stdout:0/516: dread - da/d1a/f81 zero size 2024-09-16T08:57:11.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/184: dread d2/fb [0,4194304] 0 2024-09-16T08:57:11.389 INFO:tasks.workunit.client.0.smithi031.stdout:7/185: dread - d2/d8/d11/f31 zero size 2024-09-16T08:57:11.389 INFO:tasks.workunit.client.0.smithi031.stdout:7/186: readlink l0 0 2024-09-16T08:57:11.389 INFO:tasks.workunit.client.0.smithi031.stdout:7/187: fdatasync d2/d8/f3f 0 2024-09-16T08:57:11.392 INFO:tasks.workunit.client.0.smithi031.stdout:5/155: read d2/dc/fd [2301133,2302] 0 2024-09-16T08:57:11.392 INFO:tasks.workunit.client.0.smithi031.stdout:9/90: creat d3/f1f x:0 0 0 2024-09-16T08:57:11.392 INFO:tasks.workunit.client.0.smithi031.stdout:9/91: chown d3/d5/da/f14 18616096 1 2024-09-16T08:57:11.393 INFO:tasks.workunit.client.1.smithi097.stdout:4/481: rename d0/d7/d52/c65 to d0/d7/d52/d26/d9a/cab 0 2024-09-16T08:57:11.395 INFO:tasks.workunit.client.0.smithi031.stdout:1/122: rename d2/de/c10 to d2/d4/d22/c26 0 2024-09-16T08:57:11.396 INFO:tasks.workunit.client.0.smithi031.stdout:1/123: chown d2/de/l20 1583115869 1 2024-09-16T08:57:11.396 INFO:tasks.workunit.client.0.smithi031.stdout:1/124: fdatasync d2/d4/f5 0 2024-09-16T08:57:11.396 INFO:tasks.workunit.client.0.smithi031.stdout:5/156: dread d2/dc/f20 [0,4194304] 0 2024-09-16T08:57:11.398 INFO:tasks.workunit.client.1.smithi097.stdout:3/503: truncate d7/de/d3c/f73 97873 0 2024-09-16T08:57:11.398 INFO:tasks.workunit.client.1.smithi097.stdout:3/504: read d7/de/d6c/f66 [4059160,97480] 0 2024-09-16T08:57:11.400 INFO:tasks.workunit.client.1.smithi097.stdout:7/467: dread d4/d12/f4f [0,4194304] 0 2024-09-16T08:57:11.401 INFO:tasks.workunit.client.1.smithi097.stdout:6/512: write d1/d3/d3e/d5a/f83 [492008,118160] 0 2024-09-16T08:57:11.402 INFO:tasks.workunit.client.1.smithi097.stdout:6/513: read d1/d3/d3e/d5a/f83 [289061,15880] 0 2024-09-16T08:57:11.403 INFO:tasks.workunit.client.0.smithi031.stdout:3/118: mknod d3/c27 0 2024-09-16T08:57:11.406 INFO:tasks.workunit.client.0.smithi031.stdout:7/188: creat d2/f42 x:0 0 0 2024-09-16T08:57:11.409 INFO:tasks.workunit.client.0.smithi031.stdout:7/189: creat d2/d2d/f43 x:0 0 0 2024-09-16T08:57:11.411 INFO:tasks.workunit.client.0.smithi031.stdout:9/92: mknod d3/d5/da/db/c20 0 2024-09-16T08:57:11.411 INFO:tasks.workunit.client.0.smithi031.stdout:9/93: fsync d3/d5/da/db/f18 0 2024-09-16T08:57:11.411 INFO:tasks.workunit.client.0.smithi031.stdout:9/94: stat d3/d5/da/db/f1e 0 2024-09-16T08:57:11.412 INFO:tasks.workunit.client.1.smithi097.stdout:8/490: rmdir d8/dd 39 2024-09-16T08:57:11.415 INFO:tasks.workunit.client.1.smithi097.stdout:3/505: mkdir d7/de/d29/d35/d3d/d57/d64/daa 0 2024-09-16T08:57:11.415 INFO:tasks.workunit.client.1.smithi097.stdout:3/506: write d7/de/d6c/f66 [6700577,72614] 0 2024-09-16T08:57:11.415 INFO:tasks.workunit.client.1.smithi097.stdout:3/507: dread - d7/de/d3c/f99 zero size 2024-09-16T08:57:11.423 INFO:tasks.workunit.client.0.smithi031.stdout:9/95: mkdir d3/d5/da/db/d21 0 2024-09-16T08:57:11.423 INFO:tasks.workunit.client.0.smithi031.stdout:1/125: unlink d2/d4/d22/c26 0 2024-09-16T08:57:11.423 INFO:tasks.workunit.client.1.smithi097.stdout:7/468: unlink d4/l42 0 2024-09-16T08:57:11.423 INFO:tasks.workunit.client.1.smithi097.stdout:6/514: rename d1/f64 to d1/dc/d1e/fa0 0 2024-09-16T08:57:11.424 INFO:tasks.workunit.client.1.smithi097.stdout:8/491: mknod d8/dd/d15/d2c/d4a/d4d/d62/c94 0 2024-09-16T08:57:11.425 INFO:tasks.workunit.client.0.smithi031.stdout:1/126: rmdir d2/d15 39 2024-09-16T08:57:11.426 INFO:tasks.workunit.client.1.smithi097.stdout:3/508: write d7/d28/f2f [1527962,81941] 0 2024-09-16T08:57:11.426 INFO:tasks.workunit.client.0.smithi031.stdout:7/190: dread d2/d8/f2c [0,4194304] 0 2024-09-16T08:57:11.427 INFO:tasks.workunit.client.0.smithi031.stdout:9/96: dread d3/d5/da/db/f1e [0,4194304] 0 2024-09-16T08:57:11.427 INFO:tasks.workunit.client.0.smithi031.stdout:9/97: creat d3/d5/f22 x:0 0 0 2024-09-16T08:57:11.427 INFO:tasks.workunit.client.0.smithi031.stdout:9/98: readlink d3/d5/da/l1c 0 2024-09-16T08:57:11.427 INFO:tasks.workunit.client.1.smithi097.stdout:6/515: unlink d1/d3/d81/l9b 0 2024-09-16T08:57:11.428 INFO:tasks.workunit.client.0.smithi031.stdout:1/127: symlink d2/d4/d22/l27 0 2024-09-16T08:57:11.430 INFO:tasks.workunit.client.1.smithi097.stdout:8/492: rename d8/dd/d15/d2c/d4a/d4d/d3d/f43 to d8/dd/d15/d16/f95 0 2024-09-16T08:57:11.433 INFO:tasks.workunit.client.0.smithi031.stdout:9/99: mknod d3/c23 0 2024-09-16T08:57:11.433 INFO:tasks.workunit.client.0.smithi031.stdout:6/149: dwrite d0/f4 [0,4194304] 0 2024-09-16T08:57:11.434 INFO:tasks.workunit.client.1.smithi097.stdout:3/509: mknod d7/de/d8d/d90/d92/cab 0 2024-09-16T08:57:11.434 INFO:tasks.workunit.client.0.smithi031.stdout:1/128: creat d2/d4/f28 x:0 0 0 2024-09-16T08:57:11.435 INFO:tasks.workunit.client.0.smithi031.stdout:1/129: chown f0 55904468 1 2024-09-16T08:57:11.435 INFO:tasks.workunit.client.0.smithi031.stdout:1/130: dread - d2/d4/f1c zero size 2024-09-16T08:57:11.435 INFO:tasks.workunit.client.0.smithi031.stdout:1/131: fdatasync d2/f17 0 2024-09-16T08:57:11.435 INFO:tasks.workunit.client.1.smithi097.stdout:0/517: fdatasync da/d1a/d41/f50 0 2024-09-16T08:57:11.435 INFO:tasks.workunit.client.1.smithi097.stdout:4/482: dwrite d0/d39/f6a [8388608,4194304] 0 2024-09-16T08:57:11.435 INFO:tasks.workunit.client.1.smithi097.stdout:4/483: read - d0/d36/d4c/f91 zero size 2024-09-16T08:57:11.437 INFO:tasks.workunit.client.1.smithi097.stdout:6/516: unlink d1/d3/d16/f82 0 2024-09-16T08:57:11.437 INFO:tasks.workunit.client.1.smithi097.stdout:6/517: fdatasync d1/d3/d16/d86/f8a 0 2024-09-16T08:57:11.437 INFO:tasks.workunit.client.0.smithi031.stdout:9/100: readlink d3/d5/da/db/l10 0 2024-09-16T08:57:11.437 INFO:tasks.workunit.client.0.smithi031.stdout:5/157: dwrite d2/dc/d1d/f2a [0,4194304] 0 2024-09-16T08:57:11.438 INFO:tasks.workunit.client.1.smithi097.stdout:5/530: dwrite db/d18/f62 [0,4194304] 0 2024-09-16T08:57:11.438 INFO:tasks.workunit.client.0.smithi031.stdout:2/89: dwrite d6/da/f17 [0,4194304] 0 2024-09-16T08:57:11.438 INFO:tasks.workunit.client.0.smithi031.stdout:7/191: dread d2/d8/d11/f24 [0,4194304] 0 2024-09-16T08:57:11.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/119: dwrite d3/fb [0,4194304] 0 2024-09-16T08:57:11.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/120: creat d3/df/d1a/f28 x:0 0 0 2024-09-16T08:57:11.439 INFO:tasks.workunit.client.0.smithi031.stdout:3/121: truncate d3/df/d1a/f28 713778 0 2024-09-16T08:57:11.439 INFO:tasks.workunit.client.0.smithi031.stdout:6/150: mknod d0/c25 0 2024-09-16T08:57:11.440 INFO:tasks.workunit.client.1.smithi097.stdout:3/510: mknod d7/d28/d97/cac 0 2024-09-16T08:57:11.441 INFO:tasks.workunit.client.1.smithi097.stdout:6/518: symlink d1/dc/d1e/d9e/la1 0 2024-09-16T08:57:11.442 INFO:tasks.workunit.client.0.smithi031.stdout:1/132: unlink d2/c1a 0 2024-09-16T08:57:11.446 INFO:tasks.workunit.client.0.smithi031.stdout:7/192: dread d2/fb [0,4194304] 0 2024-09-16T08:57:11.447 INFO:tasks.workunit.client.1.smithi097.stdout:5/531: link db/d22/c73 db/d18/d1c/d42/d46/d7e/d64/d7f/ca8 0 2024-09-16T08:57:11.448 INFO:tasks.workunit.client.1.smithi097.stdout:5/532: dread - db/d18/d1c/d42/d46/d7e/f67 zero size 2024-09-16T08:57:11.449 INFO:tasks.workunit.client.1.smithi097.stdout:3/511: creat d7/de/d29/d35/d3d/d57/d64/daa/fad x:0 0 0 2024-09-16T08:57:11.449 INFO:tasks.workunit.client.1.smithi097.stdout:3/512: write d7/de/d29/fa7 [554443,40823] 0 2024-09-16T08:57:11.449 INFO:tasks.workunit.client.1.smithi097.stdout:5/533: dread db/d18/d20/d2b/f7c [0,4194304] 0 2024-09-16T08:57:11.450 INFO:tasks.workunit.client.0.smithi031.stdout:5/158: getdents d2/dc/d25 0 2024-09-16T08:57:11.453 INFO:tasks.workunit.client.1.smithi097.stdout:6/519: creat d1/d3/d3e/d5a/fa2 x:0 0 0 2024-09-16T08:57:11.457 INFO:tasks.workunit.client.1.smithi097.stdout:5/534: symlink db/d18/d1c/d42/la9 0 2024-09-16T08:57:11.458 INFO:tasks.workunit.client.1.smithi097.stdout:6/520: symlink d1/d3/d16/d6a/la3 0 2024-09-16T08:57:11.458 INFO:tasks.workunit.client.1.smithi097.stdout:6/521: write d1/dc/d3c/f44 [1255353,44714] 0 2024-09-16T08:57:11.461 INFO:tasks.workunit.client.1.smithi097.stdout:5/535: creat db/d1a/faa x:0 0 0 2024-09-16T08:57:11.464 INFO:tasks.workunit.client.1.smithi097.stdout:6/522: creat d1/d3/d16/d4f/d9f/fa4 x:0 0 0 2024-09-16T08:57:11.467 INFO:tasks.workunit.client.0.smithi031.stdout:1/133: dread d2/f14 [0,4194304] 0 2024-09-16T08:57:11.468 INFO:tasks.workunit.client.1.smithi097.stdout:5/536: rename db/d18/d1c/d42/d46/c5e to db/d18/d20/d2b/cab 0 2024-09-16T08:57:11.469 INFO:tasks.workunit.client.0.smithi031.stdout:3/122: symlink d3/dc/l29 0 2024-09-16T08:57:11.469 INFO:tasks.workunit.client.0.smithi031.stdout:3/123: stat d3/df/d1a/d24/f25 0 2024-09-16T08:57:11.470 INFO:tasks.workunit.client.1.smithi097.stdout:6/523: readlink d1/d3/l14 0 2024-09-16T08:57:11.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/151: unlink d0/l1f 0 2024-09-16T08:57:11.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/152: getdents d0/dd 0 2024-09-16T08:57:11.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/153: chown d0/d10/d14/l1b 516176824 1 2024-09-16T08:57:11.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/154: creat d0/d17/d19/f26 x:0 0 0 2024-09-16T08:57:11.473 INFO:tasks.workunit.client.1.smithi097.stdout:5/537: mknod db/d18/d1c/d42/d46/d70/cac 0 2024-09-16T08:57:11.475 INFO:tasks.workunit.client.0.smithi031.stdout:9/101: getdents d3/d5/da/db 0 2024-09-16T08:57:11.475 INFO:tasks.workunit.client.0.smithi031.stdout:9/102: chown d3/d5/f17 59286610 1 2024-09-16T08:57:11.477 INFO:tasks.workunit.client.0.smithi031.stdout:1/134: rename d2/d15/f18 to d2/f29 0 2024-09-16T08:57:11.480 INFO:tasks.workunit.client.0.smithi031.stdout:9/103: rename d3/d5/da/l1c to d3/d5/da/db/d21/l24 0 2024-09-16T08:57:11.485 INFO:tasks.workunit.client.0.smithi031.stdout:9/104: truncate d3/d5/da/db/fe 2437563 0 2024-09-16T08:57:11.485 INFO:tasks.workunit.client.0.smithi031.stdout:9/105: write f2 [6050500,116628] 0 2024-09-16T08:57:11.485 INFO:tasks.workunit.client.0.smithi031.stdout:9/106: mknod d3/d5/da/db/c25 0 2024-09-16T08:57:11.485 INFO:tasks.workunit.client.0.smithi031.stdout:9/107: truncate d3/f9 2384322 0 2024-09-16T08:57:11.485 INFO:tasks.workunit.client.0.smithi031.stdout:9/108: rename d3 to d3/d5/da/db/d26 22 2024-09-16T08:57:11.486 INFO:tasks.workunit.client.1.smithi097.stdout:8/493: dwrite d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f84 [0,4194304] 0 2024-09-16T08:57:11.486 INFO:tasks.workunit.client.1.smithi097.stdout:8/494: chown d8/dd/d15/d2c/d48/d56/f6d 0 1 2024-09-16T08:57:11.487 INFO:tasks.workunit.client.1.smithi097.stdout:8/495: getdents d8/dd/d15/d2c/d4a 0 2024-09-16T08:57:11.487 INFO:tasks.workunit.client.1.smithi097.stdout:8/496: rename d8/dd/d15/d2c to d8/dd/d15/d2c/d4a/d4d/d3d/d96 22 2024-09-16T08:57:11.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/124: dread d3/f5 [0,4194304] 0 2024-09-16T08:57:11.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/125: truncate d3/df/d1a/f28 1146920 0 2024-09-16T08:57:11.488 INFO:tasks.workunit.client.1.smithi097.stdout:8/497: truncate d8/f10 2257282 0 2024-09-16T08:57:11.488 INFO:tasks.workunit.client.1.smithi097.stdout:8/498: fsync d8/dd/d42/f51 0 2024-09-16T08:57:11.490 INFO:tasks.workunit.client.1.smithi097.stdout:5/538: dread db/d1a/f57 [0,4194304] 0 2024-09-16T08:57:11.490 INFO:tasks.workunit.client.1.smithi097.stdout:8/499: rename d8/dd/d15/d16/d7c/d86 to d8/dd/d15/d2c/d4a/d4e/d97 0 2024-09-16T08:57:11.492 INFO:tasks.workunit.client.1.smithi097.stdout:8/500: symlink d8/dd/d15/d2c/d4a/d4d/d3d/d7b/l98 0 2024-09-16T08:57:11.492 INFO:tasks.workunit.client.1.smithi097.stdout:8/501: chown d8/dd/d15/d2c/d4a/d4d/d3d 6542671 1 2024-09-16T08:57:11.492 INFO:tasks.workunit.client.1.smithi097.stdout:5/539: dread db/d18/d1c/d42/d46/f47 [0,4194304] 0 2024-09-16T08:57:11.493 INFO:tasks.workunit.client.1.smithi097.stdout:5/540: creat db/d18/d1c/fad x:0 0 0 2024-09-16T08:57:11.493 INFO:tasks.workunit.client.1.smithi097.stdout:5/541: fdatasync db/f8b 0 2024-09-16T08:57:11.495 INFO:tasks.workunit.client.1.smithi097.stdout:5/542: symlink db/d1a/lae 0 2024-09-16T08:57:11.496 INFO:tasks.workunit.client.0.smithi031.stdout:1/135: write d2/de/f13 [3359891,46364] 0 2024-09-16T08:57:11.496 INFO:tasks.workunit.client.0.smithi031.stdout:3/126: dread d3/df/d1a/f28 [0,4194304] 0 2024-09-16T08:57:11.497 INFO:tasks.workunit.client.1.smithi097.stdout:5/543: getdents db/d22/d58 0 2024-09-16T08:57:11.500 INFO:tasks.workunit.client.1.smithi097.stdout:5/544: unlink db/d22/c3c 0 2024-09-16T08:57:11.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/127: rmdir d3/d14/d20 0 2024-09-16T08:57:11.502 INFO:tasks.workunit.client.0.smithi031.stdout:9/109: write d3/d5/da/db/fe [116311,109058] 0 2024-09-16T08:57:11.504 INFO:tasks.workunit.client.1.smithi097.stdout:5/545: rename db/d18/d1c/d42/d46/d70 to db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf 0 2024-09-16T08:57:11.506 INFO:tasks.workunit.client.1.smithi097.stdout:5/546: mknod db/d18/d1c/cb0 0 2024-09-16T08:57:11.506 INFO:tasks.workunit.client.1.smithi097.stdout:5/547: readlink db/d18/d1c/l94 0 2024-09-16T08:57:11.510 INFO:tasks.workunit.client.1.smithi097.stdout:4/484: dwrite d0/d23/f9f [0,4194304] 0 2024-09-16T08:57:11.510 INFO:tasks.workunit.client.1.smithi097.stdout:4/485: chown d0/d7/d52/d26 270432575 1 2024-09-16T08:57:11.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/159: dwrite d2/dc/f1f [0,4194304] 0 2024-09-16T08:57:11.511 INFO:tasks.workunit.client.0.smithi031.stdout:7/193: dwrite d2/d8/f2f [0,4194304] 0 2024-09-16T08:57:11.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:11 smithi097 ceph-mon[35478]: mgrmap e25: smithi097.mpashf(active, since 7s), standbys: smithi031.dcmqpv 2024-09-16T08:57:11.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:11 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:57:11.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:11 smithi097 ceph-mon[35478]: pgmap v7: 65 pgs: 65 active+clean; 1.0 GiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 142 MiB/s rd, 193 MiB/s wr, 417 op/s 2024-09-16T08:57:11.512 INFO:tasks.workunit.client.1.smithi097.stdout:0/518: dwrite da/d1a/f3a [0,4194304] 0 2024-09-16T08:57:11.512 INFO:tasks.workunit.client.1.smithi097.stdout:3/513: dwrite d7/f2b [4194304,4194304] 0 2024-09-16T08:57:11.512 INFO:tasks.workunit.client.1.smithi097.stdout:6/524: dwrite d1/d3/d16/d86/f8b [0,4194304] 0 2024-09-16T08:57:11.516 INFO:tasks.workunit.client.0.smithi031.stdout:6/155: dwrite d0/d17/d19/f1d [0,4194304] 0 2024-09-16T08:57:11.527 INFO:tasks.workunit.client.0.smithi031.stdout:6/156: symlink d0/d17/d19/l27 0 2024-09-16T08:57:11.529 INFO:tasks.workunit.client.0.smithi031.stdout:3/128: dwrite d3/df/f10 [4194304,4194304] 0 2024-09-16T08:57:11.531 INFO:tasks.workunit.client.1.smithi097.stdout:0/519: rmdir da/d24/d76 39 2024-09-16T08:57:11.533 INFO:tasks.workunit.client.1.smithi097.stdout:3/514: mknod d7/de/d8d/d90/d96/cae 0 2024-09-16T08:57:11.534 INFO:tasks.workunit.client.0.smithi031.stdout:7/194: rmdir d2 39 2024-09-16T08:57:11.534 INFO:tasks.workunit.client.0.smithi031.stdout:7/195: truncate d2/d8/f3f 750988 0 2024-09-16T08:57:11.534 INFO:tasks.workunit.client.0.smithi031.stdout:5/160: dwrite d2/fb [0,4194304] 0 2024-09-16T08:57:11.534 INFO:tasks.workunit.client.0.smithi031.stdout:5/161: creat d2/dc/f2d x:0 0 0 2024-09-16T08:57:11.534 INFO:tasks.workunit.client.0.smithi031.stdout:5/162: read - d2/f2b zero size 2024-09-16T08:57:11.535 INFO:tasks.workunit.client.1.smithi097.stdout:5/548: dwrite db/d18/d20/d2b/f6f [0,4194304] 0 2024-09-16T08:57:11.536 INFO:tasks.workunit.client.0.smithi031.stdout:3/129: creat d3/f2a x:0 0 0 2024-09-16T08:57:11.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/130: dread d3/df/d1a/f28 [0,4194304] 0 2024-09-16T08:57:11.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/131: readlink d3/l15 0 2024-09-16T08:57:11.538 INFO:tasks.workunit.client.1.smithi097.stdout:6/525: symlink d1/d3/d16/d86/la5 0 2024-09-16T08:57:11.538 INFO:tasks.workunit.client.1.smithi097.stdout:6/526: readlink d1/dc/l9d 0 2024-09-16T08:57:11.539 INFO:tasks.workunit.client.0.smithi031.stdout:7/196: creat d2/d8/d11/f44 x:0 0 0 2024-09-16T08:57:11.539 INFO:tasks.workunit.client.0.smithi031.stdout:7/197: chown d2/d2d/f43 36888914 1 2024-09-16T08:57:11.539 INFO:tasks.workunit.client.0.smithi031.stdout:7/198: creat d2/d8/d11/f45 x:0 0 0 2024-09-16T08:57:11.539 INFO:tasks.workunit.client.1.smithi097.stdout:4/486: rmdir d0/d7/df/d15 39 2024-09-16T08:57:11.541 INFO:tasks.workunit.client.1.smithi097.stdout:3/515: mkdir d7/de/d9c/daf 0 2024-09-16T08:57:11.541 INFO:tasks.workunit.client.1.smithi097.stdout:3/516: rename d7/de/d29/d35/d3d/d74 to d7/de/d29/d35/d3d/d74/db0 22 2024-09-16T08:57:11.545 INFO:tasks.workunit.client.0.smithi031.stdout:5/163: mkdir d2/dc/d25/d21/d2e 0 2024-09-16T08:57:11.545 INFO:tasks.workunit.client.1.smithi097.stdout:0/520: dread da/d1a/d26/d47/db3/f7b [0,4194304] 0 2024-09-16T08:57:11.546 INFO:tasks.workunit.client.1.smithi097.stdout:5/549: unlink db/d18/d20/d2b/f75 0 2024-09-16T08:57:11.546 INFO:tasks.workunit.client.1.smithi097.stdout:5/550: truncate db/d1a/f5b 1056820 0 2024-09-16T08:57:11.546 INFO:tasks.workunit.client.1.smithi097.stdout:5/551: write db/d18/d1c/d42/d46/d7e/f67 [713487,45725] 0 2024-09-16T08:57:11.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/132: creat d3/df/d1a/d1d/f2b x:0 0 0 2024-09-16T08:57:11.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/133: write d3/dc/f16 [2990652,90081] 0 2024-09-16T08:57:11.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/134: write d3/df/d1a/d24/f25 [9054141,50214] 0 2024-09-16T08:57:11.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/135: chown d3/df/d1a/d1d 16 1 2024-09-16T08:57:11.548 INFO:tasks.workunit.client.1.smithi097.stdout:4/487: truncate d0/d36/f47 119371 0 2024-09-16T08:57:11.548 INFO:tasks.workunit.client.1.smithi097.stdout:4/488: chown d0/d7/d52/f50 4117004 1 2024-09-16T08:57:11.548 INFO:tasks.workunit.client.0.smithi031.stdout:6/157: dwrite d0/dd/f22 [0,4194304] 0 2024-09-16T08:57:11.560 INFO:tasks.workunit.client.1.smithi097.stdout:0/521: dwrite da/d1a/f22 [0,4194304] 0 2024-09-16T08:57:11.570 INFO:tasks.workunit.client.1.smithi097.stdout:5/552: rename db/d22/l43 to db/d6e/d81/lb1 0 2024-09-16T08:57:11.571 INFO:tasks.workunit.client.0.smithi031.stdout:7/199: unlink d2/d8/f30 0 2024-09-16T08:57:11.572 INFO:tasks.workunit.client.0.smithi031.stdout:7/200: chown d2/d8/l25 54 1 2024-09-16T08:57:11.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/164: creat d2/dc/d25/f2f x:0 0 0 2024-09-16T08:57:11.575 INFO:tasks.workunit.client.0.smithi031.stdout:3/136: mknod d3/dc/c2c 0 2024-09-16T08:57:11.577 INFO:tasks.workunit.client.1.smithi097.stdout:0/522: dwrite da/d1a/f1c [0,4194304] 0 2024-09-16T08:57:11.578 INFO:tasks.workunit.client.0.smithi031.stdout:5/165: dread d2/f5 [0,4194304] 0 2024-09-16T08:57:11.578 INFO:tasks.workunit.client.0.smithi031.stdout:3/137: dread d3/dc/f13 [0,4194304] 0 2024-09-16T08:57:11.578 INFO:tasks.workunit.client.0.smithi031.stdout:3/138: chown f1 9714835 1 2024-09-16T08:57:11.580 INFO:tasks.workunit.client.1.smithi097.stdout:4/489: rename d0/d7/df/d15/f41 to d0/d36/d4c/fac 0 2024-09-16T08:57:11.580 INFO:tasks.workunit.client.1.smithi097.stdout:4/490: readlink d0/d7/df/l22 0 2024-09-16T08:57:11.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/158: creat d0/dd/f28 x:0 0 0 2024-09-16T08:57:11.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/159: write d0/d17/d19/f1c [618971,57527] 0 2024-09-16T08:57:11.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/160: write d0/d17/f1e [527178,43670] 0 2024-09-16T08:57:11.583 INFO:tasks.workunit.client.1.smithi097.stdout:3/517: dwrite d7/de/d3c/f45 [0,4194304] 0 2024-09-16T08:57:11.584 INFO:tasks.workunit.client.0.smithi031.stdout:7/201: symlink d2/d8/d11/l46 0 2024-09-16T08:57:11.585 INFO:tasks.workunit.client.0.smithi031.stdout:3/139: truncate d3/f5 1831022 0 2024-09-16T08:57:11.588 INFO:tasks.workunit.client.0.smithi031.stdout:6/161: link d0/d17/f18 d0/d17/d19/f29 0 2024-09-16T08:57:11.590 INFO:tasks.workunit.client.0.smithi031.stdout:3/140: dread d3/dc/f13 [0,4194304] 0 2024-09-16T08:57:11.590 INFO:tasks.workunit.client.0.smithi031.stdout:3/141: write d3/df/f10 [861823,43723] 0 2024-09-16T08:57:11.592 INFO:tasks.workunit.client.0.smithi031.stdout:7/202: mknod d2/c47 0 2024-09-16T08:57:11.599 INFO:tasks.workunit.client.1.smithi097.stdout:5/553: mknod db/d18/cb2 0 2024-09-16T08:57:11.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/166: dwrite d2/f5 [4194304,4194304] 0 2024-09-16T08:57:11.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/167: read - d2/f1a zero size 2024-09-16T08:57:11.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/168: rename d2/dc to d2/dc/d25/d21/d30 22 2024-09-16T08:57:11.605 INFO:tasks.workunit.client.0.smithi031.stdout:5/169: rename d2/l9 to d2/dc/d25/d21/d2e/l31 0 2024-09-16T08:57:11.605 INFO:tasks.workunit.client.0.smithi031.stdout:5/170: stat d2/f1a 0 2024-09-16T08:57:11.606 INFO:tasks.workunit.client.1.smithi097.stdout:3/518: write d7/de/d3c/d3f/f6d [2282626,78412] 0 2024-09-16T08:57:11.606 INFO:tasks.workunit.client.1.smithi097.stdout:3/519: stat d7/de/d6c/f5a 0 2024-09-16T08:57:11.606 INFO:tasks.workunit.client.1.smithi097.stdout:3/520: fdatasync d7/f2b 0 2024-09-16T08:57:11.606 INFO:tasks.workunit.client.1.smithi097.stdout:3/521: fsync d7/d28/f2f 0 2024-09-16T08:57:11.606 INFO:tasks.workunit.client.1.smithi097.stdout:3/522: chown d7/d28/d63/fa6 24261654 1 2024-09-16T08:57:11.607 INFO:tasks.workunit.client.0.smithi031.stdout:5/171: symlink d2/dc/l32 0 2024-09-16T08:57:11.611 INFO:tasks.workunit.client.1.smithi097.stdout:3/523: creat d7/de/d8d/d90/d92/fb1 x:0 0 0 2024-09-16T08:57:11.613 INFO:tasks.workunit.client.0.smithi031.stdout:5/172: truncate d2/f1c 84020 0 2024-09-16T08:57:11.615 INFO:tasks.workunit.client.0.smithi031.stdout:7/203: dread d2/d8/fd [0,4194304] 0 2024-09-16T08:57:11.615 INFO:tasks.workunit.client.1.smithi097.stdout:3/524: unlink d7/de/d29/f49 0 2024-09-16T08:57:11.616 INFO:tasks.workunit.client.0.smithi031.stdout:6/162: truncate d0/d17/d19/f1c 458658 0 2024-09-16T08:57:11.616 INFO:tasks.workunit.client.0.smithi031.stdout:6/163: chown d0/d17/d19/f29 90453568 1 2024-09-16T08:57:11.616 INFO:tasks.workunit.client.0.smithi031.stdout:6/164: chown d0/d10/d14/l1b 15 1 2024-09-16T08:57:11.617 INFO:tasks.workunit.client.0.smithi031.stdout:6/165: fdatasync d0/d17/d19/f26 0 2024-09-16T08:57:11.619 INFO:tasks.workunit.client.1.smithi097.stdout:3/525: mknod d7/d28/d30/d88/cb2 0 2024-09-16T08:57:11.620 INFO:tasks.workunit.client.1.smithi097.stdout:3/526: mkdir d7/d28/d30/d88/db3 0 2024-09-16T08:57:11.622 INFO:tasks.workunit.client.0.smithi031.stdout:7/204: getdents d2/d8/d11 0 2024-09-16T08:57:11.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:11 smithi031 ceph-mon[27844]: mgrmap e25: smithi097.mpashf(active, since 7s), standbys: smithi031.dcmqpv 2024-09-16T08:57:11.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:11 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:57:11.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:11 smithi031 ceph-mon[27844]: pgmap v7: 65 pgs: 65 active+clean; 1.0 GiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 142 MiB/s rd, 193 MiB/s wr, 417 op/s 2024-09-16T08:57:11.623 INFO:tasks.workunit.client.0.smithi031.stdout:7/205: creat d2/f48 x:0 0 0 2024-09-16T08:57:11.623 INFO:tasks.workunit.client.0.smithi031.stdout:7/206: fsync d2/f42 0 2024-09-16T08:57:11.624 INFO:tasks.workunit.client.0.smithi031.stdout:7/207: dread - d2/d8/d11/f44 zero size 2024-09-16T08:57:11.624 INFO:tasks.workunit.client.0.smithi031.stdout:7/208: read d2/f22 [288008,6752] 0 2024-09-16T08:57:11.626 INFO:tasks.workunit.client.1.smithi097.stdout:0/523: dwrite da/d1a/d26/f59 [0,4194304] 0 2024-09-16T08:57:11.629 INFO:tasks.workunit.client.1.smithi097.stdout:4/491: dwrite d0/d39/fa3 [0,4194304] 0 2024-09-16T08:57:11.630 INFO:tasks.workunit.client.1.smithi097.stdout:5/554: dwrite db/d18/d1c/d42/d46/d7e/f53 [0,4194304] 0 2024-09-16T08:57:11.630 INFO:tasks.workunit.client.1.smithi097.stdout:5/555: chown db/d18/d1c/d42/d46/d7e/d64/d8d 117284977 1 2024-09-16T08:57:11.630 INFO:tasks.workunit.client.0.smithi031.stdout:3/142: dwrite d3/df/d1a/d1d/f2b [0,4194304] 0 2024-09-16T08:57:11.631 INFO:tasks.workunit.client.0.smithi031.stdout:3/143: stat d3/c17 0 2024-09-16T08:57:11.631 INFO:tasks.workunit.client.1.smithi097.stdout:3/527: dread d7/de/d6c/f37 [0,4194304] 0 2024-09-16T08:57:11.631 INFO:tasks.workunit.client.1.smithi097.stdout:3/528: read d7/d28/d30/d69/f6a [3103699,83982] 0 2024-09-16T08:57:11.632 INFO:tasks.workunit.client.1.smithi097.stdout:4/492: truncate d0/d7/df/d15/d1f/f81 1207891 0 2024-09-16T08:57:11.632 INFO:tasks.workunit.client.1.smithi097.stdout:4/493: write d0/d23/f7b [175069,1485] 0 2024-09-16T08:57:11.637 INFO:tasks.workunit.client.1.smithi097.stdout:5/556: rename db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/fa4 to db/d18/d1c/d42/d46/d7e/d64/fb3 0 2024-09-16T08:57:11.639 INFO:tasks.workunit.client.1.smithi097.stdout:3/529: creat d7/de/d29/d35/d3d/d57/fb4 x:0 0 0 2024-09-16T08:57:11.640 INFO:tasks.workunit.client.1.smithi097.stdout:3/530: dread d7/de/d3c/d3f/d60/d6b/f86 [0,4194304] 0 2024-09-16T08:57:11.642 INFO:tasks.workunit.client.1.smithi097.stdout:9/429: sync 2024-09-16T08:57:11.642 INFO:tasks.workunit.client.1.smithi097.stdout:1/539: sync 2024-09-16T08:57:11.642 INFO:tasks.workunit.client.1.smithi097.stdout:9/430: readlink d2/d5/l20 0 2024-09-16T08:57:11.642 INFO:tasks.workunit.client.1.smithi097.stdout:1/540: write d5/d17/f90 [1037121,130231] 0 2024-09-16T08:57:11.642 INFO:tasks.workunit.client.1.smithi097.stdout:8/502: sync 2024-09-16T08:57:11.643 INFO:tasks.workunit.client.1.smithi097.stdout:7/469: sync 2024-09-16T08:57:11.643 INFO:tasks.workunit.client.1.smithi097.stdout:7/470: creat d4/d7/d77/f9e x:0 0 0 2024-09-16T08:57:11.643 INFO:tasks.workunit.client.1.smithi097.stdout:2/478: sync 2024-09-16T08:57:11.643 INFO:tasks.workunit.client.1.smithi097.stdout:2/479: write d3/d4/d8/f16 [6413769,64041] 0 2024-09-16T08:57:11.643 INFO:tasks.workunit.client.0.smithi031.stdout:3/144: rename d3/dc/f16 to d3/df/d1a/d1d/f2d 0 2024-09-16T08:57:11.644 INFO:tasks.workunit.client.1.smithi097.stdout:5/557: creat db/d18/d1c/d42/d46/d7e/d8f/fb4 x:0 0 0 2024-09-16T08:57:11.644 INFO:tasks.workunit.client.1.smithi097.stdout:8/503: dread d8/dd/d15/d16/f95 [0,4194304] 0 2024-09-16T08:57:11.645 INFO:tasks.workunit.client.1.smithi097.stdout:8/504: chown d8/dd/d15/d2c/d4a/d4d/d3f/f77 31711898 1 2024-09-16T08:57:11.646 INFO:tasks.workunit.client.1.smithi097.stdout:3/531: getdents d7/d28/d63 0 2024-09-16T08:57:11.646 INFO:tasks.workunit.client.0.smithi031.stdout:7/209: dwrite d2/d2d/f2e [0,4194304] 0 2024-09-16T08:57:11.647 INFO:tasks.workunit.client.0.smithi031.stdout:5/173: rmdir d2 39 2024-09-16T08:57:11.647 INFO:tasks.workunit.client.0.smithi031.stdout:3/145: unlink d3/df/d1a/d1d/d1e/c1f 0 2024-09-16T08:57:11.649 INFO:tasks.workunit.client.1.smithi097.stdout:1/541: symlink d5/d70/lb0 0 2024-09-16T08:57:11.649 INFO:tasks.workunit.client.1.smithi097.stdout:1/542: write d5/d14/d95/fa0 [4948998,124760] 0 2024-09-16T08:57:11.650 INFO:tasks.workunit.client.0.smithi031.stdout:6/166: read d0/d17/d19/f1c [242149,95117] 0 2024-09-16T08:57:11.656 INFO:tasks.workunit.client.1.smithi097.stdout:7/471: creat d4/d12/d1f/f9f x:0 0 0 2024-09-16T08:57:11.658 INFO:tasks.workunit.client.0.smithi031.stdout:7/210: rename d2/d8/l41 to d2/d8/d11/l49 0 2024-09-16T08:57:11.660 INFO:tasks.workunit.client.1.smithi097.stdout:2/480: rename d3/l7b to d3/d4/d9/l8a 0 2024-09-16T08:57:11.660 INFO:tasks.workunit.client.1.smithi097.stdout:2/481: stat d3/d1b/f83 0 2024-09-16T08:57:11.661 INFO:tasks.workunit.client.0.smithi031.stdout:5/174: rmdir d2/dc/d25/d21 39 2024-09-16T08:57:11.661 INFO:tasks.workunit.client.0.smithi031.stdout:5/175: chown d2/dc/d25/f2f 2337778 1 2024-09-16T08:57:11.661 INFO:tasks.workunit.client.0.smithi031.stdout:5/176: fdatasync d2/dc/f27 0 2024-09-16T08:57:11.661 INFO:tasks.workunit.client.1.smithi097.stdout:0/524: dwrite da/d1a/d26/d47/db3/d60/d68/d5e/f72 [0,4194304] 0 2024-09-16T08:57:11.662 INFO:tasks.workunit.client.1.smithi097.stdout:8/505: rmdir d8/dd/d15/d2c/d4a/d4e 39 2024-09-16T08:57:11.665 INFO:tasks.workunit.client.1.smithi097.stdout:6/527: sync 2024-09-16T08:57:11.666 INFO:tasks.workunit.client.0.smithi031.stdout:6/167: unlink d0/d17/d19/f1c 0 2024-09-16T08:57:11.668 INFO:tasks.workunit.client.1.smithi097.stdout:1/543: mknod d5/d17/d54/d6b/da8/cb1 0 2024-09-16T08:57:11.669 INFO:tasks.workunit.client.0.smithi031.stdout:7/211: creat d2/d8/f4a x:0 0 0 2024-09-16T08:57:11.669 INFO:tasks.workunit.client.0.smithi031.stdout:7/212: fdatasync d2/ff 0 2024-09-16T08:57:11.670 INFO:tasks.workunit.client.0.smithi031.stdout:5/177: read d2/fe [529101,64699] 0 2024-09-16T08:57:11.671 INFO:tasks.workunit.client.0.smithi031.stdout:3/146: getdents d3 0 2024-09-16T08:57:11.672 INFO:tasks.workunit.client.0.smithi031.stdout:7/213: creat d2/d2d/f4b x:0 0 0 2024-09-16T08:57:11.672 INFO:tasks.workunit.client.0.smithi031.stdout:7/214: dread - d2/d8/d11/f40 zero size 2024-09-16T08:57:11.674 INFO:tasks.workunit.client.1.smithi097.stdout:8/506: mkdir d8/dd/d15/d2c/d4a/d4d/d3f/d99 0 2024-09-16T08:57:11.674 INFO:tasks.workunit.client.0.smithi031.stdout:7/215: stat d2/l28 0 2024-09-16T08:57:11.676 INFO:tasks.workunit.client.1.smithi097.stdout:6/528: symlink d1/d3/d16/d86/la6 0 2024-09-16T08:57:11.677 INFO:tasks.workunit.client.0.smithi031.stdout:7/216: mknod d2/d8/d11/c4c 0 2024-09-16T08:57:11.677 INFO:tasks.workunit.client.0.smithi031.stdout:7/217: write d2/d8/d11/f1b [1176381,130487] 0 2024-09-16T08:57:11.677 INFO:tasks.workunit.client.1.smithi097.stdout:1/544: mknod d5/d2c/d3b/cb2 0 2024-09-16T08:57:11.677 INFO:tasks.workunit.client.1.smithi097.stdout:1/545: dread - d5/db/f59 zero size 2024-09-16T08:57:11.679 INFO:tasks.workunit.client.1.smithi097.stdout:2/482: rename d3/d4/d77/d48 to d3/d4/d68/d8b 0 2024-09-16T08:57:11.679 INFO:tasks.workunit.client.1.smithi097.stdout:2/483: write d3/d4/d68/d8b/d7c/d7d/f1f [2982925,17256] 0 2024-09-16T08:57:11.679 INFO:tasks.workunit.client.1.smithi097.stdout:2/484: fdatasync d3/d4/d9/f5a 0 2024-09-16T08:57:11.680 INFO:tasks.workunit.client.1.smithi097.stdout:8/507: symlink d8/dd/d15/d2c/d4a/d4d/d3f/d99/l9a 0 2024-09-16T08:57:11.681 INFO:tasks.workunit.client.1.smithi097.stdout:8/508: readlink d8/dd/d15/d2c/d48/l69 0 2024-09-16T08:57:11.681 INFO:tasks.workunit.client.1.smithi097.stdout:8/509: write d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f8b [871654,116316] 0 2024-09-16T08:57:11.681 INFO:tasks.workunit.client.1.smithi097.stdout:8/510: write d8/dd/d42/f89 [915600,25918] 0 2024-09-16T08:57:11.681 INFO:tasks.workunit.client.1.smithi097.stdout:8/511: truncate d8/dd/f7e 700125 0 2024-09-16T08:57:11.681 INFO:tasks.workunit.client.1.smithi097.stdout:8/512: write d8/dd/d15/d2c/d4a/d4d/f35 [4749306,80412] 0 2024-09-16T08:57:11.681 INFO:tasks.workunit.client.1.smithi097.stdout:8/513: creat d8/dd/d15/f9b x:0 0 0 2024-09-16T08:57:11.682 INFO:tasks.workunit.client.1.smithi097.stdout:8/514: stat d8/dd/d15/d2c/d4a/d4d/c39 0 2024-09-16T08:57:11.682 INFO:tasks.workunit.client.1.smithi097.stdout:8/515: fdatasync d8/f1e 0 2024-09-16T08:57:11.682 INFO:tasks.workunit.client.1.smithi097.stdout:8/516: mkdir d8/dd/d71/d9c 0 2024-09-16T08:57:11.682 INFO:tasks.workunit.client.1.smithi097.stdout:8/517: truncate d8/dd/d15/d16/f3c 4914193 0 2024-09-16T08:57:11.682 INFO:tasks.workunit.client.1.smithi097.stdout:8/518: getdents d8/dd/d71/d9c 0 2024-09-16T08:57:11.683 INFO:tasks.workunit.client.1.smithi097.stdout:8/519: creat d8/dd/d15/d2c/d4a/d4e/d85/f9d x:0 0 0 2024-09-16T08:57:11.688 INFO:tasks.workunit.client.0.smithi031.stdout:5/178: dread d2/fa [0,4194304] 0 2024-09-16T08:57:11.689 INFO:tasks.workunit.client.0.smithi031.stdout:5/179: creat d2/dc/d25/d21/d2e/f33 x:0 0 0 2024-09-16T08:57:11.690 INFO:tasks.workunit.client.0.smithi031.stdout:5/180: write d2/dc/f16 [1831186,119530] 0 2024-09-16T08:57:11.690 INFO:tasks.workunit.client.0.smithi031.stdout:5/181: fsync d2/dc/f20 0 2024-09-16T08:57:11.690 INFO:tasks.workunit.client.1.smithi097.stdout:1/546: write d5/db/f74 [1333246,67692] 0 2024-09-16T08:57:11.690 INFO:tasks.workunit.client.1.smithi097.stdout:1/547: readlink d5/d70/lb0 0 2024-09-16T08:57:11.691 INFO:tasks.workunit.client.0.smithi031.stdout:5/182: rename d2/fe to d2/f34 0 2024-09-16T08:57:11.692 INFO:tasks.workunit.client.0.smithi031.stdout:5/183: dread d2/f1c [0,4194304] 0 2024-09-16T08:57:11.692 INFO:tasks.workunit.client.0.smithi031.stdout:5/184: symlink d2/dc/d25/d21/d2e/l35 0 2024-09-16T08:57:11.693 INFO:tasks.workunit.client.0.smithi031.stdout:5/185: mkdir d2/dc/d36 0 2024-09-16T08:57:11.694 INFO:tasks.workunit.client.1.smithi097.stdout:0/525: dread da/d1a/d26/d47/db3/d60/d68/d5e/f72 [0,4194304] 0 2024-09-16T08:57:11.694 INFO:tasks.workunit.client.1.smithi097.stdout:4/494: dwrite d0/d7/d52/f8d [0,4194304] 0 2024-09-16T08:57:11.694 INFO:tasks.workunit.client.1.smithi097.stdout:4/495: chown d0/d35/d6c/c95 1 1 2024-09-16T08:57:11.697 INFO:tasks.workunit.client.1.smithi097.stdout:0/526: creat da/d1a/d9c/fb9 x:0 0 0 2024-09-16T08:57:11.699 INFO:tasks.workunit.client.1.smithi097.stdout:9/431: dwrite d2/d3f/d62/f7f [0,4194304] 0 2024-09-16T08:57:11.699 INFO:tasks.workunit.client.1.smithi097.stdout:9/432: write d2/f13 [2832999,21928] 0 2024-09-16T08:57:11.700 INFO:tasks.workunit.client.1.smithi097.stdout:9/433: write d2/f28 [4056942,81948] 0 2024-09-16T08:57:11.700 INFO:tasks.workunit.client.1.smithi097.stdout:1/548: dread d5/db/f3c [0,4194304] 0 2024-09-16T08:57:11.700 INFO:tasks.workunit.client.1.smithi097.stdout:1/549: truncate d5/db/f52 220882 0 2024-09-16T08:57:11.700 INFO:tasks.workunit.client.1.smithi097.stdout:2/485: dread d3/d4/d9/f36 [0,4194304] 0 2024-09-16T08:57:11.701 INFO:tasks.workunit.client.1.smithi097.stdout:2/486: read - d3/d1b/f83 zero size 2024-09-16T08:57:11.701 INFO:tasks.workunit.client.1.smithi097.stdout:2/487: fsync d3/d4/d68/d8b/d7c/d7d/f81 0 2024-09-16T08:57:11.701 INFO:tasks.workunit.client.1.smithi097.stdout:2/488: chown d3/d1b/l3d 0 1 2024-09-16T08:57:11.701 INFO:tasks.workunit.client.1.smithi097.stdout:0/527: creat da/d1a/d26/d47/db3/d60/d68/fba x:0 0 0 2024-09-16T08:57:11.702 INFO:tasks.workunit.client.1.smithi097.stdout:0/528: dread da/f1b [0,4194304] 0 2024-09-16T08:57:11.702 INFO:tasks.workunit.client.1.smithi097.stdout:0/529: chown da/d24/d4c 9695371 1 2024-09-16T08:57:11.703 INFO:tasks.workunit.client.1.smithi097.stdout:6/529: read d1/d3/f7 [1032676,63815] 0 2024-09-16T08:57:11.703 INFO:tasks.workunit.client.1.smithi097.stdout:6/530: write d1/d3/d81/f93 [341592,114199] 0 2024-09-16T08:57:11.706 INFO:tasks.workunit.client.1.smithi097.stdout:9/434: creat d2/d5/d56/f88 x:0 0 0 2024-09-16T08:57:11.706 INFO:tasks.workunit.client.1.smithi097.stdout:9/435: fsync d2/d3f/f55 0 2024-09-16T08:57:11.706 INFO:tasks.workunit.client.1.smithi097.stdout:7/472: dwrite d4/d12/d1f/f21 [0,4194304] 0 2024-09-16T08:57:11.709 INFO:tasks.workunit.client.1.smithi097.stdout:0/530: symlink da/d1a/d26/d47/d97/lbb 0 2024-09-16T08:57:11.714 INFO:tasks.workunit.client.1.smithi097.stdout:6/531: creat d1/d3/fa7 x:0 0 0 2024-09-16T08:57:11.714 INFO:tasks.workunit.client.1.smithi097.stdout:5/558: dwrite db/d18/d1c/d42/d46/fa6 [0,4194304] 0 2024-09-16T08:57:11.714 INFO:tasks.workunit.client.1.smithi097.stdout:2/489: dread d3/d4/d68/d8b/d7c/d7d/f5e [0,4194304] 0 2024-09-16T08:57:11.714 INFO:tasks.workunit.client.1.smithi097.stdout:2/490: truncate d3/d4/d68/d8b/d7c/d7d/f6b 616343 0 2024-09-16T08:57:11.715 INFO:tasks.workunit.client.1.smithi097.stdout:1/550: rmdir d5/db 39 2024-09-16T08:57:11.715 INFO:tasks.workunit.client.1.smithi097.stdout:0/531: creat da/d1a/d26/d47/d97/d84/fbc x:0 0 0 2024-09-16T08:57:11.717 INFO:tasks.workunit.client.0.smithi031.stdout:6/168: dwrite d0/dd/f28 [0,4194304] 0 2024-09-16T08:57:11.719 INFO:tasks.workunit.client.1.smithi097.stdout:3/532: dwrite d7/de/d29/d35/d3d/d57/f91 [0,4194304] 0 2024-09-16T08:57:11.719 INFO:tasks.workunit.client.1.smithi097.stdout:3/533: fdatasync d7/de/d8d/d90/d92/fb1 0 2024-09-16T08:57:11.719 INFO:tasks.workunit.client.1.smithi097.stdout:3/534: stat d7/de/d8d/d90/d92/cab 0 2024-09-16T08:57:11.720 INFO:tasks.workunit.client.1.smithi097.stdout:1/551: symlink d5/db/lb3 0 2024-09-16T08:57:11.724 INFO:tasks.workunit.client.1.smithi097.stdout:3/535: unlink d7/de/d3c/d4f/l9e 0 2024-09-16T08:57:11.729 INFO:tasks.workunit.client.0.smithi031.stdout:3/147: dwrite f1 [0,4194304] 0 2024-09-16T08:57:11.729 INFO:tasks.workunit.client.0.smithi031.stdout:3/148: truncate d3/df/d1a/d1d/f2d 9498931 0 2024-09-16T08:57:11.729 INFO:tasks.workunit.client.1.smithi097.stdout:3/536: mkdir d7/de/d8d/d90/d96/d9f/db5 0 2024-09-16T08:57:11.730 INFO:tasks.workunit.client.0.smithi031.stdout:6/169: dread d0/d17/d19/f29 [0,4194304] 0 2024-09-16T08:57:11.733 INFO:tasks.workunit.client.0.smithi031.stdout:6/170: rename d0/la to d0/dd/l2a 0 2024-09-16T08:57:11.734 INFO:tasks.workunit.client.1.smithi097.stdout:7/473: dread d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:11.739 INFO:tasks.workunit.client.0.smithi031.stdout:6/171: mkdir d0/dd/d2b 0 2024-09-16T08:57:11.740 INFO:tasks.workunit.client.0.smithi031.stdout:5/186: dwrite d2/f5 [4194304,4194304] 0 2024-09-16T08:57:11.741 INFO:tasks.workunit.client.0.smithi031.stdout:7/218: link d2/d8/d11/l49 d2/d8/d11/l4d 0 2024-09-16T08:57:11.793 INFO:tasks.workunit.client.0.smithi031.stdout:7/219: rename d2/d8/d11/l49 to d2/d8/l4e 0 2024-09-16T08:57:11.793 INFO:tasks.workunit.client.0.smithi031.stdout:7/220: stat d2/d8/d11/f40 0 2024-09-16T08:57:11.793 INFO:tasks.workunit.client.0.smithi031.stdout:7/221: symlink d2/d8/l4f 0 2024-09-16T08:57:11.794 INFO:tasks.workunit.client.0.smithi031.stdout:7/222: creat d2/d8/f50 x:0 0 0 2024-09-16T08:57:11.794 INFO:tasks.workunit.client.0.smithi031.stdout:7/223: chown d2/d2d/c36 64102 1 2024-09-16T08:57:11.796 INFO:tasks.workunit.client.1.smithi097.stdout:1/552: dread d5/db/dc/d19/f4d [0,4194304] 0 2024-09-16T08:57:11.796 INFO:tasks.workunit.client.1.smithi097.stdout:1/553: creat d5/d2c/d3b/fb4 x:0 0 0 2024-09-16T08:57:11.796 INFO:tasks.workunit.client.1.smithi097.stdout:1/554: fdatasync d5/f12 0 2024-09-16T08:57:11.796 INFO:tasks.workunit.client.1.smithi097.stdout:1/555: chown d5/d17/d54 2316 1 2024-09-16T08:57:11.796 INFO:tasks.workunit.client.1.smithi097.stdout:1/556: chown d5/d17/d54/d6b/da8 102695502 1 2024-09-16T08:57:11.796 INFO:tasks.workunit.client.1.smithi097.stdout:9/436: dwrite d2/f50 [0,4194304] 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/437: write d2/d5/f35 [1514384,68913] 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/438: chown d2/f28 46 1 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:4/496: dwrite d0/d23/f9f [0,4194304] 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:4/497: chown d0/d36/d4c/c8a 102367 1 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:7/474: chown d4/d7/ca 1034486 1 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:7/475: stat d4/d9/d51/d8b 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/439: creat d2/d5/d72/f89 x:0 0 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/440: stat d2/d5/d3a/d3c/f6d 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/441: readlink d2/d3f/d62/l68 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/442: read d2/fe [740080,38703] 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/443: fdatasync d2/d5/f11 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/444: write d2/d5/f35 [1644345,83245] 0 2024-09-16T08:57:11.797 INFO:tasks.workunit.client.1.smithi097.stdout:9/445: stat d2/d3f/f41 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/446: read d2/fe [4032859,91272] 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:4/498: fsync d0/d7/d52/f88 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:7/476: link d4/d7/d77/f9e d4/d7/d19/fa0 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/447: unlink d2/f1a 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/448: creat d2/d5/d56/f8a x:0 0 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/449: write d2/d5/d3a/d3c/f85 [316117,27468] 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:7/477: mkdir d4/d7/d7b/da1 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:7/478: dread - d4/d12/d1f/d22/f8c zero size 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:7/479: fdatasync d4/d7/d7b/f40 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:7/480: truncate d4/d7/d19/fa0 1047741 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/450: mknod d2/d30/d84/c8b 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/451: mkdir d2/d30/d74/d8c 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:9/452: fsync d2/d5/d3a/d3c/d73/f7b 0 2024-09-16T08:57:11.798 INFO:tasks.workunit.client.1.smithi097.stdout:0/532: fsync da/d1a/d26/d47/d97/d84/fbc 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:1/557: dread d5/db/dc/f78 [0,4194304] 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/491: dwrite d3/f23 [0,4194304] 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:3/537: dwrite d7/f21 [0,4194304] 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:3/538: creat d7/de/d29/d35/d3d/d57/fb6 x:0 0 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:3/539: read d7/de/f1e [515624,102632] 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:3/540: creat d7/de/d29/d35/d3d/fb7 x:0 0 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/492: rename d3/f88 to d3/d4/d68/d8b/d7c/d7d/f8c 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/493: write d3/d4/d68/d8b/d7c/d7d/f6d [494430,31135] 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/494: truncate d3/d4/d68/d8b/d7c/d7d/f6b 1063637 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/495: rmdir d3/d4/d68/d8b/d7c/d7d/d53 39 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/496: creat d3/d4/d68/d8b/d7c/d7d/f8d x:0 0 0 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/497: read - d3/d4/d77/f70 zero size 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/498: chown d3/d4/d9/f36 1150983299 1 2024-09-16T08:57:11.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/499: dread - d3/d4/d8/d51/f59 zero size 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/500: creat d3/d4/d8/f8e x:0 0 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/501: write d3/d4/d8/f7a [821972,119981] 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:4/499: write d0/d7/df/f37 [2140889,97843] 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:4/500: readlink d0/d7/df/d15/l17 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/502: symlink d3/d4/d9/d78/l8f 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:9/453: dread d2/d30/f3b [0,4194304] 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/503: symlink d3/d4/l90 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/504: mknod d3/d4/d8/d51/c91 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/505: creat d3/d4/d9/d67/f92 x:0 0 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/506: symlink d3/d4/d9/l93 0 2024-09-16T08:57:11.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/507: creat d3/d4/d9/d78/f94 x:0 0 0 2024-09-16T08:57:11.801 INFO:tasks.workunit.client.1.smithi097.stdout:9/454: write d2/f38 [3774067,112744] 0 2024-09-16T08:57:11.802 INFO:tasks.workunit.client.0.smithi031.stdout:3/149: dwrite d3/fa [4194304,4194304] 0 2024-09-16T08:57:11.802 INFO:tasks.workunit.client.0.smithi031.stdout:3/150: truncate d3/dc/f13 1288167 0 2024-09-16T08:57:11.802 INFO:tasks.workunit.client.0.smithi031.stdout:3/151: creat d3/d14/f2e x:0 0 0 2024-09-16T08:57:11.802 INFO:tasks.workunit.client.0.smithi031.stdout:3/152: creat d3/d14/f2f x:0 0 0 2024-09-16T08:57:11.802 INFO:tasks.workunit.client.0.smithi031.stdout:3/153: symlink d3/l30 0 2024-09-16T08:57:11.802 INFO:tasks.workunit.client.1.smithi097.stdout:3/541: dread d7/de/d29/d35/f27 [4194304,4194304] 0 2024-09-16T08:57:11.802 INFO:tasks.workunit.client.1.smithi097.stdout:3/542: mkdir d7/de/d3c/d3f/d60/d6b/db8 0 2024-09-16T08:57:11.804 INFO:tasks.workunit.client.1.smithi097.stdout:3/543: unlink d7/de/d29/d35/d3d/l41 0 2024-09-16T08:57:11.804 INFO:tasks.workunit.client.0.smithi031.stdout:3/154: getdents d3/df/d1a/d1d 0 2024-09-16T08:57:11.804 INFO:tasks.workunit.client.0.smithi031.stdout:3/155: readlink d3/l19 0 2024-09-16T08:57:11.805 INFO:tasks.workunit.client.0.smithi031.stdout:3/156: mknod d3/df/d1a/c31 0 2024-09-16T08:57:11.814 INFO:tasks.workunit.client.0.smithi031.stdout:6/172: dwrite d0/d17/d19/f1d [0,4194304] 0 2024-09-16T08:57:11.815 INFO:tasks.workunit.client.1.smithi097.stdout:5/559: dwrite db/d6e/d95/fa3 [0,4194304] 0 2024-09-16T08:57:11.815 INFO:tasks.workunit.client.1.smithi097.stdout:5/560: write db/d22/f96 [611648,94974] 0 2024-09-16T08:57:11.815 INFO:tasks.workunit.client.1.smithi097.stdout:5/561: fdatasync db/d1a/f3a 0 2024-09-16T08:57:11.815 INFO:tasks.workunit.client.1.smithi097.stdout:6/532: dwrite d1/dc/f72 [0,4194304] 0 2024-09-16T08:57:11.815 INFO:tasks.workunit.client.0.smithi031.stdout:6/173: mknod d0/d17/c2c 0 2024-09-16T08:57:11.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/187: dwrite d2/dc/d25/f22 [0,4194304] 0 2024-09-16T08:57:11.818 INFO:tasks.workunit.client.0.smithi031.stdout:7/224: dwrite d2/f42 [0,4194304] 0 2024-09-16T08:57:11.818 INFO:tasks.workunit.client.0.smithi031.stdout:7/225: write d2/d8/d11/f40 [391238,129545] 0 2024-09-16T08:57:11.820 INFO:tasks.workunit.client.0.smithi031.stdout:6/174: rename d0/fc to d0/d17/f2d 0 2024-09-16T08:57:11.821 INFO:tasks.workunit.client.1.smithi097.stdout:7/481: dwrite d4/d1b/d41/f52 [4194304,4194304] 0 2024-09-16T08:57:11.822 INFO:tasks.workunit.client.1.smithi097.stdout:9/455: dread d2/d5/d3a/d3c/f85 [0,4194304] 0 2024-09-16T08:57:11.822 INFO:tasks.workunit.client.1.smithi097.stdout:9/456: dread - d2/d5/f69 zero size 2024-09-16T08:57:11.822 INFO:tasks.workunit.client.1.smithi097.stdout:4/501: dwrite d0/d7/d52/f50 [4194304,4194304] 0 2024-09-16T08:57:11.822 INFO:tasks.workunit.client.1.smithi097.stdout:4/502: chown d0/d36/f47 748646 1 2024-09-16T08:57:11.823 INFO:tasks.workunit.client.1.smithi097.stdout:1/558: dwrite d5/db/dc/d19/f38 [0,4194304] 0 2024-09-16T08:57:11.826 INFO:tasks.workunit.client.1.smithi097.stdout:6/533: rename d1/dc/d1e/l91 to d1/dc/la8 0 2024-09-16T08:57:11.838 INFO:tasks.workunit.client.1.smithi097.stdout:7/482: rmdir d4/d9/d2b 39 2024-09-16T08:57:11.838 INFO:tasks.workunit.client.1.smithi097.stdout:7/483: creat d4/d9/dc/d8e/fa2 x:0 0 0 2024-09-16T08:57:11.838 INFO:tasks.workunit.client.1.smithi097.stdout:3/544: dwrite d7/de/d3c/f73 [0,4194304] 0 2024-09-16T08:57:11.839 INFO:tasks.workunit.client.1.smithi097.stdout:9/457: truncate d2/d5/f71 525530 0 2024-09-16T08:57:11.840 INFO:tasks.workunit.client.0.smithi031.stdout:3/157: dwrite d3/f5 [0,4194304] 0 2024-09-16T08:57:11.846 INFO:tasks.workunit.client.1.smithi097.stdout:7/484: mknod d4/d9/ca3 0 2024-09-16T08:57:11.847 INFO:tasks.workunit.client.1.smithi097.stdout:7/485: chown d4/d1b 12563 1 2024-09-16T08:57:11.850 INFO:tasks.workunit.client.1.smithi097.stdout:3/545: rename d7/de/d3c/d3f/d60/d6b/l95 to d7/d28/d30/d88/db3/lb9 0 2024-09-16T08:57:11.851 INFO:tasks.workunit.client.1.smithi097.stdout:3/546: write d7/de/d8d/d90/f9d [174070,70588] 0 2024-09-16T08:57:11.851 INFO:tasks.workunit.client.1.smithi097.stdout:3/547: chown d7/de/d29/d35/d3d 78956058 1 2024-09-16T08:57:11.851 INFO:tasks.workunit.client.1.smithi097.stdout:3/548: write d7/de/d29/d35/d3d/d57/fb4 [996634,42149] 0 2024-09-16T08:57:11.852 INFO:tasks.workunit.client.1.smithi097.stdout:9/458: mkdir d2/d5/d8d 0 2024-09-16T08:57:11.852 INFO:tasks.workunit.client.1.smithi097.stdout:9/459: readlink d2/d5/l12 0 2024-09-16T08:57:11.852 INFO:tasks.workunit.client.1.smithi097.stdout:9/460: stat d2/d5/f71 0 2024-09-16T08:57:11.853 INFO:tasks.workunit.client.1.smithi097.stdout:4/503: rmdir d0/d23/d5f 39 2024-09-16T08:57:11.854 INFO:tasks.workunit.client.1.smithi097.stdout:1/559: rmdir d5/d2c/d3b 39 2024-09-16T08:57:11.856 INFO:tasks.workunit.client.1.smithi097.stdout:9/461: mknod d2/d5/d6e/c8e 0 2024-09-16T08:57:11.857 INFO:tasks.workunit.client.1.smithi097.stdout:4/504: symlink d0/d7/df/lad 0 2024-09-16T08:57:11.858 INFO:tasks.workunit.client.1.smithi097.stdout:4/505: creat d0/d7/df/d83/fae x:0 0 0 2024-09-16T08:57:11.860 INFO:tasks.workunit.client.1.smithi097.stdout:1/560: unlink d5/db/f7c 0 2024-09-16T08:57:11.863 INFO:tasks.workunit.client.1.smithi097.stdout:6/534: dwrite d1/d3/d3e/f62 [0,4194304] 0 2024-09-16T08:57:11.864 INFO:tasks.workunit.client.0.smithi031.stdout:7/226: dwrite d2/f1a [0,4194304] 0 2024-09-16T08:57:11.865 INFO:tasks.workunit.client.0.smithi031.stdout:7/227: stat d2/c47 0 2024-09-16T08:57:11.865 INFO:tasks.workunit.client.1.smithi097.stdout:5/562: dwrite db/d18/d1c/d42/d46/fa6 [0,4194304] 0 2024-09-16T08:57:11.867 INFO:tasks.workunit.client.1.smithi097.stdout:4/506: link d0/d7/d52/f1d d0/d39/d71/faf 0 2024-09-16T08:57:11.867 INFO:tasks.workunit.client.0.smithi031.stdout:5/188: dwrite d2/f5 [4194304,4194304] 0 2024-09-16T08:57:11.870 INFO:tasks.workunit.client.1.smithi097.stdout:9/462: dread d2/d5/d3a/d3c/f54 [0,4194304] 0 2024-09-16T08:57:11.872 INFO:tasks.workunit.client.1.smithi097.stdout:4/507: mkdir d0/d7/d52/d26/d8c/db0 0 2024-09-16T08:57:11.872 INFO:tasks.workunit.client.1.smithi097.stdout:4/508: chown d0/d23/c43 1438 1 2024-09-16T08:57:11.873 INFO:tasks.workunit.client.1.smithi097.stdout:4/509: write d0/d39/f3c [3151867,34023] 0 2024-09-16T08:57:11.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/228: dread d2/d8/d11/f1b [0,4194304] 0 2024-09-16T08:57:11.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/229: stat d2/c47 0 2024-09-16T08:57:11.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/230: readlink d2/l3d 0 2024-09-16T08:57:11.874 INFO:tasks.workunit.client.1.smithi097.stdout:5/563: getdents db/d22 0 2024-09-16T08:57:11.877 INFO:tasks.workunit.client.1.smithi097.stdout:9/463: symlink d2/d5/l8f 0 2024-09-16T08:57:11.877 INFO:tasks.workunit.client.1.smithi097.stdout:9/464: chown d2/f38 113 1 2024-09-16T08:57:11.878 INFO:tasks.workunit.client.1.smithi097.stdout:3/549: dwrite d7/d28/f2f [0,4194304] 0 2024-09-16T08:57:11.878 INFO:tasks.workunit.client.1.smithi097.stdout:3/550: write d7/d28/d30/f94 [994249,92662] 0 2024-09-16T08:57:11.879 INFO:tasks.workunit.client.0.smithi031.stdout:3/158: write d3/dc/f13 [1217444,28680] 0 2024-09-16T08:57:11.879 INFO:tasks.workunit.client.0.smithi031.stdout:3/159: read - d3/f2a zero size 2024-09-16T08:57:11.879 INFO:tasks.workunit.client.0.smithi031.stdout:7/231: write d2/fb [819941,41441] 0 2024-09-16T08:57:11.882 INFO:tasks.workunit.client.1.smithi097.stdout:5/564: creat db/d18/d1c/d42/d46/d7e/fb5 x:0 0 0 2024-09-16T08:57:11.883 INFO:tasks.workunit.client.1.smithi097.stdout:3/551: unlink d7/de/d8d/d90/fa2 0 2024-09-16T08:57:11.885 INFO:tasks.workunit.client.0.smithi031.stdout:3/160: unlink d3/df/d1a/d1d/d1e/l21 0 2024-09-16T08:57:11.885 INFO:tasks.workunit.client.0.smithi031.stdout:3/161: chown d3/f4 122891571 1 2024-09-16T08:57:11.885 INFO:tasks.workunit.client.0.smithi031.stdout:3/162: readlink d3/le 0 2024-09-16T08:57:11.885 INFO:tasks.workunit.client.0.smithi031.stdout:3/163: chown d3/dc/cd 0 1 2024-09-16T08:57:11.886 INFO:tasks.workunit.client.1.smithi097.stdout:3/552: link d7/d28/d30/f94 d7/d28/d30/d88/fba 0 2024-09-16T08:57:11.887 INFO:tasks.workunit.client.0.smithi031.stdout:3/164: symlink d3/df/d1a/d24/l32 0 2024-09-16T08:57:11.889 INFO:tasks.workunit.client.0.smithi031.stdout:7/232: mknod d2/d8/c51 0 2024-09-16T08:57:11.889 INFO:tasks.workunit.client.0.smithi031.stdout:7/233: write d2/d8/f4a [124063,22003] 0 2024-09-16T08:57:11.890 INFO:tasks.workunit.client.1.smithi097.stdout:3/553: rmdir d7/de/d29/d35/d3d/d74 0 2024-09-16T08:57:11.891 INFO:tasks.workunit.client.1.smithi097.stdout:1/561: dwrite d5/d17/f5b [0,4194304] 0 2024-09-16T08:57:11.892 INFO:tasks.workunit.client.0.smithi031.stdout:7/234: rename d2/d8/d11/l1c to d2/d8/d11/l52 0 2024-09-16T08:57:11.896 INFO:tasks.workunit.client.0.smithi031.stdout:7/235: symlink d2/d8/l53 0 2024-09-16T08:57:11.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/236: write d2/d8/f50 [532012,14681] 0 2024-09-16T08:57:11.907 INFO:tasks.workunit.client.1.smithi097.stdout:4/510: getdents d0/d7/d52/d26/d8c 0 2024-09-16T08:57:11.907 INFO:tasks.workunit.client.1.smithi097.stdout:4/511: symlink d0/d23/d5f/lb1 0 2024-09-16T08:57:11.912 INFO:tasks.workunit.client.1.smithi097.stdout:1/562: dread d5/db/dc/d19/f38 [0,4194304] 0 2024-09-16T08:57:11.914 INFO:tasks.workunit.client.1.smithi097.stdout:8/520: sync 2024-09-16T08:57:11.920 INFO:tasks.workunit.client.1.smithi097.stdout:9/465: dwrite d2/d5/d3a/d3c/f5f [0,4194304] 0 2024-09-16T08:57:11.920 INFO:tasks.workunit.client.1.smithi097.stdout:9/466: write d2/d5/f35 [2182235,48112] 0 2024-09-16T08:57:11.921 INFO:tasks.workunit.client.1.smithi097.stdout:9/467: symlink d2/d5/d56/l90 0 2024-09-16T08:57:11.922 INFO:tasks.workunit.client.1.smithi097.stdout:4/512: read d0/d39/fa3 [2542828,17957] 0 2024-09-16T08:57:11.922 INFO:tasks.workunit.client.1.smithi097.stdout:3/554: write d7/de/d3c/d4f/f80 [1639233,123027] 0 2024-09-16T08:57:11.922 INFO:tasks.workunit.client.1.smithi097.stdout:9/468: creat d2/d3f/d62/d79/f91 x:0 0 0 2024-09-16T08:57:11.923 INFO:tasks.workunit.client.1.smithi097.stdout:9/469: chown d2/d3f/d62 1903400 1 2024-09-16T08:57:11.923 INFO:tasks.workunit.client.1.smithi097.stdout:4/513: creat d0/d7/d52/d26/d8c/fb2 x:0 0 0 2024-09-16T08:57:11.923 INFO:tasks.workunit.client.1.smithi097.stdout:3/555: mkdir d7/de/d8d/d90/d96/d9f/dbb 0 2024-09-16T08:57:11.924 INFO:tasks.workunit.client.1.smithi097.stdout:3/556: truncate d7/de/f1e 1125157 0 2024-09-16T08:57:11.924 INFO:tasks.workunit.client.1.smithi097.stdout:9/470: creat d2/d5/d3a/d70/f92 x:0 0 0 2024-09-16T08:57:11.925 INFO:tasks.workunit.client.1.smithi097.stdout:4/514: mknod d0/d7/d52/d26/cb3 0 2024-09-16T08:57:11.925 INFO:tasks.workunit.client.1.smithi097.stdout:4/515: stat d0/d7/d52/d26/d3d/f44 0 2024-09-16T08:57:11.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/557: symlink d7/de/d29/d35/d3d/d57/d64/daa/lbc 0 2024-09-16T08:57:11.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/558: truncate d7/de/d3c/f4d 953042 0 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/516: dread d0/d35/f5c [4194304,4194304] 0 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/517: chown d0/d23/d5f/lb1 833258 1 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:9/471: creat d2/d5/d8d/f93 x:0 0 0 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:9/472: write d2/f13 [3601616,111320] 0 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/518: mknod d0/d7/df/d15/d1f/cb4 0 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/519: creat d0/d7/df/d83/fb5 x:0 0 0 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/520: fsync d0/d36/d4c/f80 0 2024-09-16T08:57:11.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/521: readlink d0/d39/d71/l5a 0 2024-09-16T08:57:11.928 INFO:tasks.workunit.client.1.smithi097.stdout:9/473: rename d2/d30/d84 to d2/d30/d74/d8c/d94 0 2024-09-16T08:57:11.928 INFO:tasks.workunit.client.1.smithi097.stdout:9/474: chown d2/c58 238 1 2024-09-16T08:57:11.928 INFO:tasks.workunit.client.1.smithi097.stdout:9/475: truncate d2/d5/d3a/d3c/d73/f7b 184504 0 2024-09-16T08:57:11.929 INFO:tasks.workunit.client.1.smithi097.stdout:4/522: unlink d0/d36/c3f 0 2024-09-16T08:57:11.929 INFO:tasks.workunit.client.1.smithi097.stdout:9/476: mknod d2/d5/d8d/c95 0 2024-09-16T08:57:11.930 INFO:tasks.workunit.client.1.smithi097.stdout:4/523: symlink d0/d7/d52/d26/d8c/lb6 0 2024-09-16T08:57:11.931 INFO:tasks.workunit.client.1.smithi097.stdout:9/477: unlink d2/d3f/l5c 0 2024-09-16T08:57:11.932 INFO:tasks.workunit.client.1.smithi097.stdout:9/478: write d2/d5/d3a/d3c/f85 [156187,44653] 0 2024-09-16T08:57:11.933 INFO:tasks.workunit.client.1.smithi097.stdout:9/479: mkdir d2/d30/d74/d8c/d94/d96 0 2024-09-16T08:57:11.933 INFO:tasks.workunit.client.1.smithi097.stdout:9/480: rename d2/d5/d8d to d2/d5/d8d/d97 22 2024-09-16T08:57:11.934 INFO:tasks.workunit.client.1.smithi097.stdout:9/481: unlink d2/c7 0 2024-09-16T08:57:11.934 INFO:tasks.workunit.client.1.smithi097.stdout:9/482: chown d2/d5/d56/c78 7122 1 2024-09-16T08:57:11.939 INFO:tasks.workunit.client.1.smithi097.stdout:5/565: dwrite db/d18/d1c/d42/d46/d7e/f80 [0,4194304] 0 2024-09-16T08:57:11.940 INFO:tasks.workunit.client.1.smithi097.stdout:5/566: readlink db/d18/d1c/d42/d46/d7e/d76/l7a 0 2024-09-16T08:57:11.940 INFO:tasks.workunit.client.1.smithi097.stdout:6/535: dwrite d1/dc/d1e/f68 [0,4194304] 0 2024-09-16T08:57:11.940 INFO:tasks.workunit.client.1.smithi097.stdout:5/567: stat db/d18/d20/d2b/c55 0 2024-09-16T08:57:11.942 INFO:tasks.workunit.client.1.smithi097.stdout:4/524: dread d0/f64 [0,4194304] 0 2024-09-16T08:57:11.942 INFO:tasks.workunit.client.1.smithi097.stdout:6/536: mkdir d1/d3/d81/da9 0 2024-09-16T08:57:11.943 INFO:tasks.workunit.client.1.smithi097.stdout:5/568: rmdir db/d18/d1c/d42/d46/d7e/d64 39 2024-09-16T08:57:11.943 INFO:tasks.workunit.client.1.smithi097.stdout:1/563: dwrite d5/db/dc/f78 [0,4194304] 0 2024-09-16T08:57:11.943 INFO:tasks.workunit.client.1.smithi097.stdout:1/564: write d5/d70/f72 [944872,11264] 0 2024-09-16T08:57:11.943 INFO:tasks.workunit.client.1.smithi097.stdout:1/565: fdatasync d5/d2c/f4e 0 2024-09-16T08:57:11.944 INFO:tasks.workunit.client.0.smithi031.stdout:3/165: dwrite d3/df/d1a/f26 [0,4194304] 0 2024-09-16T08:57:11.944 INFO:tasks.workunit.client.0.smithi031.stdout:7/237: dwrite d2/d8/d11/f40 [0,4194304] 0 2024-09-16T08:57:11.944 INFO:tasks.workunit.client.1.smithi097.stdout:4/525: mkdir d0/d7/df/d15/db7 0 2024-09-16T08:57:11.944 INFO:tasks.workunit.client.1.smithi097.stdout:4/526: readlink d0/d7/d52/l92 0 2024-09-16T08:57:11.945 INFO:tasks.workunit.client.1.smithi097.stdout:4/527: truncate d0/d36/d4c/f91 98749 0 2024-09-16T08:57:11.945 INFO:tasks.workunit.client.1.smithi097.stdout:3/559: dwrite d7/de/d3c/f45 [0,4194304] 0 2024-09-16T08:57:11.945 INFO:tasks.workunit.client.1.smithi097.stdout:3/560: readlink d7/d15/l8f 0 2024-09-16T08:57:11.945 INFO:tasks.workunit.client.0.smithi031.stdout:7/238: dread d2/f35 [0,4194304] 0 2024-09-16T08:57:11.945 INFO:tasks.workunit.client.1.smithi097.stdout:6/537: symlink d1/dc/laa 0 2024-09-16T08:57:11.949 INFO:tasks.workunit.client.1.smithi097.stdout:1/566: mkdir d5/d70/d85/db5 0 2024-09-16T08:57:11.951 INFO:tasks.workunit.client.0.smithi031.stdout:3/166: rename d3/dc to d3/df/d33 0 2024-09-16T08:57:11.951 INFO:tasks.workunit.client.0.smithi031.stdout:3/167: stat d3/df/d33/l29 0 2024-09-16T08:57:11.953 INFO:tasks.workunit.client.1.smithi097.stdout:4/528: rename d0/d7/d52/f1d to d0/d39/fb8 0 2024-09-16T08:57:11.956 INFO:tasks.workunit.client.1.smithi097.stdout:3/561: getdents d7/d15 0 2024-09-16T08:57:11.957 INFO:tasks.workunit.client.1.smithi097.stdout:5/569: dwrite db/d1a/f38 [0,4194304] 0 2024-09-16T08:57:11.960 INFO:tasks.workunit.client.1.smithi097.stdout:4/529: symlink d0/d7/d52/d26/d3d/lb9 0 2024-09-16T08:57:11.960 INFO:tasks.workunit.client.1.smithi097.stdout:4/530: fsync d0/d7/d52/d26/d3d/f76 0 2024-09-16T08:57:11.963 INFO:tasks.workunit.client.1.smithi097.stdout:3/562: mknod d7/d28/d30/d69/cbd 0 2024-09-16T08:57:11.964 INFO:tasks.workunit.client.0.smithi031.stdout:7/239: symlink d2/d8/l54 0 2024-09-16T08:57:11.964 INFO:tasks.workunit.client.0.smithi031.stdout:7/240: chown d2/d8/l4f 2329038 1 2024-09-16T08:57:11.965 INFO:tasks.workunit.client.1.smithi097.stdout:5/570: creat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fb6 x:0 0 0 2024-09-16T08:57:11.966 INFO:tasks.workunit.client.0.smithi031.stdout:3/168: rmdir d3/df/d1a 39 2024-09-16T08:57:11.968 INFO:tasks.workunit.client.0.smithi031.stdout:7/241: creat d2/d8/d11/f55 x:0 0 0 2024-09-16T08:57:11.971 INFO:tasks.workunit.client.0.smithi031.stdout:7/242: rename d2/ff to d2/f56 0 2024-09-16T08:57:11.974 INFO:tasks.workunit.client.1.smithi097.stdout:1/567: dwrite d5/db/f74 [0,4194304] 0 2024-09-16T08:57:11.977 INFO:tasks.workunit.client.0.smithi031.stdout:7/243: rename d2/d8/d11/f1b to d2/d8/f57 0 2024-09-16T08:57:11.977 INFO:tasks.workunit.client.0.smithi031.stdout:7/244: fsync d2/f48 0 2024-09-16T08:57:11.979 INFO:tasks.workunit.client.1.smithi097.stdout:5/571: read db/d6e/d95/fa3 [3849433,16636] 0 2024-09-16T08:57:11.980 INFO:tasks.workunit.client.1.smithi097.stdout:3/563: mkdir d7/de/d3c/dbe 0 2024-09-16T08:57:11.981 INFO:tasks.workunit.client.1.smithi097.stdout:5/572: write db/d18/d1c/d42/d46/f47 [319600,84318] 0 2024-09-16T08:57:11.982 INFO:tasks.workunit.client.0.smithi031.stdout:7/245: rmdir d2/d2d 39 2024-09-16T08:57:11.985 INFO:tasks.workunit.client.1.smithi097.stdout:9/483: truncate d2/d5/d3a/d3c/f5f 2055206 0 2024-09-16T08:57:11.987 INFO:tasks.workunit.client.1.smithi097.stdout:1/568: write d5/d2c/d3b/f9a [4094773,17253] 0 2024-09-16T08:57:11.987 INFO:tasks.workunit.client.0.smithi031.stdout:3/169: dwrite d3/df/d1a/d1d/f2d [0,4194304] 0 2024-09-16T08:57:11.990 INFO:tasks.workunit.client.1.smithi097.stdout:6/538: rmdir d1/d3 39 2024-09-16T08:57:11.998 INFO:tasks.workunit.client.1.smithi097.stdout:5/573: creat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/fb7 x:0 0 0 2024-09-16T08:57:11.998 INFO:tasks.workunit.client.1.smithi097.stdout:5/574: creat db/d18/d1c/d42/d46/d7e/fb8 x:0 0 0 2024-09-16T08:57:11.999 INFO:tasks.workunit.client.1.smithi097.stdout:1/569: link d5/d70/d9d/cab d5/d14/cb6 0 2024-09-16T08:57:11.999 INFO:tasks.workunit.client.1.smithi097.stdout:1/570: chown d5/d70/f72 250472 1 2024-09-16T08:57:12.001 INFO:tasks.workunit.client.1.smithi097.stdout:6/539: rename d1/dc/d3c/c41 to d1/d3/d3e/d5a/cab 0 2024-09-16T08:57:12.002 INFO:tasks.workunit.client.1.smithi097.stdout:5/575: symlink db/d1a/lb9 0 2024-09-16T08:57:12.004 INFO:tasks.workunit.client.1.smithi097.stdout:1/571: creat d5/d2c/d3b/fb7 x:0 0 0 2024-09-16T08:57:12.004 INFO:tasks.workunit.client.1.smithi097.stdout:1/572: readlink d5/d17/d54/d6b/d71/laa 0 2024-09-16T08:57:12.004 INFO:tasks.workunit.client.1.smithi097.stdout:1/573: truncate d5/d2c/d3b/fb4 767386 0 2024-09-16T08:57:12.005 INFO:tasks.workunit.client.1.smithi097.stdout:6/540: truncate d1/dc/f72 1956545 0 2024-09-16T08:57:12.005 INFO:tasks.workunit.client.1.smithi097.stdout:6/541: truncate d1/d3/d16/f2e 1043387 0 2024-09-16T08:57:12.005 INFO:tasks.workunit.client.1.smithi097.stdout:3/564: dwrite d7/de/d3c/d4f/f7f [0,4194304] 0 2024-09-16T08:57:12.005 INFO:tasks.workunit.client.1.smithi097.stdout:3/565: write d7/de/d29/d35/d3d/f59 [1003697,93767] 0 2024-09-16T08:57:12.006 INFO:tasks.workunit.client.1.smithi097.stdout:5/576: mkdir db/d6e/d79/dba 0 2024-09-16T08:57:12.006 INFO:tasks.workunit.client.1.smithi097.stdout:5/577: getdents db/d22/d58 0 2024-09-16T08:57:12.006 INFO:tasks.workunit.client.1.smithi097.stdout:5/578: dread - db/f84 zero size 2024-09-16T08:57:12.007 INFO:tasks.workunit.client.1.smithi097.stdout:1/574: rename d5/db/f59 to d5/d14/d95/fb8 0 2024-09-16T08:57:12.007 INFO:tasks.workunit.client.1.smithi097.stdout:1/575: creat d5/fb9 x:0 0 0 2024-09-16T08:57:12.007 INFO:tasks.workunit.client.1.smithi097.stdout:6/542: rmdir d1/dc/d1e/d9e 39 2024-09-16T08:57:12.009 INFO:tasks.workunit.client.1.smithi097.stdout:9/484: dwrite d2/d5/fb [0,4194304] 0 2024-09-16T08:57:12.009 INFO:tasks.workunit.client.0.smithi031.stdout:7/246: getdents d2/d8 0 2024-09-16T08:57:12.009 INFO:tasks.workunit.client.0.smithi031.stdout:7/247: write d2/d8/f50 [973661,95003] 0 2024-09-16T08:57:12.013 INFO:tasks.workunit.client.0.smithi031.stdout:7/248: creat d2/d2d/f58 x:0 0 0 2024-09-16T08:57:12.013 INFO:tasks.workunit.client.0.smithi031.stdout:7/249: creat d2/d2d/f59 x:0 0 0 2024-09-16T08:57:12.013 INFO:tasks.workunit.client.0.smithi031.stdout:7/250: read - d2/d8/d11/f31 zero size 2024-09-16T08:57:12.015 INFO:tasks.workunit.client.1.smithi097.stdout:5/579: symlink db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/lbb 0 2024-09-16T08:57:12.015 INFO:tasks.workunit.client.1.smithi097.stdout:5/580: write db/d18/d1c/f39 [5867714,59202] 0 2024-09-16T08:57:12.015 INFO:tasks.workunit.client.1.smithi097.stdout:5/581: truncate db/d18/d1c/d42/d46/d7e/d64/d7f/f85 852159 0 2024-09-16T08:57:12.017 INFO:tasks.workunit.client.1.smithi097.stdout:6/543: symlink d1/dc/lac 0 2024-09-16T08:57:12.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/170: dwrite f1 [4194304,4194304] 0 2024-09-16T08:57:12.023 INFO:tasks.workunit.client.1.smithi097.stdout:5/582: mkdir db/d18/d1c/d42/d46/d7e/d64/d7f/d89/dbc 0 2024-09-16T08:57:12.023 INFO:tasks.workunit.client.1.smithi097.stdout:5/583: stat db/d22/f6a 0 2024-09-16T08:57:12.024 INFO:tasks.workunit.client.1.smithi097.stdout:6/544: getdents d1/d3 0 2024-09-16T08:57:12.025 INFO:tasks.workunit.client.1.smithi097.stdout:5/584: creat db/d18/d1c/d42/d46/d7e/d64/fbd x:0 0 0 2024-09-16T08:57:12.026 INFO:tasks.workunit.client.1.smithi097.stdout:3/566: dread d7/d15/f17 [0,4194304] 0 2024-09-16T08:57:12.026 INFO:tasks.workunit.client.1.smithi097.stdout:0/533: sync 2024-09-16T08:57:12.026 INFO:tasks.workunit.client.1.smithi097.stdout:6/545: truncate d1/d3/f7 2328421 0 2024-09-16T08:57:12.028 INFO:tasks.workunit.client.1.smithi097.stdout:5/585: symlink db/d6e/lbe 0 2024-09-16T08:57:12.029 INFO:tasks.workunit.client.1.smithi097.stdout:3/567: creat d7/d15/d52/fbf x:0 0 0 2024-09-16T08:57:12.030 INFO:tasks.workunit.client.1.smithi097.stdout:1/576: dwrite d5/d17/d54/d6b/d71/f76 [0,4194304] 0 2024-09-16T08:57:12.031 INFO:tasks.workunit.client.1.smithi097.stdout:0/534: mknod da/d1a/d41/cbd 0 2024-09-16T08:57:12.031 INFO:tasks.workunit.client.1.smithi097.stdout:0/535: stat da/d1a/d41 0 2024-09-16T08:57:12.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/485: dwrite d2/d30/f3b [0,4194304] 0 2024-09-16T08:57:12.032 INFO:tasks.workunit.client.1.smithi097.stdout:6/546: rename d1/d3/d3e to d1/d3/d81/dad 0 2024-09-16T08:57:12.035 INFO:tasks.workunit.client.1.smithi097.stdout:5/586: unlink db/d18/d1c/d42/d46/d7e/f80 0 2024-09-16T08:57:12.039 INFO:tasks.workunit.client.1.smithi097.stdout:1/577: mknod d5/d17/d54/d6b/cba 0 2024-09-16T08:57:12.039 INFO:tasks.workunit.client.1.smithi097.stdout:1/578: fdatasync d5/db/dc/fad 0 2024-09-16T08:57:12.041 INFO:tasks.workunit.client.1.smithi097.stdout:0/536: symlink da/d7e/lbe 0 2024-09-16T08:57:12.046 INFO:tasks.workunit.client.1.smithi097.stdout:9/486: symlink d2/d30/d74/d8c/d94/d96/l98 0 2024-09-16T08:57:12.047 INFO:tasks.workunit.client.1.smithi097.stdout:5/587: link db/d18/d20/c4d db/d18/d20/d2b/cbf 0 2024-09-16T08:57:12.048 INFO:tasks.workunit.client.1.smithi097.stdout:0/537: mkdir da/d1a/d26/d47/db3/dbf 0 2024-09-16T08:57:12.048 INFO:tasks.workunit.client.1.smithi097.stdout:9/487: creat d2/d30/d74/d8c/d94/f99 x:0 0 0 2024-09-16T08:57:12.049 INFO:tasks.workunit.client.1.smithi097.stdout:5/588: creat db/d18/d1c/d42/d46/d7e/d76/fc0 x:0 0 0 2024-09-16T08:57:12.050 INFO:tasks.workunit.client.1.smithi097.stdout:0/538: getdents da/d24/d4c/d80/d3b 0 2024-09-16T08:57:12.051 INFO:tasks.workunit.client.1.smithi097.stdout:1/579: dread d5/d2c/f4b [0,4194304] 0 2024-09-16T08:57:12.051 INFO:tasks.workunit.client.1.smithi097.stdout:1/580: creat d5/d17/d54/d6b/d71/d94/fbb x:0 0 0 2024-09-16T08:57:12.051 INFO:tasks.workunit.client.1.smithi097.stdout:9/488: mkdir d2/d30/d9a 0 2024-09-16T08:57:12.052 INFO:tasks.workunit.client.1.smithi097.stdout:9/489: fdatasync d2/d30/f47 0 2024-09-16T08:57:12.052 INFO:tasks.workunit.client.1.smithi097.stdout:2/508: sync 2024-09-16T08:57:12.052 INFO:tasks.workunit.client.1.smithi097.stdout:1/581: mknod d5/d17/d54/d6b/d71/d94/cbc 0 2024-09-16T08:57:12.053 INFO:tasks.workunit.client.0.smithi031.stdout:7/251: dwrite d2/d8/d11/f40 [0,4194304] 0 2024-09-16T08:57:12.053 INFO:tasks.workunit.client.0.smithi031.stdout:7/252: chown d2/fb 2613789 1 2024-09-16T08:57:12.054 INFO:tasks.workunit.client.1.smithi097.stdout:9/490: symlink d2/d5/d8d/l9b 0 2024-09-16T08:57:12.054 INFO:tasks.workunit.client.1.smithi097.stdout:3/568: dwrite d7/d15/d52/fbf [0,4194304] 0 2024-09-16T08:57:12.054 INFO:tasks.workunit.client.0.smithi031.stdout:7/253: rename d2/c2a to d2/c5a 0 2024-09-16T08:57:12.056 INFO:tasks.workunit.client.1.smithi097.stdout:9/491: unlink d2/d5/d56/l90 0 2024-09-16T08:57:12.056 INFO:tasks.workunit.client.1.smithi097.stdout:9/492: write d2/d5/d72/f89 [631568,88182] 0 2024-09-16T08:57:12.056 INFO:tasks.workunit.client.1.smithi097.stdout:9/493: stat d2/d3f/d62/d79/c7e 0 2024-09-16T08:57:12.058 INFO:tasks.workunit.client.1.smithi097.stdout:3/569: dread d7/de/d3c/d3f/f6d [0,4194304] 0 2024-09-16T08:57:12.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/570: write d7/d28/d30/d88/fa5 [856410,42766] 0 2024-09-16T08:57:12.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/571: fsync d7/de/d6c/f4a 0 2024-09-16T08:57:12.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/572: truncate d7/de/d29/d35/d3d/d57/d64/f8b 398893 0 2024-09-16T08:57:12.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/573: chown d7/de/d3c/d3f/f50 1139 1 2024-09-16T08:57:12.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/574: stat d7/de/d8d/d90/d96/d9f/db5 0 2024-09-16T08:57:12.059 INFO:tasks.workunit.client.0.smithi031.stdout:3/171: dwrite d3/fa [8388608,4194304] 0 2024-09-16T08:57:12.061 INFO:tasks.workunit.client.1.smithi097.stdout:3/575: read d7/d28/d30/f94 [624612,80466] 0 2024-09-16T08:57:12.061 INFO:tasks.workunit.client.1.smithi097.stdout:0/539: dread da/d1a/f3a [0,4194304] 0 2024-09-16T08:57:12.062 INFO:tasks.workunit.client.0.smithi031.stdout:3/172: rename d3/df/d1a/d24/f25 to d3/df/d1a/d1d/f34 0 2024-09-16T08:57:12.064 INFO:tasks.workunit.client.1.smithi097.stdout:3/576: symlink d7/d15/d52/lc0 0 2024-09-16T08:57:12.069 INFO:tasks.workunit.client.1.smithi097.stdout:3/577: dread d7/de/d6c/f66 [4194304,4194304] 0 2024-09-16T08:57:12.070 INFO:tasks.workunit.client.1.smithi097.stdout:3/578: creat d7/d28/d30/d88/fc1 x:0 0 0 2024-09-16T08:57:12.070 INFO:tasks.workunit.client.1.smithi097.stdout:3/579: fsync d7/de/d3c/d3f/d60/d6b/f86 0 2024-09-16T08:57:12.072 INFO:tasks.workunit.client.1.smithi097.stdout:9/494: dread d2/d5/f71 [0,4194304] 0 2024-09-16T08:57:12.075 INFO:tasks.workunit.client.0.smithi031.stdout:7/254: write d2/d2d/f2e [419481,10645] 0 2024-09-16T08:57:12.077 INFO:tasks.workunit.client.1.smithi097.stdout:8/521: sync 2024-09-16T08:57:12.077 INFO:tasks.workunit.client.1.smithi097.stdout:7/486: sync 2024-09-16T08:57:12.077 INFO:tasks.workunit.client.1.smithi097.stdout:4/531: sync 2024-09-16T08:57:12.079 INFO:tasks.workunit.client.1.smithi097.stdout:9/495: creat d2/d3f/f9c x:0 0 0 2024-09-16T08:57:12.083 INFO:tasks.workunit.client.1.smithi097.stdout:7/487: unlink d4/d9/dc/f48 0 2024-09-16T08:57:12.085 INFO:tasks.workunit.client.1.smithi097.stdout:0/540: dwrite da/d24/d4c/d80/d3b/f4e [0,4194304] 0 2024-09-16T08:57:12.085 INFO:tasks.workunit.client.1.smithi097.stdout:0/541: creat da/df/d3d/fc0 x:0 0 0 2024-09-16T08:57:12.085 INFO:tasks.workunit.client.1.smithi097.stdout:0/542: fdatasync da/d24/d4b/f52 0 2024-09-16T08:57:12.086 INFO:tasks.workunit.client.1.smithi097.stdout:8/522: creat d8/dd/d42/f9e x:0 0 0 2024-09-16T08:57:12.088 INFO:tasks.workunit.client.0.smithi031.stdout:3/173: dread d3/fa [8388608,4194304] 0 2024-09-16T08:57:12.092 INFO:tasks.workunit.client.1.smithi097.stdout:2/509: dwrite d3/d4/d68/d8b/d7c/d7d/f5e [0,4194304] 0 2024-09-16T08:57:12.092 INFO:tasks.workunit.client.1.smithi097.stdout:2/510: creat d3/d4/d9/f95 x:0 0 0 2024-09-16T08:57:12.093 INFO:tasks.workunit.client.1.smithi097.stdout:2/511: write d3/d4/d68/d8b/d7c/d7d/d53/f6c [4959263,60644] 0 2024-09-16T08:57:12.093 INFO:tasks.workunit.client.1.smithi097.stdout:2/512: truncate d3/d4/d68/d8b/d7c/d7d/f8c 437678 0 2024-09-16T08:57:12.093 INFO:tasks.workunit.client.1.smithi097.stdout:2/513: fsync d3/d4/d8/d51/f59 0 2024-09-16T08:57:12.094 INFO:tasks.workunit.client.1.smithi097.stdout:8/523: link d8/dd/d15/d2c/d4a/d4d/d3f/l44 d8/dd/d15/d2c/l9f 0 2024-09-16T08:57:12.094 INFO:tasks.workunit.client.1.smithi097.stdout:8/524: write d8/fa [5140237,52814] 0 2024-09-16T08:57:12.094 INFO:tasks.workunit.client.1.smithi097.stdout:8/525: write d8/dd/d15/f9b [314604,11154] 0 2024-09-16T08:57:12.095 INFO:tasks.workunit.client.0.smithi031.stdout:3/174: creat d3/f35 x:0 0 0 2024-09-16T08:57:12.095 INFO:tasks.workunit.client.0.smithi031.stdout:3/175: fsync d3/f5 0 2024-09-16T08:57:12.095 INFO:tasks.workunit.client.0.smithi031.stdout:3/176: chown d3/d14/c22 6698 1 2024-09-16T08:57:12.096 INFO:tasks.workunit.client.1.smithi097.stdout:4/532: rmdir d0/d7/d52/d26 39 2024-09-16T08:57:12.097 INFO:tasks.workunit.client.1.smithi097.stdout:7/488: read d4/d9/dc/f72 [402253,15891] 0 2024-09-16T08:57:12.097 INFO:tasks.workunit.client.0.smithi031.stdout:3/177: truncate d3/df/d1a/f26 2530813 0 2024-09-16T08:57:12.097 INFO:tasks.workunit.client.0.smithi031.stdout:3/178: dread - d3/f35 zero size 2024-09-16T08:57:12.100 INFO:tasks.workunit.client.1.smithi097.stdout:2/514: unlink d3/f40 0 2024-09-16T08:57:12.104 INFO:tasks.workunit.client.1.smithi097.stdout:4/533: mknod d0/d35/d6c/cba 0 2024-09-16T08:57:12.105 INFO:tasks.workunit.client.1.smithi097.stdout:7/489: mkdir d4/d1b/da4 0 2024-09-16T08:57:12.109 INFO:tasks.workunit.client.1.smithi097.stdout:1/582: dwrite d5/f21 [0,4194304] 0 2024-09-16T08:57:12.110 INFO:tasks.workunit.client.1.smithi097.stdout:9/496: dwrite d2/d5/d3a/d3c/f53 [0,4194304] 0 2024-09-16T08:57:12.111 INFO:tasks.workunit.client.1.smithi097.stdout:3/580: dwrite d7/de/d6c/f66 [0,4194304] 0 2024-09-16T08:57:12.112 INFO:tasks.workunit.client.1.smithi097.stdout:1/583: creat d5/d70/d9d/fbd x:0 0 0 2024-09-16T08:57:12.112 INFO:tasks.workunit.client.1.smithi097.stdout:1/584: chown d5/db/dc/l6d 297492014 1 2024-09-16T08:57:12.116 INFO:tasks.workunit.client.1.smithi097.stdout:9/497: rmdir d2/d5 39 2024-09-16T08:57:12.116 INFO:tasks.workunit.client.1.smithi097.stdout:9/498: chown d2/d3f/d62/f7f 315899 1 2024-09-16T08:57:12.116 INFO:tasks.workunit.client.1.smithi097.stdout:2/515: dread d3/d4/d8/f2b [0,4194304] 0 2024-09-16T08:57:12.116 INFO:tasks.workunit.client.1.smithi097.stdout:2/516: dread - d3/d4/d77/f70 zero size 2024-09-16T08:57:12.116 INFO:tasks.workunit.client.1.smithi097.stdout:2/517: write d3/d4/d68/d8b/d7c/d7d/f8d [1028922,101937] 0 2024-09-16T08:57:12.117 INFO:tasks.workunit.client.1.smithi097.stdout:3/581: creat d7/de/d9c/daf/fc2 x:0 0 0 2024-09-16T08:57:12.117 INFO:tasks.workunit.client.1.smithi097.stdout:1/585: mknod d5/d2c/cbe 0 2024-09-16T08:57:12.118 INFO:tasks.workunit.client.1.smithi097.stdout:9/499: symlink d2/d5/d72/l9d 0 2024-09-16T08:57:12.119 INFO:tasks.workunit.client.1.smithi097.stdout:1/586: symlink d5/d17/d54/d6b/lbf 0 2024-09-16T08:57:12.119 INFO:tasks.workunit.client.1.smithi097.stdout:7/490: dread d4/d1b/f78 [0,4194304] 0 2024-09-16T08:57:12.120 INFO:tasks.workunit.client.1.smithi097.stdout:1/587: creat d5/db/dc/fc0 x:0 0 0 2024-09-16T08:57:12.121 INFO:tasks.workunit.client.1.smithi097.stdout:7/491: symlink d4/d9/d1c/la5 0 2024-09-16T08:57:12.121 INFO:tasks.workunit.client.1.smithi097.stdout:9/500: dread d2/d5/d3a/d3c/f85 [0,4194304] 0 2024-09-16T08:57:12.121 INFO:tasks.workunit.client.1.smithi097.stdout:1/588: truncate d5/f9 4701615 0 2024-09-16T08:57:12.123 INFO:tasks.workunit.client.1.smithi097.stdout:9/501: rename d2/d5/c3e to d2/c9e 0 2024-09-16T08:57:12.123 INFO:tasks.workunit.client.1.smithi097.stdout:9/502: creat d2/d5/d8d/f9f x:0 0 0 2024-09-16T08:57:12.124 INFO:tasks.workunit.client.1.smithi097.stdout:9/503: stat d2/d3f/f55 0 2024-09-16T08:57:12.124 INFO:tasks.workunit.client.1.smithi097.stdout:8/526: dwrite d8/dd/d42/f51 [0,4194304] 0 2024-09-16T08:57:12.124 INFO:tasks.workunit.client.1.smithi097.stdout:1/589: rename d5/c43 to d5/d70/d85/db5/cc1 0 2024-09-16T08:57:12.125 INFO:tasks.workunit.client.1.smithi097.stdout:0/543: dwrite da/d1a/d26/d47/db3/d60/d68/d5e/f72 [4194304,4194304] 0 2024-09-16T08:57:12.125 INFO:tasks.workunit.client.1.smithi097.stdout:7/492: dread d4/d9/d1c/d5f/f6c [0,4194304] 0 2024-09-16T08:57:12.126 INFO:tasks.workunit.client.1.smithi097.stdout:3/582: write d7/de/d29/d35/d3d/d57/f91 [3620649,18931] 0 2024-09-16T08:57:12.127 INFO:tasks.workunit.client.1.smithi097.stdout:9/504: link d2/l15 d2/d5/d56/d6b/la0 0 2024-09-16T08:57:12.128 INFO:tasks.workunit.client.0.smithi031.stdout:9/110: sync 2024-09-16T08:57:12.128 INFO:tasks.workunit.client.0.smithi031.stdout:2/90: sync 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/148: sync 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:1/136: sync 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:4/48: sync 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/149: read - d6/d12/d13/f30 zero size 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:2/91: chown d6/da/l14 937804 1 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:2/92: fdatasync d6/da/fd 0 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/150: creat d6/f34 x:0 0 0 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:4/49: write d0/d2/f13 [1730949,126195] 0 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/151: write d6/fa [1657772,114929] 0 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:4/50: dread - d0/d2/f10 zero size 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/152: dread - d6/f27 zero size 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:4/51: stat d0/d2/f10 0 2024-09-16T08:57:12.129 INFO:tasks.workunit.client.0.smithi031.stdout:8/90: sync 2024-09-16T08:57:12.130 INFO:tasks.workunit.client.0.smithi031.stdout:8/91: chown d4/f19 98 1 2024-09-16T08:57:12.130 INFO:tasks.workunit.client.0.smithi031.stdout:9/111: readlink d3/d5/da/db/l15 0 2024-09-16T08:57:12.130 INFO:tasks.workunit.client.0.smithi031.stdout:9/112: chown d3/d5/da/f14 6088635 1 2024-09-16T08:57:12.130 INFO:tasks.workunit.client.1.smithi097.stdout:2/518: write d3/d4/d68/d8b/d7c/d7d/d53/f58 [3480866,96142] 0 2024-09-16T08:57:12.131 INFO:tasks.workunit.client.1.smithi097.stdout:8/527: rename d8/dd/d15/c6e to d8/dd/d15/ca0 0 2024-09-16T08:57:12.131 INFO:tasks.workunit.client.0.smithi031.stdout:1/137: link d2/d4/f5 d2/de/f2a 0 2024-09-16T08:57:12.131 INFO:tasks.workunit.client.0.smithi031.stdout:1/138: truncate d2/f24 2868 0 2024-09-16T08:57:12.131 INFO:tasks.workunit.client.0.smithi031.stdout:1/139: truncate d2/d4/d22/f23 490816 0 2024-09-16T08:57:12.131 INFO:tasks.workunit.client.1.smithi097.stdout:1/590: symlink d5/d17/d54/d6b/d71/lc2 0 2024-09-16T08:57:12.139 INFO:tasks.workunit.client.1.smithi097.stdout:0/544: rename da/d7e/lbe to da/d24/lc1 0 2024-09-16T08:57:12.139 INFO:tasks.workunit.client.1.smithi097.stdout:0/545: creat da/df/fc2 x:0 0 0 2024-09-16T08:57:12.139 INFO:tasks.workunit.client.1.smithi097.stdout:0/546: fdatasync da/d1a/f63 0 2024-09-16T08:57:12.141 INFO:tasks.workunit.client.1.smithi097.stdout:7/493: mknod d4/d12/d1f/d22/ca6 0 2024-09-16T08:57:12.141 INFO:tasks.workunit.client.1.smithi097.stdout:7/494: truncate d4/d9/dc/f49 1937470 0 2024-09-16T08:57:12.143 INFO:tasks.workunit.client.0.smithi031.stdout:4/52: mkdir d0/d2/d16 0 2024-09-16T08:57:12.145 INFO:tasks.workunit.client.1.smithi097.stdout:3/583: write d7/d15/d52/fbf [2873877,69039] 0 2024-09-16T08:57:12.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/534: dwrite d0/d7/d52/f8d [0,4194304] 0 2024-09-16T08:57:12.148 INFO:tasks.workunit.client.0.smithi031.stdout:9/113: truncate f2 66410 0 2024-09-16T08:57:12.148 INFO:tasks.workunit.client.0.smithi031.stdout:9/114: readlink d3/d5/da/db/l15 0 2024-09-16T08:57:12.148 INFO:tasks.workunit.client.0.smithi031.stdout:9/115: creat d3/d5/da/f27 x:0 0 0 2024-09-16T08:57:12.148 INFO:tasks.workunit.client.0.smithi031.stdout:9/116: write d3/d5/da/f27 [996353,78829] 0 2024-09-16T08:57:12.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/140: creat d2/d4/d1b/f2b x:0 0 0 2024-09-16T08:57:12.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/141: dread - d2/d4/f1c zero size 2024-09-16T08:57:12.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/142: creat d2/f2c x:0 0 0 2024-09-16T08:57:12.151 INFO:tasks.workunit.client.0.smithi031.stdout:0/153: mknod d6/d12/d13/c35 0 2024-09-16T08:57:12.152 INFO:tasks.workunit.client.1.smithi097.stdout:2/519: creat d3/d4/d77/f96 x:0 0 0 2024-09-16T08:57:12.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/53: creat d0/d2/f17 x:0 0 0 2024-09-16T08:57:12.153 INFO:tasks.workunit.client.0.smithi031.stdout:2/93: dwrite d6/da/f17 [4194304,4194304] 0 2024-09-16T08:57:12.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/94: fdatasync d6/da/fe 0 2024-09-16T08:57:12.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/95: creat d6/da/f1d x:0 0 0 2024-09-16T08:57:12.157 INFO:tasks.workunit.client.1.smithi097.stdout:8/528: creat d8/dd/d15/d16/fa1 x:0 0 0 2024-09-16T08:57:12.157 INFO:tasks.workunit.client.1.smithi097.stdout:8/529: write d8/dd/d15/d2c/f41 [1131460,57976] 0 2024-09-16T08:57:12.160 INFO:tasks.workunit.client.0.smithi031.stdout:8/92: rmdir d4 39 2024-09-16T08:57:12.160 INFO:tasks.workunit.client.0.smithi031.stdout:8/93: chown d4/d7/f15 8071 1 2024-09-16T08:57:12.164 INFO:tasks.workunit.client.0.smithi031.stdout:9/117: getdents d3/d5/da/db/d21 0 2024-09-16T08:57:12.167 INFO:tasks.workunit.client.0.smithi031.stdout:1/143: symlink d2/l2d 0 2024-09-16T08:57:12.167 INFO:tasks.workunit.client.0.smithi031.stdout:1/144: fdatasync d2/f29 0 2024-09-16T08:57:12.169 INFO:tasks.workunit.client.1.smithi097.stdout:9/505: dwrite d2/d5/fa [0,4194304] 0 2024-09-16T08:57:12.175 INFO:tasks.workunit.client.1.smithi097.stdout:1/591: rename d5/db/dc/f33 to d5/d2c/fc3 0 2024-09-16T08:57:12.175 INFO:tasks.workunit.client.1.smithi097.stdout:7/495: dwrite d4/d1b/f91 [0,4194304] 0 2024-09-16T08:57:12.175 INFO:tasks.workunit.client.1.smithi097.stdout:7/496: write d4/d7/d7b/f46 [193618,25000] 0 2024-09-16T08:57:12.175 INFO:tasks.workunit.client.1.smithi097.stdout:7/497: fdatasync d4/f6f 0 2024-09-16T08:57:12.175 INFO:tasks.workunit.client.1.smithi097.stdout:7/498: dread - d4/d7/d79/f82 zero size 2024-09-16T08:57:12.175 INFO:tasks.workunit.client.0.smithi031.stdout:3/179: dwrite d3/df/d1a/f26 [0,4194304] 0 2024-09-16T08:57:12.176 INFO:tasks.workunit.client.0.smithi031.stdout:8/94: mknod d4/d7/c1b 0 2024-09-16T08:57:12.176 INFO:tasks.workunit.client.0.smithi031.stdout:8/95: write d4/f8 [17066145,41888] 0 2024-09-16T08:57:12.176 INFO:tasks.workunit.client.0.smithi031.stdout:8/96: write d4/d7/f1a [631748,73664] 0 2024-09-16T08:57:12.176 INFO:tasks.workunit.client.0.smithi031.stdout:4/54: dwrite d0/d2/f17 [0,4194304] 0 2024-09-16T08:57:12.176 INFO:tasks.workunit.client.1.smithi097.stdout:0/547: symlink da/d1a/d26/d94/lc3 0 2024-09-16T08:57:12.178 INFO:tasks.workunit.client.0.smithi031.stdout:9/118: link d3/d5/l12 d3/d5/da/l28 0 2024-09-16T08:57:12.179 INFO:tasks.workunit.client.0.smithi031.stdout:9/119: fdatasync d3/fc 0 2024-09-16T08:57:12.179 INFO:tasks.workunit.client.0.smithi031.stdout:9/120: fdatasync d3/fc 0 2024-09-16T08:57:12.179 INFO:tasks.workunit.client.1.smithi097.stdout:9/506: write d2/d5/f11 [1501384,6448] 0 2024-09-16T08:57:12.179 INFO:tasks.workunit.client.1.smithi097.stdout:9/507: fsync d2/d30/f3b 0 2024-09-16T08:57:12.181 INFO:tasks.workunit.client.0.smithi031.stdout:0/154: mkdir d6/d12/d13/d36 0 2024-09-16T08:57:12.182 INFO:tasks.workunit.client.0.smithi031.stdout:0/155: creat d6/d12/f37 x:0 0 0 2024-09-16T08:57:12.182 INFO:tasks.workunit.client.0.smithi031.stdout:0/156: chown d6/d12/d13/c19 0 1 2024-09-16T08:57:12.183 INFO:tasks.workunit.client.1.smithi097.stdout:3/584: mkdir d7/de/d29/d35/d3d/d57/dc3 0 2024-09-16T08:57:12.183 INFO:tasks.workunit.client.1.smithi097.stdout:3/585: chown d7/de/l24 19530 1 2024-09-16T08:57:12.183 INFO:tasks.workunit.client.1.smithi097.stdout:3/586: chown d7/de/d6c 151445960 1 2024-09-16T08:57:12.183 INFO:tasks.workunit.client.0.smithi031.stdout:3/180: getdents d3/df/d1a/d1d 0 2024-09-16T08:57:12.187 INFO:tasks.workunit.client.1.smithi097.stdout:8/530: unlink d8/dd/d15/d16/l53 0 2024-09-16T08:57:12.187 INFO:tasks.workunit.client.1.smithi097.stdout:8/531: truncate d8/f7f 620364 0 2024-09-16T08:57:12.187 INFO:tasks.workunit.client.1.smithi097.stdout:8/532: chown d8/dd/d15/f34 15399 1 2024-09-16T08:57:12.190 INFO:tasks.workunit.client.1.smithi097.stdout:2/520: rename d3/f3a to d3/d1b/f97 0 2024-09-16T08:57:12.190 INFO:tasks.workunit.client.1.smithi097.stdout:0/548: dread da/df/d3d/f44 [0,4194304] 0 2024-09-16T08:57:12.191 INFO:tasks.workunit.client.1.smithi097.stdout:0/549: stat da/d24/f33 0 2024-09-16T08:57:12.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/121: link d3/d5/da/db/l10 d3/d5/da/db/d21/l29 0 2024-09-16T08:57:12.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/157: creat d6/d12/d13/d36/f38 x:0 0 0 2024-09-16T08:57:12.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/158: read - d6/f31 zero size 2024-09-16T08:57:12.203 INFO:tasks.workunit.client.0.smithi031.stdout:3/181: link d3/df/d33/cd d3/df/d1a/d24/c36 0 2024-09-16T08:57:12.205 INFO:tasks.workunit.client.0.smithi031.stdout:8/97: dwrite d4/f17 [0,4194304] 0 2024-09-16T08:57:12.205 INFO:tasks.workunit.client.0.smithi031.stdout:1/145: link d2/d4/l7 d2/d4/l2e 0 2024-09-16T08:57:12.205 INFO:tasks.workunit.client.0.smithi031.stdout:8/98: write d4/d7/f1a [103767,18485] 0 2024-09-16T08:57:12.206 INFO:tasks.workunit.client.1.smithi097.stdout:7/499: creat d4/d7/d7b/d43/fa7 x:0 0 0 2024-09-16T08:57:12.206 INFO:tasks.workunit.client.1.smithi097.stdout:7/500: fsync d4/d7/d7b/f46 0 2024-09-16T08:57:12.209 INFO:tasks.workunit.client.1.smithi097.stdout:9/508: mknod d2/d5/d6e/ca1 0 2024-09-16T08:57:12.212 INFO:tasks.workunit.client.1.smithi097.stdout:8/533: dwrite d8/dd/d15/f29 [4194304,4194304] 0 2024-09-16T08:57:12.214 INFO:tasks.workunit.client.0.smithi031.stdout:8/99: creat d4/d7/f1c x:0 0 0 2024-09-16T08:57:12.216 INFO:tasks.workunit.client.0.smithi031.stdout:8/100: symlink d4/l1d 0 2024-09-16T08:57:12.217 INFO:tasks.workunit.client.1.smithi097.stdout:4/535: rmdir d0/d36 39 2024-09-16T08:57:12.217 INFO:tasks.workunit.client.1.smithi097.stdout:4/536: write d0/d7/df/d83/fa5 [390240,74901] 0 2024-09-16T08:57:12.217 INFO:tasks.workunit.client.0.smithi031.stdout:3/182: dread d3/df/d1a/d1d/f34 [0,4194304] 0 2024-09-16T08:57:12.219 INFO:tasks.workunit.client.1.smithi097.stdout:2/521: creat d3/d4/d68/d8b/d7c/d7d/d53/d47/f98 x:0 0 0 2024-09-16T08:57:12.219 INFO:tasks.workunit.client.1.smithi097.stdout:2/522: dread - d3/d4/d8/d51/f59 zero size 2024-09-16T08:57:12.219 INFO:tasks.workunit.client.1.smithi097.stdout:2/523: read - d3/d1b/f83 zero size 2024-09-16T08:57:12.219 INFO:tasks.workunit.client.1.smithi097.stdout:2/524: chown d3/d4/d77 617535 1 2024-09-16T08:57:12.220 INFO:tasks.workunit.client.1.smithi097.stdout:0/550: mkdir da/d7e/dc4 0 2024-09-16T08:57:12.220 INFO:tasks.workunit.client.1.smithi097.stdout:0/551: dread - da/d1a/d26/d47/db3/d60/d73/d85/f8d zero size 2024-09-16T08:57:12.220 INFO:tasks.workunit.client.1.smithi097.stdout:0/552: chown da/d1a/d26/l92 3751835 1 2024-09-16T08:57:12.223 INFO:tasks.workunit.client.1.smithi097.stdout:1/592: rmdir d5/d2c 39 2024-09-16T08:57:12.223 INFO:tasks.workunit.client.1.smithi097.stdout:1/593: truncate d5/db/dc/d19/f88 243572 0 2024-09-16T08:57:12.223 INFO:tasks.workunit.client.1.smithi097.stdout:9/509: dread d2/fe [0,4194304] 0 2024-09-16T08:57:12.223 INFO:tasks.workunit.client.1.smithi097.stdout:9/510: chown d2/d3f/c60 13 1 2024-09-16T08:57:12.224 INFO:tasks.workunit.client.1.smithi097.stdout:7/501: mkdir d4/d7/d77/da8 0 2024-09-16T08:57:12.225 INFO:tasks.workunit.client.1.smithi097.stdout:5/589: sync 2024-09-16T08:57:12.225 INFO:tasks.workunit.client.1.smithi097.stdout:6/547: sync 2024-09-16T08:57:12.225 INFO:tasks.workunit.client.1.smithi097.stdout:5/590: dread - db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 zero size 2024-09-16T08:57:12.225 INFO:tasks.workunit.client.1.smithi097.stdout:5/591: chown db/d18/d1c/l94 218885 1 2024-09-16T08:57:12.226 INFO:tasks.workunit.client.1.smithi097.stdout:8/534: unlink d8/dd/f6a 0 2024-09-16T08:57:12.227 INFO:tasks.workunit.client.1.smithi097.stdout:8/535: readlink d8/dd/d42/l93 0 2024-09-16T08:57:12.227 INFO:tasks.workunit.client.1.smithi097.stdout:8/536: chown d8/dd/d15/d2c/l45 130575 1 2024-09-16T08:57:12.227 INFO:tasks.workunit.client.1.smithi097.stdout:0/553: rmdir da/d1a/d26/d94 39 2024-09-16T08:57:12.227 INFO:tasks.workunit.client.1.smithi097.stdout:0/554: stat da/d24/d4c/d80/d3b/c82 0 2024-09-16T08:57:12.228 INFO:tasks.workunit.client.1.smithi097.stdout:0/555: creat da/d1a/d26/d47/d97/d84/fc5 x:0 0 0 2024-09-16T08:57:12.228 INFO:tasks.workunit.client.1.smithi097.stdout:5/592: dread db/d18/d1c/d42/d46/d7e/d64/d7f/f85 [0,4194304] 0 2024-09-16T08:57:12.229 INFO:tasks.workunit.client.1.smithi097.stdout:1/594: truncate d5/d14/d95/fa0 3211418 0 2024-09-16T08:57:12.229 INFO:tasks.workunit.client.1.smithi097.stdout:1/595: creat d5/d70/fc4 x:0 0 0 2024-09-16T08:57:12.230 INFO:tasks.workunit.client.0.smithi031.stdout:3/183: rename d3/df to d3/d14/d37 0 2024-09-16T08:57:12.231 INFO:tasks.workunit.client.0.smithi031.stdout:3/184: chown d3/f4 125 1 2024-09-16T08:57:12.232 INFO:tasks.workunit.client.1.smithi097.stdout:9/511: mkdir d2/d5/d72/da2 0 2024-09-16T08:57:12.232 INFO:tasks.workunit.client.1.smithi097.stdout:9/512: fdatasync d2/d3f/d62/f7f 0 2024-09-16T08:57:12.232 INFO:tasks.workunit.client.1.smithi097.stdout:9/513: fsync d2/d5/fa 0 2024-09-16T08:57:12.232 INFO:tasks.workunit.client.1.smithi097.stdout:9/514: truncate d2/d5/d8d/f9f 491365 0 2024-09-16T08:57:12.232 INFO:tasks.workunit.client.1.smithi097.stdout:9/515: fdatasync d2/d5/d3a/d3c/d73/f7a 0 2024-09-16T08:57:12.233 INFO:tasks.workunit.client.1.smithi097.stdout:6/548: rename d1/d3/d16/c6c to d1/d3/d16/d4f/d9f/cae 0 2024-09-16T08:57:12.233 INFO:tasks.workunit.client.1.smithi097.stdout:2/525: dread d3/d4/d8/f2b [0,4194304] 0 2024-09-16T08:57:12.233 INFO:tasks.workunit.client.1.smithi097.stdout:2/526: read d3/f32 [747299,45116] 0 2024-09-16T08:57:12.233 INFO:tasks.workunit.client.0.smithi031.stdout:3/185: stat c0 0 2024-09-16T08:57:12.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/186: chown d3/d14/d37/d33/l29 128 1 2024-09-16T08:57:12.234 INFO:tasks.workunit.client.1.smithi097.stdout:5/593: getdents db/d6e 0 2024-09-16T08:57:12.236 INFO:tasks.workunit.client.1.smithi097.stdout:8/537: write d8/dd/f19 [467458,119863] 0 2024-09-16T08:57:12.237 INFO:tasks.workunit.client.1.smithi097.stdout:9/516: rename d2/c58 to d2/d5/d3a/d3c/ca3 0 2024-09-16T08:57:12.238 INFO:tasks.workunit.client.1.smithi097.stdout:6/549: mknod d1/dc/d3c/caf 0 2024-09-16T08:57:12.239 INFO:tasks.workunit.client.1.smithi097.stdout:4/537: dwrite d0/fd [0,4194304] 0 2024-09-16T08:57:12.240 INFO:tasks.workunit.client.1.smithi097.stdout:2/527: mknod d3/d4/d68/d8b/d7c/c99 0 2024-09-16T08:57:12.240 INFO:tasks.workunit.client.1.smithi097.stdout:2/528: fdatasync d3/d4/d77/f3c 0 2024-09-16T08:57:12.240 INFO:tasks.workunit.client.1.smithi097.stdout:8/538: mkdir d8/dd/d15/d16/d7c/da2 0 2024-09-16T08:57:12.241 INFO:tasks.workunit.client.1.smithi097.stdout:5/594: rename db/d18/d1c/d42/d46/f6c to db/d18/d1c/fc1 0 2024-09-16T08:57:12.243 INFO:tasks.workunit.client.1.smithi097.stdout:9/517: link d2/d5/d3a/f4c d2/d5/d3a/fa4 0 2024-09-16T08:57:12.243 INFO:tasks.workunit.client.1.smithi097.stdout:9/518: read d2/d5/d72/f89 [673998,129627] 0 2024-09-16T08:57:12.243 INFO:tasks.workunit.client.1.smithi097.stdout:0/556: read da/d1a/d26/f59 [3991144,71771] 0 2024-09-16T08:57:12.244 INFO:tasks.workunit.client.1.smithi097.stdout:9/519: write d2/d30/f32 [4517034,80050] 0 2024-09-16T08:57:12.246 INFO:tasks.workunit.client.1.smithi097.stdout:4/538: mknod d0/d7/d52/d26/cbb 0 2024-09-16T08:57:12.246 INFO:tasks.workunit.client.1.smithi097.stdout:8/539: mknod d8/dd/d42/ca3 0 2024-09-16T08:57:12.246 INFO:tasks.workunit.client.1.smithi097.stdout:8/540: getdents d8/dd/d71/d9c 0 2024-09-16T08:57:12.246 INFO:tasks.workunit.client.1.smithi097.stdout:8/541: creat d8/dd/d15/d16/fa4 x:0 0 0 2024-09-16T08:57:12.247 INFO:tasks.workunit.client.1.smithi097.stdout:0/557: creat da/d1a/d26/d47/db3/d60/d68/d5e/fc6 x:0 0 0 2024-09-16T08:57:12.247 INFO:tasks.workunit.client.1.smithi097.stdout:0/558: creat da/d1a/d26/d47/db3/d60/d68/d5e/fc7 x:0 0 0 2024-09-16T08:57:12.247 INFO:tasks.workunit.client.1.smithi097.stdout:0/559: truncate da/d1a/f58 1512374 0 2024-09-16T08:57:12.247 INFO:tasks.workunit.client.1.smithi097.stdout:0/560: fdatasync da/d1a/d26/d47/db3/d60/d73/fa4 0 2024-09-16T08:57:12.248 INFO:tasks.workunit.client.1.smithi097.stdout:4/539: dread d0/d36/d4c/f80 [0,4194304] 0 2024-09-16T08:57:12.248 INFO:tasks.workunit.client.1.smithi097.stdout:8/542: creat d8/dd/d15/d2c/d4a/d4d/fa5 x:0 0 0 2024-09-16T08:57:12.250 INFO:tasks.workunit.client.1.smithi097.stdout:0/561: rename da/d1a/f81 to da/d1a/d26/d47/db3/d60/d73/d85/fc8 0 2024-09-16T08:57:12.252 INFO:tasks.workunit.client.1.smithi097.stdout:2/529: write d3/d4/d68/d8b/d7c/d7d/d53/d47/f4b [2335561,36187] 0 2024-09-16T08:57:12.253 INFO:tasks.workunit.client.1.smithi097.stdout:4/540: rename d0/d7/d52/d26/d9a to d0/d23/d5f/d7d/dbc 0 2024-09-16T08:57:12.258 INFO:tasks.workunit.client.1.smithi097.stdout:0/562: unlink da/df/d3d/fc0 0 2024-09-16T08:57:12.260 INFO:tasks.workunit.client.1.smithi097.stdout:2/530: mknod d3/d4/d9/d78/c9a 0 2024-09-16T08:57:12.260 INFO:tasks.workunit.client.1.smithi097.stdout:2/531: write d3/d4/d9/f5a [2198772,51916] 0 2024-09-16T08:57:12.260 INFO:tasks.workunit.client.1.smithi097.stdout:2/532: creat d3/f9b x:0 0 0 2024-09-16T08:57:12.260 INFO:tasks.workunit.client.1.smithi097.stdout:2/533: read d3/d4/d68/d8b/d7c/d7d/f1f [2113454,12271] 0 2024-09-16T08:57:12.261 INFO:tasks.workunit.client.1.smithi097.stdout:2/534: truncate d3/d4/d9/d67/f92 43279 0 2024-09-16T08:57:12.261 INFO:tasks.workunit.client.1.smithi097.stdout:2/535: fsync d3/d4/d68/d8b/d7c/d7d/f18 0 2024-09-16T08:57:12.261 INFO:tasks.workunit.client.1.smithi097.stdout:2/536: creat d3/d4/d9/f9c x:0 0 0 2024-09-16T08:57:12.261 INFO:tasks.workunit.client.1.smithi097.stdout:2/537: chown d3/d4/d8/c2e 10369 1 2024-09-16T08:57:12.261 INFO:tasks.workunit.client.1.smithi097.stdout:2/538: dread - d3/d4/f6e zero size 2024-09-16T08:57:12.261 INFO:tasks.workunit.client.1.smithi097.stdout:4/541: creat d0/d23/d5f/d7d/dbc/fbd x:0 0 0 2024-09-16T08:57:12.294 INFO:tasks.workunit.client.0.smithi031.stdout:0/159: dwrite d6/d12/f16 [0,4194304] 0 2024-09-16T08:57:12.294 INFO:tasks.workunit.client.1.smithi097.stdout:4/542: dwrite d0/d7/d52/f8d [0,4194304] 0 2024-09-16T08:57:12.294 INFO:tasks.workunit.client.1.smithi097.stdout:4/543: fsync d0/d7/df/f13 0 2024-09-16T08:57:12.315 INFO:tasks.workunit.client.1.smithi097.stdout:1/596: dwrite d5/d70/fa3 [0,4194304] 0 2024-09-16T08:57:12.316 INFO:tasks.workunit.client.1.smithi097.stdout:1/597: fsync d5/db/dc/fc0 0 2024-09-16T08:57:12.323 INFO:tasks.workunit.client.0.smithi031.stdout:1/146: dwrite d2/de/f2a [0,4194304] 0 2024-09-16T08:57:12.323 INFO:tasks.workunit.client.0.smithi031.stdout:1/147: stat d2/d15/d1f 0 2024-09-16T08:57:12.324 INFO:tasks.workunit.client.0.smithi031.stdout:1/148: creat d2/de/f2f x:0 0 0 2024-09-16T08:57:12.324 INFO:tasks.workunit.client.0.smithi031.stdout:3/187: dwrite d3/d14/d37/d1a/f26 [0,4194304] 0 2024-09-16T08:57:12.324 INFO:tasks.workunit.client.1.smithi097.stdout:5/595: dwrite db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/f9c [4194304,4194304] 0 2024-09-16T08:57:12.324 INFO:tasks.workunit.client.1.smithi097.stdout:2/539: dwrite d3/d4/d9/f9c [0,4194304] 0 2024-09-16T08:57:12.324 INFO:tasks.workunit.client.1.smithi097.stdout:9/520: dwrite d2/d5/f21 [0,4194304] 0 2024-09-16T08:57:12.325 INFO:tasks.workunit.client.1.smithi097.stdout:6/550: dwrite d1/f22 [0,4194304] 0 2024-09-16T08:57:12.327 INFO:tasks.workunit.client.1.smithi097.stdout:6/551: dread d1/d3/d81/dad/d5a/f83 [0,4194304] 0 2024-09-16T08:57:12.333 INFO:tasks.workunit.client.1.smithi097.stdout:5/596: dread db/d22/d58/f7d [0,4194304] 0 2024-09-16T08:57:12.333 INFO:tasks.workunit.client.1.smithi097.stdout:5/597: read - db/d1a/faa zero size 2024-09-16T08:57:12.334 INFO:tasks.workunit.client.1.smithi097.stdout:5/598: readlink db/d6e/d79/l8c 0 2024-09-16T08:57:12.334 INFO:tasks.workunit.client.1.smithi097.stdout:5/599: creat db/d18/d1c/d42/d46/d7e/d8f/fc2 x:0 0 0 2024-09-16T08:57:12.343 INFO:tasks.workunit.client.1.smithi097.stdout:6/552: dwrite d1/d3/d16/f7b [0,4194304] 0 2024-09-16T08:57:12.348 INFO:tasks.workunit.client.1.smithi097.stdout:0/563: mkdir da/d1a/d26/d94/dc9 0 2024-09-16T08:57:12.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/160: creat d6/d12/d13/f39 x:0 0 0 2024-09-16T08:57:12.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/161: stat d6/f11 0 2024-09-16T08:57:12.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/162: creat d6/d12/d13/d36/f3a x:0 0 0 2024-09-16T08:57:12.360 INFO:tasks.workunit.client.0.smithi031.stdout:1/149: creat d2/d4/f30 x:0 0 0 2024-09-16T08:57:12.369 INFO:tasks.workunit.client.1.smithi097.stdout:1/598: mkdir d5/d17/d54/d6b/da8/dc5 0 2024-09-16T08:57:12.369 INFO:tasks.workunit.client.1.smithi097.stdout:1/599: chown d5/d14/c44 2309 1 2024-09-16T08:57:12.372 INFO:tasks.workunit.client.0.smithi031.stdout:0/163: dwrite d6/d12/d13/f2e [0,4194304] 0 2024-09-16T08:57:12.372 INFO:tasks.workunit.client.0.smithi031.stdout:0/164: stat d6/f26 0 2024-09-16T08:57:12.373 INFO:tasks.workunit.client.0.smithi031.stdout:3/188: creat d3/d14/d37/d33/f38 x:0 0 0 2024-09-16T08:57:12.373 INFO:tasks.workunit.client.0.smithi031.stdout:3/189: stat d3/d14/d37/d1a/d24/l32 0 2024-09-16T08:57:12.373 INFO:tasks.workunit.client.0.smithi031.stdout:3/190: chown d3/l15 70588 1 2024-09-16T08:57:12.373 INFO:tasks.workunit.client.0.smithi031.stdout:3/191: creat d3/d14/d37/d33/f39 x:0 0 0 2024-09-16T08:57:12.376 INFO:tasks.workunit.client.0.smithi031.stdout:1/150: creat d2/d4/f31 x:0 0 0 2024-09-16T08:57:12.397 INFO:tasks.workunit.client.0.smithi031.stdout:3/192: mknod d3/d14/d37/d1a/d1d/c3a 0 2024-09-16T08:57:12.400 INFO:tasks.workunit.client.1.smithi097.stdout:1/600: dwrite d5/d2c/f4e [0,4194304] 0 2024-09-16T08:57:12.400 INFO:tasks.workunit.client.1.smithi097.stdout:1/601: dread - d5/d2c/d3b/fa5 zero size 2024-09-16T08:57:12.402 INFO:tasks.workunit.client.0.smithi031.stdout:1/151: unlink d2/l1d 0 2024-09-16T08:57:12.403 INFO:tasks.workunit.client.1.smithi097.stdout:9/521: creat d2/d30/d74/d8c/d94/d96/fa5 x:0 0 0 2024-09-16T08:57:12.404 INFO:tasks.workunit.client.0.smithi031.stdout:0/165: unlink d6/d12/d13/d36/f3a 0 2024-09-16T08:57:12.405 INFO:tasks.workunit.client.0.smithi031.stdout:0/166: write d6/f8 [1653747,48777] 0 2024-09-16T08:57:12.405 INFO:tasks.workunit.client.0.smithi031.stdout:0/167: creat d6/f3b x:0 0 0 2024-09-16T08:57:12.407 INFO:tasks.workunit.client.0.smithi031.stdout:3/193: truncate d3/d14/d37/d1a/d1d/f2b 1746263 0 2024-09-16T08:57:12.410 INFO:tasks.workunit.client.0.smithi031.stdout:1/152: unlink c1 0 2024-09-16T08:57:12.411 INFO:tasks.workunit.client.0.smithi031.stdout:1/153: truncate d2/d4/f28 137000 0 2024-09-16T08:57:12.415 INFO:tasks.workunit.client.1.smithi097.stdout:6/553: mkdir d1/dc/d1e/db0 0 2024-09-16T08:57:12.417 INFO:tasks.workunit.client.1.smithi097.stdout:0/564: unlink da/d24/lc1 0 2024-09-16T08:57:12.419 INFO:tasks.workunit.client.0.smithi031.stdout:6/175: sync 2024-09-16T08:57:12.420 INFO:tasks.workunit.client.0.smithi031.stdout:6/176: creat d0/d17/d19/f2e x:0 0 0 2024-09-16T08:57:12.420 INFO:tasks.workunit.client.0.smithi031.stdout:6/177: creat d0/dd/f2f x:0 0 0 2024-09-16T08:57:12.422 INFO:tasks.workunit.client.0.smithi031.stdout:3/194: write d3/d14/d37/f10 [391741,1153] 0 2024-09-16T08:57:12.422 INFO:tasks.workunit.client.0.smithi031.stdout:3/195: dread - d3/d14/d37/d33/f38 zero size 2024-09-16T08:57:12.422 INFO:tasks.workunit.client.0.smithi031.stdout:3/196: dread - d3/f2a zero size 2024-09-16T08:57:12.422 INFO:tasks.workunit.client.0.smithi031.stdout:3/197: read - d3/d14/d37/d33/f39 zero size 2024-09-16T08:57:12.423 INFO:tasks.workunit.client.1.smithi097.stdout:4/544: truncate d0/d7/f5b 449231 0 2024-09-16T08:57:12.426 INFO:tasks.workunit.client.0.smithi031.stdout:5/189: sync 2024-09-16T08:57:12.427 INFO:tasks.workunit.client.0.smithi031.stdout:6/178: rename d0/dd/f28 to d0/d17/f30 0 2024-09-16T08:57:12.427 INFO:tasks.workunit.client.0.smithi031.stdout:6/179: chown d0/d17/f2d 18607514 1 2024-09-16T08:57:12.429 INFO:tasks.workunit.client.1.smithi097.stdout:2/540: rename d3/d4/d68/d8b/d7c/d7d/f5e to d3/d4/d68/d8b/f9d 0 2024-09-16T08:57:12.432 INFO:tasks.workunit.client.1.smithi097.stdout:9/522: unlink d2/d5/d6e/ca1 0 2024-09-16T08:57:12.432 INFO:tasks.workunit.client.1.smithi097.stdout:9/523: fdatasync d2/d5/d3a/d3c/f53 0 2024-09-16T08:57:12.432 INFO:tasks.workunit.client.1.smithi097.stdout:9/524: read d2/d3f/f41 [268260,46053] 0 2024-09-16T08:57:12.432 INFO:tasks.workunit.client.0.smithi031.stdout:5/190: mknod d2/c37 0 2024-09-16T08:57:12.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/168: dwrite d6/d12/d13/d36/f38 [0,4194304] 0 2024-09-16T08:57:12.434 INFO:tasks.workunit.client.1.smithi097.stdout:6/554: symlink d1/lb1 0 2024-09-16T08:57:12.434 INFO:tasks.workunit.client.1.smithi097.stdout:6/555: write d1/d3/d16/d4f/f92 [114553,103020] 0 2024-09-16T08:57:12.435 INFO:tasks.workunit.client.1.smithi097.stdout:1/602: dwrite d5/d17/d54/d6b/d71/d94/f99 [0,4194304] 0 2024-09-16T08:57:12.435 INFO:tasks.workunit.client.1.smithi097.stdout:1/603: read - d5/d70/fc4 zero size 2024-09-16T08:57:12.435 INFO:tasks.workunit.client.0.smithi031.stdout:1/154: dwrite d2/d4/d1b/f2b [0,4194304] 0 2024-09-16T08:57:12.435 INFO:tasks.workunit.client.0.smithi031.stdout:6/180: creat d0/dd/d2b/f31 x:0 0 0 2024-09-16T08:57:12.435 INFO:tasks.workunit.client.0.smithi031.stdout:6/181: readlink d0/l16 0 2024-09-16T08:57:12.437 INFO:tasks.workunit.client.1.smithi097.stdout:0/565: symlink da/d24/d4c/lca 0 2024-09-16T08:57:12.437 INFO:tasks.workunit.client.0.smithi031.stdout:3/198: write f1 [4652664,6832] 0 2024-09-16T08:57:12.438 INFO:tasks.workunit.client.1.smithi097.stdout:4/545: mknod d0/d7/d52/d26/d8c/db0/cbe 0 2024-09-16T08:57:12.441 INFO:tasks.workunit.client.1.smithi097.stdout:5/600: rename db/d18/d1c/d42/d46/d7e/f78 to db/d18/d1c/d42/d46/d7e/d64/d7f/d89/fc3 0 2024-09-16T08:57:12.450 INFO:tasks.workunit.client.0.smithi031.stdout:1/155: mkdir d2/d15/d32 0 2024-09-16T08:57:12.451 INFO:tasks.workunit.client.0.smithi031.stdout:1/156: creat d2/de/f33 x:0 0 0 2024-09-16T08:57:12.451 INFO:tasks.workunit.client.0.smithi031.stdout:1/157: getdents d2/d15/d32 0 2024-09-16T08:57:12.451 INFO:tasks.workunit.client.1.smithi097.stdout:5/601: write db/d22/f4b [4664021,128336] 0 2024-09-16T08:57:12.452 INFO:tasks.workunit.client.1.smithi097.stdout:6/556: creat d1/d3/d16/d86/fb2 x:0 0 0 2024-09-16T08:57:12.455 INFO:tasks.workunit.client.1.smithi097.stdout:1/604: symlink d5/d70/d85/db5/lc6 0 2024-09-16T08:57:12.456 INFO:tasks.workunit.client.1.smithi097.stdout:0/566: mkdir da/d1a/d26/d94/dc9/dcb 0 2024-09-16T08:57:12.460 INFO:tasks.workunit.client.1.smithi097.stdout:2/541: getdents d3/d4/d9/d67 0 2024-09-16T08:57:12.461 INFO:tasks.workunit.client.0.smithi031.stdout:1/158: mknod d2/c34 0 2024-09-16T08:57:12.462 INFO:tasks.workunit.client.1.smithi097.stdout:5/602: mkdir db/d18/d1c/d42/d46/d7e/d76/dc4 0 2024-09-16T08:57:12.463 INFO:tasks.workunit.client.1.smithi097.stdout:1/605: truncate d5/d70/fa3 3352955 0 2024-09-16T08:57:12.464 INFO:tasks.workunit.client.1.smithi097.stdout:0/567: getdents da/d24/d4c/d80/d42 0 2024-09-16T08:57:12.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/191: dwrite d2/f11 [4194304,4194304] 0 2024-09-16T08:57:12.465 INFO:tasks.workunit.client.0.smithi031.stdout:5/192: readlink d2/dc/d25/d21/d2e/l35 0 2024-09-16T08:57:12.466 INFO:tasks.workunit.client.1.smithi097.stdout:4/546: mkdir d0/d7/d52/dbf 0 2024-09-16T08:57:12.466 INFO:tasks.workunit.client.0.smithi031.stdout:1/159: mknod d2/d4/c35 0 2024-09-16T08:57:12.467 INFO:tasks.workunit.client.1.smithi097.stdout:2/542: rename d3/d1b/f5f to d3/d4/d9/d78/f9e 0 2024-09-16T08:57:12.467 INFO:tasks.workunit.client.1.smithi097.stdout:5/603: creat db/d18/d20/fc5 x:0 0 0 2024-09-16T08:57:12.468 INFO:tasks.workunit.client.1.smithi097.stdout:1/606: mkdir d5/d17/d54/d6b/da8/dc5/dc7 0 2024-09-16T08:57:12.469 INFO:tasks.workunit.client.1.smithi097.stdout:0/568: mkdir da/d1a/d26/d47/db3/dbf/dcc 0 2024-09-16T08:57:12.469 INFO:tasks.workunit.client.1.smithi097.stdout:0/569: truncate da/d24/d4c/d80/f2a 449150 0 2024-09-16T08:57:12.471 INFO:tasks.workunit.client.0.smithi031.stdout:1/160: mkdir d2/d4/d36 0 2024-09-16T08:57:12.471 INFO:tasks.workunit.client.0.smithi031.stdout:5/193: dread d2/dc/f27 [0,4194304] 0 2024-09-16T08:57:12.472 INFO:tasks.workunit.client.1.smithi097.stdout:4/547: rmdir d0/d36/d4c 39 2024-09-16T08:57:12.472 INFO:tasks.workunit.client.1.smithi097.stdout:4/548: chown d0/d7/df/d15/l17 1 1 2024-09-16T08:57:12.473 INFO:tasks.workunit.client.1.smithi097.stdout:2/543: symlink d3/d4/d9/d67/l9f 0 2024-09-16T08:57:12.473 INFO:tasks.workunit.client.1.smithi097.stdout:2/544: write d3/d4/d8/d51/f63 [927204,26156] 0 2024-09-16T08:57:12.475 INFO:tasks.workunit.client.0.smithi031.stdout:5/194: creat d2/dc/d36/f38 x:0 0 0 2024-09-16T08:57:12.476 INFO:tasks.workunit.client.1.smithi097.stdout:5/604: mkdir db/d1a/d6d/dc6 0 2024-09-16T08:57:12.476 INFO:tasks.workunit.client.1.smithi097.stdout:5/605: write db/f84 [10091,13267] 0 2024-09-16T08:57:12.476 INFO:tasks.workunit.client.1.smithi097.stdout:7/502: sync 2024-09-16T08:57:12.477 INFO:tasks.workunit.client.1.smithi097.stdout:8/543: sync 2024-09-16T08:57:12.477 INFO:tasks.workunit.client.1.smithi097.stdout:3/587: sync 2024-09-16T08:57:12.477 INFO:tasks.workunit.client.1.smithi097.stdout:3/588: creat d7/de/d29/d35/d3d/d57/d64/daa/fc4 x:0 0 0 2024-09-16T08:57:12.477 INFO:tasks.workunit.client.0.smithi031.stdout:1/161: mknod d2/d4/d1b/c37 0 2024-09-16T08:57:12.477 INFO:tasks.workunit.client.0.smithi031.stdout:1/162: chown d2/d4/d22/l27 85715 1 2024-09-16T08:57:12.478 INFO:tasks.workunit.client.1.smithi097.stdout:1/607: creat d5/d17/d54/d6b/da8/fc8 x:0 0 0 2024-09-16T08:57:12.480 INFO:tasks.workunit.client.0.smithi031.stdout:5/195: rename d2/dc/d1d/c24 to d2/dc/d25/d21/d2e/c39 0 2024-09-16T08:57:12.480 INFO:tasks.workunit.client.0.smithi031.stdout:5/196: chown d2/f2b 31584050 1 2024-09-16T08:57:12.480 INFO:tasks.workunit.client.0.smithi031.stdout:5/197: dread - d2/f2b zero size 2024-09-16T08:57:12.480 INFO:tasks.workunit.client.0.smithi031.stdout:5/198: fsync d2/dc/f12 0 2024-09-16T08:57:12.480 INFO:tasks.workunit.client.0.smithi031.stdout:0/169: dwrite d6/d12/f16 [4194304,4194304] 0 2024-09-16T08:57:12.481 INFO:tasks.workunit.client.1.smithi097.stdout:9/525: dwrite d2/f46 [0,4194304] 0 2024-09-16T08:57:12.481 INFO:tasks.workunit.client.1.smithi097.stdout:6/557: dwrite d1/d3/d16/d86/f8a [0,4194304] 0 2024-09-16T08:57:12.482 INFO:tasks.workunit.client.0.smithi031.stdout:6/182: dwrite d0/d17/d19/f29 [0,4194304] 0 2024-09-16T08:57:12.482 INFO:tasks.workunit.client.0.smithi031.stdout:3/199: dwrite d3/d14/d37/d33/f38 [0,4194304] 0 2024-09-16T08:57:12.482 INFO:tasks.workunit.client.0.smithi031.stdout:6/183: write d0/d17/f1e [1266599,14907] 0 2024-09-16T08:57:12.482 INFO:tasks.workunit.client.0.smithi031.stdout:6/184: stat d0/dd/c21 0 2024-09-16T08:57:12.482 INFO:tasks.workunit.client.0.smithi031.stdout:3/200: fdatasync d3/d14/d37/d1a/d1d/f2b 0 2024-09-16T08:57:12.482 INFO:tasks.workunit.client.1.smithi097.stdout:4/549: truncate d0/d7/fb 4093439 0 2024-09-16T08:57:12.485 INFO:tasks.workunit.client.0.smithi031.stdout:1/163: mkdir d2/d15/d38 0 2024-09-16T08:57:12.486 INFO:tasks.workunit.client.1.smithi097.stdout:7/503: link d4/d9/dc/d8e/f92 d4/d9/d2b/fa9 0 2024-09-16T08:57:12.487 INFO:tasks.workunit.client.1.smithi097.stdout:7/504: chown d4/d7/d7b/c70 28946494 1 2024-09-16T08:57:12.489 INFO:tasks.workunit.client.1.smithi097.stdout:8/544: write d8/dd/f80 [552664,67345] 0 2024-09-16T08:57:12.493 INFO:tasks.workunit.client.1.smithi097.stdout:3/589: creat d7/de/d3c/d3f/d60/d6b/fc5 x:0 0 0 2024-09-16T08:57:12.494 INFO:tasks.workunit.client.0.smithi031.stdout:6/185: dread d0/d17/d19/f1d [0,4194304] 0 2024-09-16T08:57:12.496 INFO:tasks.workunit.client.1.smithi097.stdout:1/608: rename d5/db/dc/f7d to d5/d17/fc9 0 2024-09-16T08:57:12.500 INFO:tasks.workunit.client.0.smithi031.stdout:3/201: dread f1 [0,4194304] 0 2024-09-16T08:57:12.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/164: truncate d2/d4/f12 170343 0 2024-09-16T08:57:12.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/165: truncate d2/de/f2f 15428 0 2024-09-16T08:57:12.504 INFO:tasks.workunit.client.1.smithi097.stdout:6/558: mkdir d1/dc/d1e/d9e/db3 0 2024-09-16T08:57:12.504 INFO:tasks.workunit.client.1.smithi097.stdout:6/559: creat d1/dc/d1e/fb4 x:0 0 0 2024-09-16T08:57:12.505 INFO:tasks.workunit.client.0.smithi031.stdout:0/170: mkdir d6/d12/d13/d3c 0 2024-09-16T08:57:12.513 INFO:tasks.workunit.client.0.smithi031.stdout:3/202: truncate d3/fa 11408716 0 2024-09-16T08:57:12.514 INFO:tasks.workunit.client.0.smithi031.stdout:3/203: stat d3/f4 0 2024-09-16T08:57:12.515 INFO:tasks.workunit.client.1.smithi097.stdout:4/550: mkdir d0/d35/dc0 0 2024-09-16T08:57:12.517 INFO:tasks.workunit.client.0.smithi031.stdout:1/166: link d2/d4/c25 d2/de/c39 0 2024-09-16T08:57:12.519 INFO:tasks.workunit.client.1.smithi097.stdout:7/505: mkdir d4/daa 0 2024-09-16T08:57:12.520 INFO:tasks.workunit.client.0.smithi031.stdout:5/199: rmdir d2/dc/d36 39 2024-09-16T08:57:12.523 INFO:tasks.workunit.client.0.smithi031.stdout:6/186: rmdir d0/d10 39 2024-09-16T08:57:12.523 INFO:tasks.workunit.client.0.smithi031.stdout:6/187: fdatasync d0/d17/d19/f29 0 2024-09-16T08:57:12.523 INFO:tasks.workunit.client.0.smithi031.stdout:6/188: dread - d0/d10/d14/f24 zero size 2024-09-16T08:57:12.523 INFO:tasks.workunit.client.0.smithi031.stdout:6/189: creat d0/d17/d19/f32 x:0 0 0 2024-09-16T08:57:12.525 INFO:tasks.workunit.client.1.smithi097.stdout:9/526: rename d2/d30/d9a to d2/d5/d3a/d70/da6 0 2024-09-16T08:57:12.525 INFO:tasks.workunit.client.1.smithi097.stdout:9/527: creat d2/d3f/d62/d79/fa7 x:0 0 0 2024-09-16T08:57:12.525 INFO:tasks.workunit.client.1.smithi097.stdout:9/528: dread - d2/d5/d3a/f4c zero size 2024-09-16T08:57:12.526 INFO:tasks.workunit.client.0.smithi031.stdout:3/204: mkdir d3/d14/d37/d1a/d1d/d3b 0 2024-09-16T08:57:12.527 INFO:tasks.workunit.client.0.smithi031.stdout:3/205: fdatasync d3/f35 0 2024-09-16T08:57:12.527 INFO:tasks.workunit.client.0.smithi031.stdout:3/206: write d3/f35 [903515,6554] 0 2024-09-16T08:57:12.527 INFO:tasks.workunit.client.1.smithi097.stdout:1/609: mknod d5/d17/d54/d6b/d71/d94/cca 0 2024-09-16T08:57:12.528 INFO:tasks.workunit.client.1.smithi097.stdout:5/606: dwrite db/d18/d1c/d42/d46/d7e/f9e [0,4194304] 0 2024-09-16T08:57:12.530 INFO:tasks.workunit.client.0.smithi031.stdout:5/200: rename d2/dc/f20 to d2/dc/f3a 0 2024-09-16T08:57:12.530 INFO:tasks.workunit.client.0.smithi031.stdout:5/201: truncate d2/dc/d36/f38 690238 0 2024-09-16T08:57:12.530 INFO:tasks.workunit.client.0.smithi031.stdout:5/202: chown d2 0 1 2024-09-16T08:57:12.531 INFO:tasks.workunit.client.0.smithi031.stdout:1/167: write d2/d4/d1b/f2b [375328,97730] 0 2024-09-16T08:57:12.534 INFO:tasks.workunit.client.0.smithi031.stdout:6/190: unlink d0/dd/c21 0 2024-09-16T08:57:12.534 INFO:tasks.workunit.client.0.smithi031.stdout:6/191: creat d0/dd/d2b/f33 x:0 0 0 2024-09-16T08:57:12.535 INFO:tasks.workunit.client.0.smithi031.stdout:6/192: dread - d0/dd/d2b/f31 zero size 2024-09-16T08:57:12.535 INFO:tasks.workunit.client.0.smithi031.stdout:6/193: readlink d0/dd/l2a 0 2024-09-16T08:57:12.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/207: unlink d3/d14/d37/d1a/d1d/f2b 0 2024-09-16T08:57:12.537 INFO:tasks.workunit.client.0.smithi031.stdout:3/208: truncate d3/f35 1581827 0 2024-09-16T08:57:12.538 INFO:tasks.workunit.client.0.smithi031.stdout:5/203: rename d2/dc/d1d/f2a to d2/dc/f3b 0 2024-09-16T08:57:12.538 INFO:tasks.workunit.client.0.smithi031.stdout:5/204: fsync d2/f34 0 2024-09-16T08:57:12.538 INFO:tasks.workunit.client.0.smithi031.stdout:5/205: dread - d2/f1a zero size 2024-09-16T08:57:12.539 INFO:tasks.workunit.client.0.smithi031.stdout:5/206: chown d2/dc/d25/d21/f29 3983 1 2024-09-16T08:57:12.540 INFO:tasks.workunit.client.1.smithi097.stdout:9/529: write d2/d3f/d62/f7f [666128,79608] 0 2024-09-16T08:57:12.540 INFO:tasks.workunit.client.1.smithi097.stdout:5/607: symlink db/d18/d1c/lc7 0 2024-09-16T08:57:12.540 INFO:tasks.workunit.client.1.smithi097.stdout:9/530: truncate d2/d5/d56/f88 941463 0 2024-09-16T08:57:12.541 INFO:tasks.workunit.client.0.smithi031.stdout:5/207: symlink d2/l3c 0 2024-09-16T08:57:12.542 INFO:tasks.workunit.client.1.smithi097.stdout:5/608: creat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/fc8 x:0 0 0 2024-09-16T08:57:12.542 INFO:tasks.workunit.client.1.smithi097.stdout:5/609: chown db/d1a/f86 3 1 2024-09-16T08:57:12.544 INFO:tasks.workunit.client.1.smithi097.stdout:9/531: mkdir d2/d30/d63/da8 0 2024-09-16T08:57:12.546 INFO:tasks.workunit.client.0.smithi031.stdout:5/208: write d2/fa [2262412,83794] 0 2024-09-16T08:57:12.547 INFO:tasks.workunit.client.1.smithi097.stdout:5/610: truncate db/d6e/d95/fa3 3133843 0 2024-09-16T08:57:12.547 INFO:tasks.workunit.client.1.smithi097.stdout:5/611: chown db/d22/d58 19 1 2024-09-16T08:57:12.547 INFO:tasks.workunit.client.1.smithi097.stdout:5/612: fsync db/fa7 0 2024-09-16T08:57:12.547 INFO:tasks.workunit.client.1.smithi097.stdout:5/613: fdatasync db/f2e 0 2024-09-16T08:57:12.549 INFO:tasks.workunit.client.1.smithi097.stdout:8/545: dwrite d8/dd/f7e [0,4194304] 0 2024-09-16T08:57:12.550 INFO:tasks.workunit.client.1.smithi097.stdout:3/590: dwrite d7/de/d29/d35/f22 [0,4194304] 0 2024-09-16T08:57:12.550 INFO:tasks.workunit.client.1.smithi097.stdout:3/591: creat d7/de/d8d/d90/d92/fc6 x:0 0 0 2024-09-16T08:57:12.550 INFO:tasks.workunit.client.1.smithi097.stdout:6/560: dwrite d1/d3/d16/d4f/d9f/fa4 [0,4194304] 0 2024-09-16T08:57:12.554 INFO:tasks.workunit.client.1.smithi097.stdout:8/546: rename d8/dd/d15/d2c/d4a/d4d/d3f/f77 to d8/dd/d15/d2c/d4a/d4d/d3f/fa6 0 2024-09-16T08:57:12.554 INFO:tasks.workunit.client.1.smithi097.stdout:8/547: fsync d8/dd/d15/d2c/d4a/f87 0 2024-09-16T08:57:12.555 INFO:tasks.workunit.client.0.smithi031.stdout:0/171: dwrite d6/f1e [0,4194304] 0 2024-09-16T08:57:12.557 INFO:tasks.workunit.client.0.smithi031.stdout:3/209: dread d3/d14/d37/d33/f38 [0,4194304] 0 2024-09-16T08:57:12.557 INFO:tasks.workunit.client.0.smithi031.stdout:3/210: fdatasync d3/d14/d37/d1a/f28 0 2024-09-16T08:57:12.557 INFO:tasks.workunit.client.0.smithi031.stdout:3/211: write d3/d14/f2f [1027267,93952] 0 2024-09-16T08:57:12.557 INFO:tasks.workunit.client.0.smithi031.stdout:3/212: creat d3/d14/d37/d33/f3c x:0 0 0 2024-09-16T08:57:12.558 INFO:tasks.workunit.client.1.smithi097.stdout:6/561: rename d1/d3/d16/c42 to d1/dc/d3c/cb5 0 2024-09-16T08:57:12.558 INFO:tasks.workunit.client.1.smithi097.stdout:8/548: symlink d8/dd/d15/d2c/d4a/d4d/d62/la7 0 2024-09-16T08:57:12.559 INFO:tasks.workunit.client.1.smithi097.stdout:4/551: dwrite d0/d7/d52/f67 [0,4194304] 0 2024-09-16T08:57:12.559 INFO:tasks.workunit.client.1.smithi097.stdout:7/506: dwrite d4/d9/d1c/f38 [4194304,4194304] 0 2024-09-16T08:57:12.570 INFO:tasks.workunit.client.1.smithi097.stdout:6/562: getdents d1/d3/d16/d86 0 2024-09-16T08:57:12.570 INFO:tasks.workunit.client.1.smithi097.stdout:6/563: chown d1/dc/d3c/caf 6 1 2024-09-16T08:57:12.571 INFO:tasks.workunit.client.1.smithi097.stdout:6/564: readlink d1/d3/d16/l40 0 2024-09-16T08:57:12.572 INFO:tasks.workunit.client.1.smithi097.stdout:1/610: dwrite d5/d17/f24 [4194304,4194304] 0 2024-09-16T08:57:12.572 INFO:tasks.workunit.client.1.smithi097.stdout:1/611: write d5/d17/d54/d6b/da8/fc8 [975018,87034] 0 2024-09-16T08:57:12.573 INFO:tasks.workunit.client.1.smithi097.stdout:6/565: mkdir d1/d3/db6 0 2024-09-16T08:57:12.577 INFO:tasks.workunit.client.0.smithi031.stdout:5/209: dwrite d2/dc/d25/f2f [0,4194304] 0 2024-09-16T08:57:12.578 INFO:tasks.workunit.client.0.smithi031.stdout:0/172: getdents d6/d12/d13/d36 0 2024-09-16T08:57:12.583 INFO:tasks.workunit.client.1.smithi097.stdout:1/612: rename d5/db/d25 to d5/d2c/dcb 0 2024-09-16T08:57:12.583 INFO:tasks.workunit.client.1.smithi097.stdout:1/613: write d5/d2c/d3b/f7b [515235,101550] 0 2024-09-16T08:57:12.583 INFO:tasks.workunit.client.1.smithi097.stdout:9/532: dwrite d2/d5/d3a/d3c/d73/f7a [0,4194304] 0 2024-09-16T08:57:12.587 INFO:tasks.workunit.client.0.smithi031.stdout:5/210: unlink d2/c37 0 2024-09-16T08:57:12.589 INFO:tasks.workunit.client.0.smithi031.stdout:0/173: symlink d6/l3d 0 2024-09-16T08:57:12.590 INFO:tasks.workunit.client.0.smithi031.stdout:0/174: readlink d6/lf 0 2024-09-16T08:57:12.590 INFO:tasks.workunit.client.0.smithi031.stdout:0/175: write d6/f26 [316499,77439] 0 2024-09-16T08:57:12.593 INFO:tasks.workunit.client.1.smithi097.stdout:6/566: mkdir d1/db7 0 2024-09-16T08:57:12.593 INFO:tasks.workunit.client.1.smithi097.stdout:1/614: link d5/d2c/f4b d5/d70/fcc 0 2024-09-16T08:57:12.593 INFO:tasks.workunit.client.1.smithi097.stdout:1/615: dread - d5/d70/fc4 zero size 2024-09-16T08:57:12.593 INFO:tasks.workunit.client.1.smithi097.stdout:4/552: dwrite d0/d39/f6a [4194304,4194304] 0 2024-09-16T08:57:12.593 INFO:tasks.workunit.client.1.smithi097.stdout:4/553: chown d0/d7/df/d15/c96 0 1 2024-09-16T08:57:12.594 INFO:tasks.workunit.client.1.smithi097.stdout:4/554: readlink d0/d7/df/lad 0 2024-09-16T08:57:12.594 INFO:tasks.workunit.client.1.smithi097.stdout:7/507: dwrite d4/d1b/f35 [0,4194304] 0 2024-09-16T08:57:12.594 INFO:tasks.workunit.client.1.smithi097.stdout:7/508: stat d4/d7/d77/da8 0 2024-09-16T08:57:12.595 INFO:tasks.workunit.client.1.smithi097.stdout:5/614: dwrite db/f84 [0,4194304] 0 2024-09-16T08:57:12.595 INFO:tasks.workunit.client.1.smithi097.stdout:5/615: creat db/d18/d20/fc9 x:0 0 0 2024-09-16T08:57:12.599 INFO:tasks.workunit.client.1.smithi097.stdout:9/533: link f0 d2/d30/d74/d8c/d94/fa9 0 2024-09-16T08:57:12.599 INFO:tasks.workunit.client.0.smithi031.stdout:0/176: getdents d6/d12 0 2024-09-16T08:57:12.600 INFO:tasks.workunit.client.0.smithi031.stdout:0/177: truncate d6/d12/d13/f23 1335821 0 2024-09-16T08:57:12.601 INFO:tasks.workunit.client.0.smithi031.stdout:3/213: dread d3/d14/d37/d1a/d1d/f2d [8388608,4194304] 0 2024-09-16T08:57:12.602 INFO:tasks.workunit.client.0.smithi031.stdout:6/194: rmdir d0 39 2024-09-16T08:57:12.603 INFO:tasks.workunit.client.0.smithi031.stdout:0/178: dread d6/d12/f2c [0,4194304] 0 2024-09-16T08:57:12.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/211: dwrite d2/fb [0,4194304] 0 2024-09-16T08:57:12.604 INFO:tasks.workunit.client.0.smithi031.stdout:0/179: stat d6/d12/f2d 0 2024-09-16T08:57:12.604 INFO:tasks.workunit.client.1.smithi097.stdout:1/616: write d5/d2c/dcb/f93 [3355416,96883] 0 2024-09-16T08:57:12.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/214: mknod d3/d14/d37/d33/c3d 0 2024-09-16T08:57:12.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/215: stat d3/d14/d37/d33 0 2024-09-16T08:57:12.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/216: stat d3/d14/d37/d33 0 2024-09-16T08:57:12.609 INFO:tasks.workunit.client.0.smithi031.stdout:6/195: dread d0/d17/f2d [0,4194304] 0 2024-09-16T08:57:12.609 INFO:tasks.workunit.client.0.smithi031.stdout:6/196: chown d0/d17/d19/f2e 17910247 1 2024-09-16T08:57:12.614 INFO:tasks.workunit.client.0.smithi031.stdout:6/197: creat d0/d17/d19/f34 x:0 0 0 2024-09-16T08:57:12.615 INFO:tasks.workunit.client.1.smithi097.stdout:4/555: getdents d0/d7/d52/d26 0 2024-09-16T08:57:12.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/180: rename d6/f11 to d6/d12/d13/f3e 0 2024-09-16T08:57:12.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/181: fdatasync d6/f33 0 2024-09-16T08:57:12.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/198: mkdir d0/dd/d35 0 2024-09-16T08:57:12.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/199: fsync d0/d17/d19/f26 0 2024-09-16T08:57:12.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/200: readlink d0/dd/l2a 0 2024-09-16T08:57:12.624 INFO:tasks.workunit.client.1.smithi097.stdout:1/617: dread d5/db/dc/f78 [0,4194304] 0 2024-09-16T08:57:12.625 INFO:tasks.workunit.client.0.smithi031.stdout:0/182: mknod d6/c3f 0 2024-09-16T08:57:12.626 INFO:tasks.workunit.client.1.smithi097.stdout:5/616: mkdir db/d18/d1c/d42/d46/d7e/d76/dca 0 2024-09-16T08:57:12.629 INFO:tasks.workunit.client.0.smithi031.stdout:6/201: creat d0/dd/f36 x:0 0 0 2024-09-16T08:57:12.630 INFO:tasks.workunit.client.1.smithi097.stdout:6/567: dwrite d1/f22 [0,4194304] 0 2024-09-16T08:57:12.630 INFO:tasks.workunit.client.1.smithi097.stdout:7/509: dwrite d4/d7/f32 [0,4194304] 0 2024-09-16T08:57:12.631 INFO:tasks.workunit.client.0.smithi031.stdout:5/212: dwrite d2/fa [4194304,4194304] 0 2024-09-16T08:57:12.632 INFO:tasks.workunit.client.1.smithi097.stdout:9/534: symlink d2/d30/d63/da8/laa 0 2024-09-16T08:57:12.632 INFO:tasks.workunit.client.1.smithi097.stdout:9/535: chown d2/d3f/d62/f81 62028912 1 2024-09-16T08:57:12.633 INFO:tasks.workunit.client.0.smithi031.stdout:6/202: mknod d0/d10/d14/c37 0 2024-09-16T08:57:12.633 INFO:tasks.workunit.client.0.smithi031.stdout:6/203: chown d0/d17/f1e 115 1 2024-09-16T08:57:12.635 INFO:tasks.workunit.client.1.smithi097.stdout:4/556: mknod d0/d7/df/d15/d1f/cc1 0 2024-09-16T08:57:12.636 INFO:tasks.workunit.client.0.smithi031.stdout:5/213: mkdir d2/dc/d3d 0 2024-09-16T08:57:12.637 INFO:tasks.workunit.client.0.smithi031.stdout:5/214: truncate d2/dc/d25/d21/f26 285912 0 2024-09-16T08:57:12.637 INFO:tasks.workunit.client.0.smithi031.stdout:5/215: fsync d2/dc/f1f 0 2024-09-16T08:57:12.638 INFO:tasks.workunit.client.0.smithi031.stdout:6/204: symlink d0/l38 0 2024-09-16T08:57:12.639 INFO:tasks.workunit.client.1.smithi097.stdout:1/618: symlink d5/d14/d95/lcd 0 2024-09-16T08:57:12.640 INFO:tasks.workunit.client.0.smithi031.stdout:0/183: dwrite d6/d12/d13/f30 [0,4194304] 0 2024-09-16T08:57:12.640 INFO:tasks.workunit.client.1.smithi097.stdout:5/617: creat db/d22/fcb x:0 0 0 2024-09-16T08:57:12.641 INFO:tasks.workunit.client.0.smithi031.stdout:5/216: symlink d2/dc/d3d/l3e 0 2024-09-16T08:57:12.643 INFO:tasks.workunit.client.0.smithi031.stdout:4/55: sync 2024-09-16T08:57:12.643 INFO:tasks.workunit.client.0.smithi031.stdout:9/122: sync 2024-09-16T08:57:12.643 INFO:tasks.workunit.client.0.smithi031.stdout:2/96: sync 2024-09-16T08:57:12.643 INFO:tasks.workunit.client.0.smithi031.stdout:9/123: write d3/d5/f13 [1979920,16212] 0 2024-09-16T08:57:12.643 INFO:tasks.workunit.client.0.smithi031.stdout:9/124: creat d3/f2a x:0 0 0 2024-09-16T08:57:12.644 INFO:tasks.workunit.client.0.smithi031.stdout:8/101: sync 2024-09-16T08:57:12.644 INFO:tasks.workunit.client.0.smithi031.stdout:7/255: sync 2024-09-16T08:57:12.644 INFO:tasks.workunit.client.0.smithi031.stdout:9/125: creat d3/d5/da/f2b x:0 0 0 2024-09-16T08:57:12.644 INFO:tasks.workunit.client.0.smithi031.stdout:7/256: chown d2/d8/d11 1797078267 1 2024-09-16T08:57:12.645 INFO:tasks.workunit.client.1.smithi097.stdout:6/568: rmdir d1/d3/d81/dad/d5a 39 2024-09-16T08:57:12.645 INFO:tasks.workunit.client.0.smithi031.stdout:4/56: read d0/d2/f13 [1051781,53019] 0 2024-09-16T08:57:12.646 INFO:tasks.workunit.client.0.smithi031.stdout:4/57: creat d0/f18 x:0 0 0 2024-09-16T08:57:12.646 INFO:tasks.workunit.client.0.smithi031.stdout:4/58: getdents d0/d2/d16 0 2024-09-16T08:57:12.648 INFO:tasks.workunit.client.0.smithi031.stdout:2/97: unlink d6/l18 0 2024-09-16T08:57:12.648 INFO:tasks.workunit.client.0.smithi031.stdout:2/98: creat d6/da/f1e x:0 0 0 2024-09-16T08:57:12.651 INFO:tasks.workunit.client.1.smithi097.stdout:5/618: truncate db/d1a/f5b 621354 0 2024-09-16T08:57:12.651 INFO:tasks.workunit.client.1.smithi097.stdout:5/619: fsync db/d18/f68 0 2024-09-16T08:57:12.651 INFO:tasks.workunit.client.1.smithi097.stdout:5/620: chown db/d1a/d6d/c71 109 1 2024-09-16T08:57:12.652 INFO:tasks.workunit.client.1.smithi097.stdout:5/621: creat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/fcc x:0 0 0 2024-09-16T08:57:12.653 INFO:tasks.workunit.client.1.smithi097.stdout:4/557: write d0/d7/fb [4437275,128600] 0 2024-09-16T08:57:12.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/102: mknod d4/c1e 0 2024-09-16T08:57:12.661 INFO:tasks.workunit.client.1.smithi097.stdout:6/569: write d1/d3/d81/f88 [3916512,81830] 0 2024-09-16T08:57:12.662 INFO:tasks.workunit.client.0.smithi031.stdout:4/59: dread d0/d2/f17 [0,4194304] 0 2024-09-16T08:57:12.663 INFO:tasks.workunit.client.1.smithi097.stdout:5/622: unlink db/d1a/l5d 0 2024-09-16T08:57:12.663 INFO:tasks.workunit.client.1.smithi097.stdout:5/623: fsync db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fb6 0 2024-09-16T08:57:12.663 INFO:tasks.workunit.client.1.smithi097.stdout:5/624: truncate db/d18/d1c/d42/d46/d7e/d64/fb3 795274 0 2024-09-16T08:57:12.663 INFO:tasks.workunit.client.1.smithi097.stdout:5/625: truncate db/d18/d1c/d42/d46/f9b 121778 0 2024-09-16T08:57:12.663 INFO:tasks.workunit.client.0.smithi031.stdout:9/126: rename d3/d5/f17 to d3/d5/da/f2c 0 2024-09-16T08:57:12.666 INFO:tasks.workunit.client.1.smithi097.stdout:6/570: creat d1/d3/db6/fb8 x:0 0 0 2024-09-16T08:57:12.670 INFO:tasks.workunit.client.1.smithi097.stdout:5/626: unlink db/d22/fcb 0 2024-09-16T08:57:12.671 INFO:tasks.workunit.client.0.smithi031.stdout:8/103: symlink d4/l1f 0 2024-09-16T08:57:12.672 INFO:tasks.workunit.client.1.smithi097.stdout:6/571: mknod d1/dc/d1e/cb9 0 2024-09-16T08:57:12.674 INFO:tasks.workunit.client.0.smithi031.stdout:2/99: rename d6/da/c1b to d6/c1f 0 2024-09-16T08:57:12.674 INFO:tasks.workunit.client.0.smithi031.stdout:2/100: chown d6/da/c1c 165781 1 2024-09-16T08:57:12.675 INFO:tasks.workunit.client.1.smithi097.stdout:5/627: mknod db/d18/d20/d2b/ccd 0 2024-09-16T08:57:12.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/101: mkdir d6/d20 0 2024-09-16T08:57:12.678 INFO:tasks.workunit.client.0.smithi031.stdout:2/102: chown d6/da/fe 200 1 2024-09-16T08:57:12.678 INFO:tasks.workunit.client.1.smithi097.stdout:6/572: creat d1/d3/d16/d4f/d9f/fba x:0 0 0 2024-09-16T08:57:12.679 INFO:tasks.workunit.client.0.smithi031.stdout:2/103: write d6/da/fe [18945,71506] 0 2024-09-16T08:57:12.680 INFO:tasks.workunit.client.0.smithi031.stdout:2/104: mknod d6/da/c21 0 2024-09-16T08:57:12.685 INFO:tasks.workunit.client.1.smithi097.stdout:9/536: dwrite d2/d5/d56/f5d [0,4194304] 0 2024-09-16T08:57:12.690 INFO:tasks.workunit.client.1.smithi097.stdout:7/510: dwrite d4/d7/d30/f68 [0,4194304] 0 2024-09-16T08:57:12.694 INFO:tasks.workunit.client.1.smithi097.stdout:7/511: mkdir d4/d7/dab 0 2024-09-16T08:57:12.694 INFO:tasks.workunit.client.1.smithi097.stdout:7/512: chown d4/d9/d1c/l45 3 1 2024-09-16T08:57:12.695 INFO:tasks.workunit.client.1.smithi097.stdout:7/513: chown d4/d7/d7b 457 1 2024-09-16T08:57:12.707 INFO:tasks.workunit.client.0.smithi031.stdout:2/105: dread d6/da/fd [0,4194304] 0 2024-09-16T08:57:12.707 INFO:tasks.workunit.client.0.smithi031.stdout:2/106: write d6/f9 [3386909,102468] 0 2024-09-16T08:57:12.708 INFO:tasks.workunit.client.0.smithi031.stdout:5/217: dwrite d2/dc/f2d [0,4194304] 0 2024-09-16T08:57:12.708 INFO:tasks.workunit.client.0.smithi031.stdout:6/205: dwrite d0/ff [0,4194304] 0 2024-09-16T08:57:12.708 INFO:tasks.workunit.client.0.smithi031.stdout:2/107: truncate d6/fc 1972236 0 2024-09-16T08:57:12.708 INFO:tasks.workunit.client.0.smithi031.stdout:6/206: write d0/d17/f1e [413054,105484] 0 2024-09-16T08:57:12.709 INFO:tasks.workunit.client.0.smithi031.stdout:6/207: write d0/dd/d2b/f33 [137817,101774] 0 2024-09-16T08:57:12.709 INFO:tasks.workunit.client.0.smithi031.stdout:5/218: write d2/f13 [1818123,32739] 0 2024-09-16T08:57:12.710 INFO:tasks.workunit.client.0.smithi031.stdout:2/108: symlink d6/l22 0 2024-09-16T08:57:12.710 INFO:tasks.workunit.client.0.smithi031.stdout:6/208: getdents d0/d10 0 2024-09-16T08:57:12.712 INFO:tasks.workunit.client.0.smithi031.stdout:5/219: creat d2/dc/d1d/f3f x:0 0 0 2024-09-16T08:57:12.717 INFO:tasks.workunit.client.0.smithi031.stdout:6/209: symlink d0/dd/d2b/l39 0 2024-09-16T08:57:12.717 INFO:tasks.workunit.client.0.smithi031.stdout:6/210: truncate d0/dd/f23 379241 0 2024-09-16T08:57:12.718 INFO:tasks.workunit.client.0.smithi031.stdout:6/211: write d0/f15 [210138,19711] 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/220: symlink d2/dc/d25/l40 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:6/212: mknod d0/d17/d19/c3a 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:6/213: write d0/dd/f36 [319932,111429] 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/221: creat d2/dc/d36/f41 x:0 0 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:7/257: dwrite d2/d8/d11/f31 [0,4194304] 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:6/214: link d0/dd/d2b/f33 d0/d10/f3b 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/222: rename d2/dc/f27 to d2/dc/d3d/f42 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/223: write d2/dc/f3b [5079703,10406] 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/224: readlink d2/dc/l17 0 2024-09-16T08:57:12.719 INFO:tasks.workunit.client.0.smithi031.stdout:5/225: fdatasync d2/dc/d1d/f3f 0 2024-09-16T08:57:12.720 INFO:tasks.workunit.client.1.smithi097.stdout:7/514: dread d4/d7/d7b/f7c [0,4194304] 0 2024-09-16T08:57:12.721 INFO:tasks.workunit.client.1.smithi097.stdout:7/515: creat d4/d1b/fac x:0 0 0 2024-09-16T08:57:12.722 INFO:tasks.workunit.client.1.smithi097.stdout:7/516: mkdir d4/d9/d2b/dad 0 2024-09-16T08:57:12.722 INFO:tasks.workunit.client.1.smithi097.stdout:7/517: dread - d4/d9/dc/f93 zero size 2024-09-16T08:57:12.723 INFO:tasks.workunit.client.1.smithi097.stdout:7/518: write d4/d9/d51/d88/f9b [2162197,94852] 0 2024-09-16T08:57:12.724 INFO:tasks.workunit.client.1.smithi097.stdout:7/519: creat d4/d7/d79/fae x:0 0 0 2024-09-16T08:57:12.725 INFO:tasks.workunit.client.1.smithi097.stdout:7/520: mkdir d4/d7/d7b/d43/daf 0 2024-09-16T08:57:12.726 INFO:tasks.workunit.client.0.smithi031.stdout:7/258: truncate d2/d8/f50 820149 0 2024-09-16T08:57:12.728 INFO:tasks.workunit.client.1.smithi097.stdout:1/619: dwrite d5/db/dc/d19/f3f [4194304,4194304] 0 2024-09-16T08:57:12.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/215: unlink d0/f8 0 2024-09-16T08:57:12.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/216: read - d0/dd/f2f zero size 2024-09-16T08:57:12.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/217: chown d0/dd/f23 21533 1 2024-09-16T08:57:12.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/218: chown d0/f15 14 1 2024-09-16T08:57:12.729 INFO:tasks.workunit.client.0.smithi031.stdout:4/60: dwrite d0/d2/f13 [0,4194304] 0 2024-09-16T08:57:12.729 INFO:tasks.workunit.client.0.smithi031.stdout:5/226: truncate d2/f5 3367059 0 2024-09-16T08:57:12.730 INFO:tasks.workunit.client.1.smithi097.stdout:7/521: dread d4/d9/dc/f6b [0,4194304] 0 2024-09-16T08:57:12.731 INFO:tasks.workunit.client.0.smithi031.stdout:7/259: truncate d2/d8/f4a 116112 0 2024-09-16T08:57:12.731 INFO:tasks.workunit.client.0.smithi031.stdout:7/260: creat d2/f5b x:0 0 0 2024-09-16T08:57:12.731 INFO:tasks.workunit.client.0.smithi031.stdout:7/261: stat d2/d8/l10 0 2024-09-16T08:57:12.732 INFO:tasks.workunit.client.0.smithi031.stdout:6/219: symlink d0/d17/d19/l3c 0 2024-09-16T08:57:12.732 INFO:tasks.workunit.client.0.smithi031.stdout:6/220: chown d0/fb 25955 1 2024-09-16T08:57:12.734 INFO:tasks.workunit.client.0.smithi031.stdout:2/109: dwrite d6/f8 [4194304,4194304] 0 2024-09-16T08:57:12.734 INFO:tasks.workunit.client.1.smithi097.stdout:6/573: dwrite d1/d3/d16/d4f/f78 [0,4194304] 0 2024-09-16T08:57:12.735 INFO:tasks.workunit.client.0.smithi031.stdout:4/61: creat d0/d2/d16/f19 x:0 0 0 2024-09-16T08:57:12.735 INFO:tasks.workunit.client.0.smithi031.stdout:4/62: write d0/d2/f10 [375063,27223] 0 2024-09-16T08:57:12.736 INFO:tasks.workunit.client.0.smithi031.stdout:9/127: dwrite d3/d5/da/f2b [0,4194304] 0 2024-09-16T08:57:12.736 INFO:tasks.workunit.client.0.smithi031.stdout:8/104: dwrite d4/f6 [0,4194304] 0 2024-09-16T08:57:12.736 INFO:tasks.workunit.client.1.smithi097.stdout:4/558: dwrite d0/d36/d4c/f91 [0,4194304] 0 2024-09-16T08:57:12.736 INFO:tasks.workunit.client.1.smithi097.stdout:9/537: dwrite d2/d5/d56/f8a [0,4194304] 0 2024-09-16T08:57:12.736 INFO:tasks.workunit.client.1.smithi097.stdout:4/559: write d0/f3e [4496577,32210] 0 2024-09-16T08:57:12.737 INFO:tasks.workunit.client.1.smithi097.stdout:4/560: fdatasync d0/d7/d52/f8d 0 2024-09-16T08:57:12.737 INFO:tasks.workunit.client.1.smithi097.stdout:5/628: dwrite db/d18/d20/d2b/f7c [0,4194304] 0 2024-09-16T08:57:12.738 INFO:tasks.workunit.client.0.smithi031.stdout:5/227: creat d2/dc/f43 x:0 0 0 2024-09-16T08:57:12.741 INFO:tasks.workunit.client.0.smithi031.stdout:9/128: dread d3/d5/f13 [0,4194304] 0 2024-09-16T08:57:12.743 INFO:tasks.workunit.client.0.smithi031.stdout:7/262: rename d2/d2d/f59 to d2/d8/d11/f5c 0 2024-09-16T08:57:12.748 INFO:tasks.workunit.client.1.smithi097.stdout:1/620: dwrite d5/f77 [0,4194304] 0 2024-09-16T08:57:12.748 INFO:tasks.workunit.client.1.smithi097.stdout:1/621: read d5/d2c/fc3 [3268220,104796] 0 2024-09-16T08:57:12.750 INFO:tasks.workunit.client.1.smithi097.stdout:7/522: rename d4/d12/d1f/f9f to d4/d1b/d41/fb0 0 2024-09-16T08:57:12.752 INFO:tasks.workunit.client.1.smithi097.stdout:9/538: write d2/d5/f21 [2029114,104722] 0 2024-09-16T08:57:12.752 INFO:tasks.workunit.client.1.smithi097.stdout:9/539: creat d2/d5/d72/fab x:0 0 0 2024-09-16T08:57:12.753 INFO:tasks.workunit.client.0.smithi031.stdout:6/221: dread d0/dd/d2b/f33 [0,4194304] 0 2024-09-16T08:57:12.758 INFO:tasks.workunit.client.0.smithi031.stdout:2/110: dread f4 [0,4194304] 0 2024-09-16T08:57:12.758 INFO:tasks.workunit.client.0.smithi031.stdout:2/111: fdatasync d6/da/f1e 0 2024-09-16T08:57:12.758 INFO:tasks.workunit.client.0.smithi031.stdout:2/112: chown d6/c10 24 1 2024-09-16T08:57:12.759 INFO:tasks.workunit.client.1.smithi097.stdout:6/574: rmdir d1/d3/d81/dad 39 2024-09-16T08:57:12.759 INFO:tasks.workunit.client.1.smithi097.stdout:6/575: truncate d1/d3/fa7 670858 0 2024-09-16T08:57:12.760 INFO:tasks.workunit.client.0.smithi031.stdout:4/63: getdents d0/d2 0 2024-09-16T08:57:12.763 INFO:tasks.workunit.client.0.smithi031.stdout:6/222: mknod d0/d10/d14/c3d 0 2024-09-16T08:57:12.763 INFO:tasks.workunit.client.0.smithi031.stdout:6/223: creat d0/dd/f3e x:0 0 0 2024-09-16T08:57:12.764 INFO:tasks.workunit.client.0.smithi031.stdout:6/224: truncate d0/dd/f3e 315077 0 2024-09-16T08:57:12.765 INFO:tasks.workunit.client.1.smithi097.stdout:7/523: dread d4/d9/dc/f16 [0,4194304] 0 2024-09-16T08:57:12.765 INFO:tasks.workunit.client.1.smithi097.stdout:7/524: creat d4/d1b/d41/fb1 x:0 0 0 2024-09-16T08:57:12.766 INFO:tasks.workunit.client.0.smithi031.stdout:8/105: rmdir d4/d7 39 2024-09-16T08:57:12.767 INFO:tasks.workunit.client.0.smithi031.stdout:5/228: dwrite d2/dc/d25/d21/f29 [0,4194304] 0 2024-09-16T08:57:12.769 INFO:tasks.workunit.client.0.smithi031.stdout:9/129: dwrite d3/f9 [0,4194304] 0 2024-09-16T08:57:12.770 INFO:tasks.workunit.client.1.smithi097.stdout:7/525: dread d4/d7/d30/f5c [0,4194304] 0 2024-09-16T08:57:12.770 INFO:tasks.workunit.client.1.smithi097.stdout:7/526: dread - d4/d7/d7b/d43/fa7 zero size 2024-09-16T08:57:12.773 INFO:tasks.workunit.client.1.smithi097.stdout:4/561: mknod d0/d7/d52/d26/d8c/db0/cc2 0 2024-09-16T08:57:12.774 INFO:tasks.workunit.client.1.smithi097.stdout:4/562: truncate d0/d7/d52/d26/d3d/f82 1333359 0 2024-09-16T08:57:12.774 INFO:tasks.workunit.client.1.smithi097.stdout:4/563: fdatasync d0/d39/d71/faf 0 2024-09-16T08:57:12.774 INFO:tasks.workunit.client.1.smithi097.stdout:4/564: dread - d0/d23/d5f/d7d/f85 zero size 2024-09-16T08:57:12.774 INFO:tasks.workunit.client.1.smithi097.stdout:4/565: chown d0/d36/d4c 35777 1 2024-09-16T08:57:12.774 INFO:tasks.workunit.client.0.smithi031.stdout:4/64: dread d0/fa [0,4194304] 0 2024-09-16T08:57:12.775 INFO:tasks.workunit.client.0.smithi031.stdout:8/106: mknod d4/d7/c20 0 2024-09-16T08:57:12.780 INFO:tasks.workunit.client.1.smithi097.stdout:5/629: mkdir db/d1a/d6d/dce 0 2024-09-16T08:57:12.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/130: rename d3/d5/f22 to d3/d5/f2d 0 2024-09-16T08:57:12.788 INFO:tasks.workunit.client.1.smithi097.stdout:1/622: mkdir d5/dce 0 2024-09-16T08:57:12.788 INFO:tasks.workunit.client.1.smithi097.stdout:1/623: write d5/d14/d95/fb8 [105769,43400] 0 2024-09-16T08:57:12.788 INFO:tasks.workunit.client.1.smithi097.stdout:5/630: write db/d22/f4b [3858476,110246] 0 2024-09-16T08:57:12.788 INFO:tasks.workunit.client.1.smithi097.stdout:5/631: chown db/d18/d1c/d42/d46/d7e/d64/d7f 2 1 2024-09-16T08:57:12.791 INFO:tasks.workunit.client.1.smithi097.stdout:6/576: mkdir d1/d3/dbb 0 2024-09-16T08:57:12.792 INFO:tasks.workunit.client.0.smithi031.stdout:4/65: unlink d0/d2/f17 0 2024-09-16T08:57:12.793 INFO:tasks.workunit.client.0.smithi031.stdout:4/66: truncate d0/f18 10505 0 2024-09-16T08:57:12.793 INFO:tasks.workunit.client.0.smithi031.stdout:4/67: chown d0/d2/l8 10885183 1 2024-09-16T08:57:12.793 INFO:tasks.workunit.client.0.smithi031.stdout:4/68: write d0/fa [4602215,768] 0 2024-09-16T08:57:12.795 INFO:tasks.workunit.client.0.smithi031.stdout:8/107: mkdir d4/d7/d21 0 2024-09-16T08:57:12.800 INFO:tasks.workunit.client.1.smithi097.stdout:7/527: link d4/d9/l7a d4/d7/d77/da8/lb2 0 2024-09-16T08:57:12.801 INFO:tasks.workunit.client.0.smithi031.stdout:4/69: creat d0/d2/f1a x:0 0 0 2024-09-16T08:57:12.802 INFO:tasks.workunit.client.0.smithi031.stdout:9/131: dwrite d3/f9 [0,4194304] 0 2024-09-16T08:57:12.802 INFO:tasks.workunit.client.0.smithi031.stdout:9/132: chown d3/d5/f2d 14631164 1 2024-09-16T08:57:12.803 INFO:tasks.workunit.client.1.smithi097.stdout:4/566: symlink d0/d7/df/lc3 0 2024-09-16T08:57:12.804 INFO:tasks.workunit.client.0.smithi031.stdout:8/108: rename d4/l16 to d4/d7/d21/l22 0 2024-09-16T08:57:12.805 INFO:tasks.workunit.client.1.smithi097.stdout:6/577: write d1/d3/d16/d4f/f78 [2556853,81484] 0 2024-09-16T08:57:12.805 INFO:tasks.workunit.client.1.smithi097.stdout:6/578: write d1/d3/d81/f93 [817071,73812] 0 2024-09-16T08:57:12.807 INFO:tasks.workunit.client.0.smithi031.stdout:4/70: symlink d0/l1b 0 2024-09-16T08:57:12.812 INFO:tasks.workunit.client.0.smithi031.stdout:9/133: mknod d3/c2e 0 2024-09-16T08:57:12.813 INFO:tasks.workunit.client.0.smithi031.stdout:9/134: dread d3/d5/da/f14 [0,4194304] 0 2024-09-16T08:57:12.813 INFO:tasks.workunit.client.0.smithi031.stdout:9/135: read d3/d5/da/db/f18 [34466,45216] 0 2024-09-16T08:57:12.814 INFO:tasks.workunit.client.1.smithi097.stdout:4/567: link d0/d7/d52/d26/cb3 d0/d35/d6c/cc4 0 2024-09-16T08:57:12.814 INFO:tasks.workunit.client.1.smithi097.stdout:4/568: readlink d0/d7/l61 0 2024-09-16T08:57:12.814 INFO:tasks.workunit.client.1.smithi097.stdout:4/569: fdatasync d0/d7/df/d15/f63 0 2024-09-16T08:57:12.816 INFO:tasks.workunit.client.0.smithi031.stdout:4/71: link d0/c14 d0/c1c 0 2024-09-16T08:57:12.816 INFO:tasks.workunit.client.1.smithi097.stdout:2/545: sync 2024-09-16T08:57:12.816 INFO:tasks.workunit.client.1.smithi097.stdout:0/570: sync 2024-09-16T08:57:12.816 INFO:tasks.workunit.client.1.smithi097.stdout:3/592: sync 2024-09-16T08:57:12.816 INFO:tasks.workunit.client.1.smithi097.stdout:2/546: fdatasync d3/d1b/f97 0 2024-09-16T08:57:12.817 INFO:tasks.workunit.client.1.smithi097.stdout:8/549: sync 2024-09-16T08:57:12.818 INFO:tasks.workunit.client.0.smithi031.stdout:9/136: creat d3/d5/da/db/d21/f2f x:0 0 0 2024-09-16T08:57:12.818 INFO:tasks.workunit.client.0.smithi031.stdout:9/137: dread - d3/d5/da/db/d21/f2f zero size 2024-09-16T08:57:12.818 INFO:tasks.workunit.client.0.smithi031.stdout:9/138: chown d3/d5/lf 2 1 2024-09-16T08:57:12.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/229: getdents d2/dc 0 2024-09-16T08:57:12.821 INFO:tasks.workunit.client.0.smithi031.stdout:4/72: rename d0/d2/cd to d0/d2/d16/c1d 0 2024-09-16T08:57:12.821 INFO:tasks.workunit.client.0.smithi031.stdout:4/73: read - d0/d2/d16/f19 zero size 2024-09-16T08:57:12.823 INFO:tasks.workunit.client.1.smithi097.stdout:6/579: rename d1/dc/d1e/c5d to d1/dc/d1e/cbc 0 2024-09-16T08:57:12.824 INFO:tasks.workunit.client.1.smithi097.stdout:4/570: dread d0/d7/df/f2c [0,4194304] 0 2024-09-16T08:57:12.825 INFO:tasks.workunit.client.1.smithi097.stdout:1/624: getdents d5/db 0 2024-09-16T08:57:12.826 INFO:tasks.workunit.client.0.smithi031.stdout:4/74: mkdir d0/d2/d16/d1e 0 2024-09-16T08:57:12.827 INFO:tasks.workunit.client.0.smithi031.stdout:7/263: dwrite d2/d8/f4a [0,4194304] 0 2024-09-16T08:57:12.828 INFO:tasks.workunit.client.1.smithi097.stdout:0/571: creat da/d1a/d26/d47/db3/d60/d73/d85/fcd x:0 0 0 2024-09-16T08:57:12.829 INFO:tasks.workunit.client.0.smithi031.stdout:8/109: dwrite d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:12.830 INFO:tasks.workunit.client.1.smithi097.stdout:3/593: mknod d7/de/d3c/d3f/d60/d6b/db8/cc7 0 2024-09-16T08:57:12.831 INFO:tasks.workunit.client.1.smithi097.stdout:2/547: truncate d3/d4/d68/d8b/d7c/d7d/d53/f6c 440063 0 2024-09-16T08:57:12.832 INFO:tasks.workunit.client.1.smithi097.stdout:2/548: write d3/d4/d68/d8b/d7c/d7d/f8c [259283,63420] 0 2024-09-16T08:57:12.833 INFO:tasks.workunit.client.1.smithi097.stdout:8/550: mknod d8/dd/d71/d9c/ca8 0 2024-09-16T08:57:12.834 INFO:tasks.workunit.client.0.smithi031.stdout:2/113: truncate d6/f8 7431643 0 2024-09-16T08:57:12.834 INFO:tasks.workunit.client.0.smithi031.stdout:4/75: mknod d0/d2/d16/d1e/c1f 0 2024-09-16T08:57:12.836 INFO:tasks.workunit.client.1.smithi097.stdout:6/580: creat d1/dc/d1e/d9e/db3/fbd x:0 0 0 2024-09-16T08:57:12.838 INFO:tasks.workunit.client.1.smithi097.stdout:4/571: mkdir d0/d35/da4/dc5 0 2024-09-16T08:57:12.838 INFO:tasks.workunit.client.1.smithi097.stdout:4/572: chown d0/d35 0 1 2024-09-16T08:57:12.840 INFO:tasks.workunit.client.1.smithi097.stdout:1/625: mknod d5/d70/ccf 0 2024-09-16T08:57:12.843 INFO:tasks.workunit.client.1.smithi097.stdout:0/572: mkdir da/d1a/d26/d47/db3/d60/d73/d85/dce 0 2024-09-16T08:57:12.843 INFO:tasks.workunit.client.1.smithi097.stdout:2/549: dread d3/d4/d9/f9c [0,4194304] 0 2024-09-16T08:57:12.845 INFO:tasks.workunit.client.1.smithi097.stdout:3/594: stat d7/d28/l62 0 2024-09-16T08:57:12.846 INFO:tasks.workunit.client.0.smithi031.stdout:7/264: dwrite d2/d8/fd [0,4194304] 0 2024-09-16T08:57:12.846 INFO:tasks.workunit.client.0.smithi031.stdout:8/110: creat d4/d7/d21/f23 x:0 0 0 2024-09-16T08:57:12.847 INFO:tasks.workunit.client.1.smithi097.stdout:1/626: dread d5/d2c/dcb/f93 [0,4194304] 0 2024-09-16T08:57:12.848 INFO:tasks.workunit.client.1.smithi097.stdout:6/581: symlink d1/d3/lbe 0 2024-09-16T08:57:12.848 INFO:tasks.workunit.client.1.smithi097.stdout:8/551: dread d8/dd/d15/d2c/d4a/d4d/f35 [0,4194304] 0 2024-09-16T08:57:12.849 INFO:tasks.workunit.client.0.smithi031.stdout:2/114: mkdir d6/d20/d23 0 2024-09-16T08:57:12.850 INFO:tasks.workunit.client.1.smithi097.stdout:2/550: creat d3/d4/d68/d8b/fa0 x:0 0 0 2024-09-16T08:57:12.850 INFO:tasks.workunit.client.1.smithi097.stdout:2/551: chown d3/d1b/f97 10 1 2024-09-16T08:57:12.853 INFO:tasks.workunit.client.0.smithi031.stdout:4/76: unlink d0/d2/d16/d1e/c1f 0 2024-09-16T08:57:12.853 INFO:tasks.workunit.client.1.smithi097.stdout:3/595: truncate d7/d15/f17 2610368 0 2024-09-16T08:57:12.853 INFO:tasks.workunit.client.1.smithi097.stdout:3/596: write d7/de/d29/d35/d3d/fb7 [902343,55933] 0 2024-09-16T08:57:12.857 INFO:tasks.workunit.client.1.smithi097.stdout:1/627: rename d5/d2c/d3b/fb4 to d5/d70/d9d/fd0 0 2024-09-16T08:57:12.857 INFO:tasks.workunit.client.1.smithi097.stdout:8/552: rename d8/dd/d15/d2c/d4a/d4e to d8/dd/d15/d2c/d4a/d4e/da9 22 2024-09-16T08:57:12.859 INFO:tasks.workunit.client.0.smithi031.stdout:7/265: mkdir d2/d8/d11/d5d 0 2024-09-16T08:57:12.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/111: write d4/f6 [5755821,15721] 0 2024-09-16T08:57:12.863 INFO:tasks.workunit.client.0.smithi031.stdout:2/115: mknod d6/c24 0 2024-09-16T08:57:12.863 INFO:tasks.workunit.client.0.smithi031.stdout:2/116: stat d6/da/c19 0 2024-09-16T08:57:12.864 INFO:tasks.workunit.client.1.smithi097.stdout:6/582: symlink d1/dc/d1e/d9e/db3/lbf 0 2024-09-16T08:57:12.866 INFO:tasks.workunit.client.0.smithi031.stdout:4/77: unlink d0/l1b 0 2024-09-16T08:57:12.867 INFO:tasks.workunit.client.1.smithi097.stdout:3/597: truncate d7/de/d29/fa7 435449 0 2024-09-16T08:57:12.868 INFO:tasks.workunit.client.0.smithi031.stdout:7/266: stat d2/c33 0 2024-09-16T08:57:12.870 INFO:tasks.workunit.client.1.smithi097.stdout:0/573: dwrite da/df/d3d/f44 [4194304,4194304] 0 2024-09-16T08:57:12.871 INFO:tasks.workunit.client.0.smithi031.stdout:2/117: mknod d6/d20/d23/c25 0 2024-09-16T08:57:12.873 INFO:tasks.workunit.client.1.smithi097.stdout:2/552: dwrite d3/d4/d68/d8b/d7c/d7d/f8c [0,4194304] 0 2024-09-16T08:57:12.873 INFO:tasks.workunit.client.1.smithi097.stdout:1/628: rename d5/db/dc/d19/l3a to d5/db/dc/ld1 0 2024-09-16T08:57:12.873 INFO:tasks.workunit.client.1.smithi097.stdout:2/553: creat d3/d4/d9/fa1 x:0 0 0 2024-09-16T08:57:12.875 INFO:tasks.workunit.client.0.smithi031.stdout:7/267: symlink d2/d8/d11/l5e 0 2024-09-16T08:57:12.876 INFO:tasks.workunit.client.1.smithi097.stdout:6/583: truncate d1/d3/d16/d86/f8a 2527076 0 2024-09-16T08:57:12.879 INFO:tasks.workunit.client.0.smithi031.stdout:2/118: creat d6/d20/f26 x:0 0 0 2024-09-16T08:57:12.881 INFO:tasks.workunit.client.1.smithi097.stdout:8/553: mknod d8/dd/caa 0 2024-09-16T08:57:12.881 INFO:tasks.workunit.client.1.smithi097.stdout:8/554: chown d8/dd/d15/d2c/d4a/d4d/c39 347297 1 2024-09-16T08:57:12.881 INFO:tasks.workunit.client.0.smithi031.stdout:7/268: symlink d2/d2d/l5f 0 2024-09-16T08:57:12.882 INFO:tasks.workunit.client.0.smithi031.stdout:8/112: dwrite d4/f14 [0,4194304] 0 2024-09-16T08:57:12.882 INFO:tasks.workunit.client.0.smithi031.stdout:8/113: chown d4/f19 407 1 2024-09-16T08:57:12.882 INFO:tasks.workunit.client.0.smithi031.stdout:8/114: truncate d4/f18 33189 0 2024-09-16T08:57:12.882 INFO:tasks.workunit.client.0.smithi031.stdout:8/115: chown d4/d7/c20 164886247 1 2024-09-16T08:57:12.883 INFO:tasks.workunit.client.1.smithi097.stdout:0/574: mknod da/d1a/d41/ccf 0 2024-09-16T08:57:12.887 INFO:tasks.workunit.client.1.smithi097.stdout:5/632: truncate db/d22/f4b 763976 0 2024-09-16T08:57:12.888 INFO:tasks.workunit.client.0.smithi031.stdout:8/116: truncate d4/f8 15339910 0 2024-09-16T08:57:12.889 INFO:tasks.workunit.client.0.smithi031.stdout:9/139: rmdir d3 39 2024-09-16T08:57:12.889 INFO:tasks.workunit.client.1.smithi097.stdout:5/633: creat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fcf x:0 0 0 2024-09-16T08:57:12.890 INFO:tasks.workunit.client.1.smithi097.stdout:1/629: mknod d5/d70/d85/cd2 0 2024-09-16T08:57:12.890 INFO:tasks.workunit.client.1.smithi097.stdout:5/634: creat db/fd0 x:0 0 0 2024-09-16T08:57:12.892 INFO:tasks.workunit.client.0.smithi031.stdout:9/140: symlink d3/l30 0 2024-09-16T08:57:12.895 INFO:tasks.workunit.client.1.smithi097.stdout:6/584: creat d1/d3/d16/fc0 x:0 0 0 2024-09-16T08:57:12.896 INFO:tasks.workunit.client.0.smithi031.stdout:4/78: dwrite d0/fb [0,4194304] 0 2024-09-16T08:57:12.897 INFO:tasks.workunit.client.0.smithi031.stdout:4/79: chown d0/d2/f10 881370 1 2024-09-16T08:57:12.898 INFO:tasks.workunit.client.0.smithi031.stdout:7/269: read d2/d8/d11/f24 [3288278,59977] 0 2024-09-16T08:57:12.898 INFO:tasks.workunit.client.1.smithi097.stdout:0/575: mkdir da/d1a/d26/d47/d97/dd0 0 2024-09-16T08:57:12.898 INFO:tasks.workunit.client.1.smithi097.stdout:2/554: fsync d3/d4/d68/d8b/d7c/d7d/d53/f6c 0 2024-09-16T08:57:12.898 INFO:tasks.workunit.client.1.smithi097.stdout:2/555: chown d3/d4/d68/d8b/d7c/d7d/f1f 291 1 2024-09-16T08:57:12.899 INFO:tasks.workunit.client.1.smithi097.stdout:2/556: chown d3/d4/d68/d8b/d7c/d7d/f6b 47 1 2024-09-16T08:57:12.903 INFO:tasks.workunit.client.1.smithi097.stdout:1/630: mkdir d5/d17/d54/d6b/d71/dd3 0 2024-09-16T08:57:12.904 INFO:tasks.workunit.client.1.smithi097.stdout:1/631: stat d5/db/dc/d19/f3f 0 2024-09-16T08:57:12.904 INFO:tasks.workunit.client.1.smithi097.stdout:5/635: mknod db/d18/d1c/d42/d46/d7e/d76/cd1 0 2024-09-16T08:57:12.906 INFO:tasks.workunit.client.0.smithi031.stdout:4/80: dread d0/fb [0,4194304] 0 2024-09-16T08:57:12.907 INFO:tasks.workunit.client.1.smithi097.stdout:6/585: link d1/f5b d1/dc/d1e/fc1 0 2024-09-16T08:57:12.907 INFO:tasks.workunit.client.1.smithi097.stdout:6/586: write d1/d3/d81/f93 [152204,74875] 0 2024-09-16T08:57:12.907 INFO:tasks.workunit.client.1.smithi097.stdout:6/587: creat d1/dc/d1e/fc2 x:0 0 0 2024-09-16T08:57:12.907 INFO:tasks.workunit.client.1.smithi097.stdout:6/588: truncate d1/d3/d16/d86/fb2 356048 0 2024-09-16T08:57:12.907 INFO:tasks.workunit.client.1.smithi097.stdout:6/589: chown d1/l13 0 1 2024-09-16T08:57:12.908 INFO:tasks.workunit.client.0.smithi031.stdout:8/117: read d4/f6 [1950072,8155] 0 2024-09-16T08:57:12.908 INFO:tasks.workunit.client.1.smithi097.stdout:0/576: link da/d1a/d41/f71 da/df/d8a/dac/fd1 0 2024-09-16T08:57:12.908 INFO:tasks.workunit.client.1.smithi097.stdout:0/577: write da/d1a/f2b [221925,95077] 0 2024-09-16T08:57:12.910 INFO:tasks.workunit.client.0.smithi031.stdout:4/81: mknod d0/d2/d16/c20 0 2024-09-16T08:57:12.910 INFO:tasks.workunit.client.0.smithi031.stdout:4/82: creat d0/d2/d16/f21 x:0 0 0 2024-09-16T08:57:12.910 INFO:tasks.workunit.client.0.smithi031.stdout:4/83: readlink d0/d2/l8 0 2024-09-16T08:57:12.910 INFO:tasks.workunit.client.1.smithi097.stdout:5/636: symlink db/d1a/d6d/dce/ld2 0 2024-09-16T08:57:12.911 INFO:tasks.workunit.client.1.smithi097.stdout:5/637: chown db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/fc8 770944119 1 2024-09-16T08:57:12.915 INFO:tasks.workunit.client.0.smithi031.stdout:4/84: write d0/fa [1422122,96620] 0 2024-09-16T08:57:12.915 INFO:tasks.workunit.client.1.smithi097.stdout:3/598: dwrite d7/de/d3c/d3f/f7b [0,4194304] 0 2024-09-16T08:57:12.923 INFO:tasks.workunit.client.1.smithi097.stdout:2/557: getdents d3/d4/d68/d8b 0 2024-09-16T08:57:12.923 INFO:tasks.workunit.client.1.smithi097.stdout:2/558: fdatasync d3/d4/d68/d8b/f4d 0 2024-09-16T08:57:12.923 INFO:tasks.workunit.client.1.smithi097.stdout:2/559: dread - d3/d1b/f83 zero size 2024-09-16T08:57:12.923 INFO:tasks.workunit.client.1.smithi097.stdout:2/560: dread - d3/d4/d68/d8b/f4d zero size 2024-09-16T08:57:12.923 INFO:tasks.workunit.client.1.smithi097.stdout:2/561: write d3/d4/d9/d78/f9e [124391,76680] 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:2/562: truncate d3/d4/f6e 525650 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:2/563: readlink d3/d1b/l3d 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:2/564: fsync d3/d4/d68/d8b/d7c/d7d/f8c 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:2/565: chown d3/f23 18112 1 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:2/566: creat d3/d4/fa2 x:0 0 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:3/599: creat d7/de/d3c/dbe/fc8 x:0 0 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:3/600: chown d7/de/d29/l9a 18 1 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:3/601: stat d7/d28/d30/d88/l8c 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:3/602: write d7/de/d3c/d3f/d60/d6b/fc5 [500413,90083] 0 2024-09-16T08:57:12.924 INFO:tasks.workunit.client.1.smithi097.stdout:6/590: write d1/d3/d81/dad/f62 [2602169,6155] 0 2024-09-16T08:57:12.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/603: chown d7/de/d3c/d3f/d60/d6b/fa3 1149 1 2024-09-16T08:57:12.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/604: read d7/de/d29/f2d [114092,53498] 0 2024-09-16T08:57:12.925 INFO:tasks.workunit.client.1.smithi097.stdout:5/638: rmdir db/d18/d20/d2b 39 2024-09-16T08:57:12.925 INFO:tasks.workunit.client.1.smithi097.stdout:5/639: write db/d18/d1c/d42/d46/d7e/d8f/fc2 [273182,125034] 0 2024-09-16T08:57:12.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/605: mknod d7/de/d3c/d3f/d60/d6b/db8/cc9 0 2024-09-16T08:57:12.927 INFO:tasks.workunit.client.1.smithi097.stdout:5/640: mknod db/d18/d1c/d42/cd3 0 2024-09-16T08:57:12.928 INFO:tasks.workunit.client.1.smithi097.stdout:5/641: creat db/d6e/fd4 x:0 0 0 2024-09-16T08:57:12.928 INFO:tasks.workunit.client.1.smithi097.stdout:5/642: chown db/d18/d20/l77 1 1 2024-09-16T08:57:12.928 INFO:tasks.workunit.client.1.smithi097.stdout:5/643: chown db/d18/d1c/fad 10 1 2024-09-16T08:57:12.938 INFO:tasks.workunit.client.0.smithi031.stdout:2/119: dwrite d6/da/fd [0,4194304] 0 2024-09-16T08:57:12.940 INFO:tasks.workunit.client.1.smithi097.stdout:2/567: dread d3/d4/d68/d8b/d7c/d7d/f18 [0,4194304] 0 2024-09-16T08:57:12.940 INFO:tasks.workunit.client.0.smithi031.stdout:2/120: link d6/da/f1e d6/da/f27 0 2024-09-16T08:57:12.940 INFO:tasks.workunit.client.0.smithi031.stdout:2/121: write d6/fc [2305323,34069] 0 2024-09-16T08:57:12.941 INFO:tasks.workunit.client.1.smithi097.stdout:2/568: mkdir d3/d4/d68/d8b/d7c/d7d/d53/da3 0 2024-09-16T08:57:12.941 INFO:tasks.workunit.client.0.smithi031.stdout:2/122: truncate d6/f13 1492076 0 2024-09-16T08:57:12.942 INFO:tasks.workunit.client.1.smithi097.stdout:2/569: mkdir d3/d4/d68/d8b/d7c/d7d/d53/da3/da4 0 2024-09-16T08:57:12.943 INFO:tasks.workunit.client.0.smithi031.stdout:2/123: mkdir d6/d20/d23/d28 0 2024-09-16T08:57:12.943 INFO:tasks.workunit.client.0.smithi031.stdout:2/124: write f1 [4480645,17112] 0 2024-09-16T08:57:12.944 INFO:tasks.workunit.client.1.smithi097.stdout:2/570: mknod d3/d4/d9/d78/ca5 0 2024-09-16T08:57:12.944 INFO:tasks.workunit.client.1.smithi097.stdout:2/571: write d3/d4/d8/f2b [4088948,100307] 0 2024-09-16T08:57:12.944 INFO:tasks.workunit.client.1.smithi097.stdout:2/572: read d3/d4/d68/d8b/d7c/d7d/d53/f58 [2347292,117922] 0 2024-09-16T08:57:12.944 INFO:tasks.workunit.client.0.smithi031.stdout:7/270: dwrite d2/f1a [0,4194304] 0 2024-09-16T08:57:12.945 INFO:tasks.workunit.client.1.smithi097.stdout:8/555: dwrite d8/f7f [0,4194304] 0 2024-09-16T08:57:12.945 INFO:tasks.workunit.client.1.smithi097.stdout:0/578: dwrite f6 [4194304,4194304] 0 2024-09-16T08:57:12.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/125: mknod d6/d20/c29 0 2024-09-16T08:57:12.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/126: truncate d6/da/f1e 607570 0 2024-09-16T08:57:12.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/127: chown d6/da/ff 383523844 1 2024-09-16T08:57:12.947 INFO:tasks.workunit.client.0.smithi031.stdout:9/141: dwrite d3/f9 [0,4194304] 0 2024-09-16T08:57:12.948 INFO:tasks.workunit.client.1.smithi097.stdout:2/573: truncate d3/d4/d68/d8b/d7c/d7d/d53/d47/f49 2746358 0 2024-09-16T08:57:12.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/271: creat d2/d8/f60 x:0 0 0 2024-09-16T08:57:12.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/272: readlink d2/d8/d11/l5e 0 2024-09-16T08:57:12.957 INFO:tasks.workunit.client.1.smithi097.stdout:8/556: truncate d8/fc 8106566 0 2024-09-16T08:57:12.969 INFO:tasks.workunit.client.1.smithi097.stdout:0/579: link da/c45 da/d24/d4c/d80/d42/cd2 0 2024-09-16T08:57:12.973 INFO:tasks.workunit.client.0.smithi031.stdout:4/85: dwrite d0/fa [0,4194304] 0 2024-09-16T08:57:12.974 INFO:tasks.workunit.client.0.smithi031.stdout:4/86: write d0/fb [4227375,122325] 0 2024-09-16T08:57:12.974 INFO:tasks.workunit.client.1.smithi097.stdout:2/574: rename d3/d4/d8/c2e to d3/d4/d68/d7f/ca6 0 2024-09-16T08:57:12.974 INFO:tasks.workunit.client.1.smithi097.stdout:2/575: write d3/d1b/f83 [775412,55794] 0 2024-09-16T08:57:12.975 INFO:tasks.workunit.client.0.smithi031.stdout:9/142: dwrite d3/f2a [0,4194304] 0 2024-09-16T08:57:12.975 INFO:tasks.workunit.client.0.smithi031.stdout:9/143: write d3/d5/da/db/d21/f2f [348025,87069] 0 2024-09-16T08:57:12.975 INFO:tasks.workunit.client.0.smithi031.stdout:4/87: write d0/f1 [547322,109554] 0 2024-09-16T08:57:12.978 INFO:tasks.workunit.client.0.smithi031.stdout:9/144: dread d3/d5/da/f27 [0,4194304] 0 2024-09-16T08:57:12.978 INFO:tasks.workunit.client.1.smithi097.stdout:8/557: mknod d8/dd/cab 0 2024-09-16T08:57:12.980 INFO:tasks.workunit.client.0.smithi031.stdout:9/145: write d3/d5/f13 [625200,111864] 0 2024-09-16T08:57:12.980 INFO:tasks.workunit.client.0.smithi031.stdout:9/146: write d3/d5/f6 [468404,96912] 0 2024-09-16T08:57:12.981 INFO:tasks.workunit.client.0.smithi031.stdout:8/118: dwrite d4/f8 [8388608,4194304] 0 2024-09-16T08:57:12.983 INFO:tasks.workunit.client.0.smithi031.stdout:9/147: symlink d3/d5/da/l31 0 2024-09-16T08:57:12.983 INFO:tasks.workunit.client.0.smithi031.stdout:9/148: write d3/d5/da/db/f18 [852810,32854] 0 2024-09-16T08:57:12.986 INFO:tasks.workunit.client.0.smithi031.stdout:8/119: mkdir d4/d7/d21/d24 0 2024-09-16T08:57:12.989 INFO:tasks.workunit.client.0.smithi031.stdout:8/120: unlink d4/f14 0 2024-09-16T08:57:12.991 INFO:tasks.workunit.client.0.smithi031.stdout:8/121: mknod d4/d7/d21/d24/c25 0 2024-09-16T08:57:12.998 INFO:tasks.workunit.client.0.smithi031.stdout:8/122: dread d4/f6 [0,4194304] 0 2024-09-16T08:57:12.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/88: dread d0/fb [0,4194304] 0 2024-09-16T08:57:12.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/89: creat d0/d2/d16/f22 x:0 0 0 2024-09-16T08:57:13.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/90: creat d0/d2/d16/d1e/f23 x:0 0 0 2024-09-16T08:57:13.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/91: dread - d0/d2/d16/f21 zero size 2024-09-16T08:57:13.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/92: creat d0/f24 x:0 0 0 2024-09-16T08:57:13.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/93: fsync d0/fa 0 2024-09-16T08:57:13.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/94: dread - d0/d2/d16/f19 zero size 2024-09-16T08:57:13.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/95: chown d0/fa 5468 1 2024-09-16T08:57:13.004 INFO:tasks.workunit.client.0.smithi031.stdout:4/96: mknod d0/d2/d16/c25 0 2024-09-16T08:57:13.004 INFO:tasks.workunit.client.0.smithi031.stdout:4/97: chown d0/l3 17161074 1 2024-09-16T08:57:13.008 INFO:tasks.workunit.client.1.smithi097.stdout:2/576: rmdir d3/d4/d68 39 2024-09-16T08:57:13.008 INFO:tasks.workunit.client.1.smithi097.stdout:2/577: write d3/d4/f6e [1269638,61485] 0 2024-09-16T08:57:13.008 INFO:tasks.workunit.client.1.smithi097.stdout:9/540: sync 2024-09-16T08:57:13.009 INFO:tasks.workunit.client.1.smithi097.stdout:9/541: dread - d2/d5/d72/fab zero size 2024-09-16T08:57:13.009 INFO:tasks.workunit.client.1.smithi097.stdout:9/542: truncate d2/d5/d3a/d3c/f53 1055925 0 2024-09-16T08:57:13.016 INFO:tasks.workunit.client.1.smithi097.stdout:8/558: dwrite d8/dd/d15/d16/f95 [0,4194304] 0 2024-09-16T08:57:13.016 INFO:tasks.workunit.client.1.smithi097.stdout:8/559: truncate d8/dd/d15/d2c/d4a/d4d/d3f/fa6 568975 0 2024-09-16T08:57:13.020 INFO:tasks.workunit.client.1.smithi097.stdout:8/560: mkdir d8/dd/d71/d9c/dac 0 2024-09-16T08:57:13.020 INFO:tasks.workunit.client.1.smithi097.stdout:8/561: stat d8/f31 0 2024-09-16T08:57:13.024 INFO:tasks.workunit.client.0.smithi031.stdout:8/123: dwrite d4/d7/d21/f23 [0,4194304] 0 2024-09-16T08:57:13.031 INFO:tasks.workunit.client.0.smithi031.stdout:9/149: dwrite d3/d5/f2d [0,4194304] 0 2024-09-16T08:57:13.032 INFO:tasks.workunit.client.0.smithi031.stdout:9/150: readlink d3/d5/da/db/d21/l29 0 2024-09-16T08:57:13.032 INFO:tasks.workunit.client.1.smithi097.stdout:0/580: dwrite da/d1a/d26/d47/db3/d60/d73/d85/dad/fb7 [0,4194304] 0 2024-09-16T08:57:13.032 INFO:tasks.workunit.client.1.smithi097.stdout:8/562: dread d8/dd/d15/f29 [0,4194304] 0 2024-09-16T08:57:13.032 INFO:tasks.workunit.client.1.smithi097.stdout:8/563: creat d8/fad x:0 0 0 2024-09-16T08:57:13.033 INFO:tasks.workunit.client.1.smithi097.stdout:0/581: write da/d24/d4b/fa3 [365941,128697] 0 2024-09-16T08:57:13.035 INFO:tasks.workunit.client.1.smithi097.stdout:9/543: dwrite d2/d3f/f43 [0,4194304] 0 2024-09-16T08:57:13.036 INFO:tasks.workunit.client.1.smithi097.stdout:2/578: dwrite d3/d4/d9/f36 [0,4194304] 0 2024-09-16T08:57:13.037 INFO:tasks.workunit.client.0.smithi031.stdout:9/151: link d3/d5/lf d3/d5/da/l32 0 2024-09-16T08:57:13.038 INFO:tasks.workunit.client.1.smithi097.stdout:8/564: rmdir d8/dd/d71/d9c 39 2024-09-16T08:57:13.038 INFO:tasks.workunit.client.1.smithi097.stdout:8/565: write d8/dd/d15/d2c/f41 [680240,93741] 0 2024-09-16T08:57:13.041 INFO:tasks.workunit.client.1.smithi097.stdout:0/582: rename da/d24/f46 to da/df/d8a/fd3 0 2024-09-16T08:57:13.042 INFO:tasks.workunit.client.0.smithi031.stdout:9/152: creat d3/d5/da/f33 x:0 0 0 2024-09-16T08:57:13.043 INFO:tasks.workunit.client.0.smithi031.stdout:9/153: fdatasync d3/d5/da/f27 0 2024-09-16T08:57:13.043 INFO:tasks.workunit.client.0.smithi031.stdout:9/154: chown d3/l30 378639 1 2024-09-16T08:57:13.043 INFO:tasks.workunit.client.1.smithi097.stdout:0/583: dread da/d24/d4c/d80/d3b/f5a [0,4194304] 0 2024-09-16T08:57:13.043 INFO:tasks.workunit.client.1.smithi097.stdout:0/584: chown da/d24/d4b/fa1 489 1 2024-09-16T08:57:13.044 INFO:tasks.workunit.client.1.smithi097.stdout:2/579: mkdir d3/d1b/da7 0 2024-09-16T08:57:13.044 INFO:tasks.workunit.client.0.smithi031.stdout:9/155: read d3/d5/da/f27 [845691,3889] 0 2024-09-16T08:57:13.052 INFO:tasks.workunit.client.1.smithi097.stdout:9/544: rename d2/d3f/d62 to d2/d30/d74/d8c/d94/d96/dac 0 2024-09-16T08:57:13.053 INFO:tasks.workunit.client.1.smithi097.stdout:9/545: chown d2/d5/d3a/d3c/f53 25 1 2024-09-16T08:57:13.053 INFO:tasks.workunit.client.0.smithi031.stdout:8/124: truncate d4/d7/d21/f23 1257547 0 2024-09-16T08:57:13.059 INFO:tasks.workunit.client.1.smithi097.stdout:0/585: truncate da/d1a/d26/d47/db3/d60/d73/d85/dad/fb7 2652173 0 2024-09-16T08:57:13.060 INFO:tasks.workunit.client.1.smithi097.stdout:7/528: sync 2024-09-16T08:57:13.061 INFO:tasks.workunit.client.1.smithi097.stdout:0/586: rmdir da/d1a/d26/d47/db3/d60 39 2024-09-16T08:57:13.061 INFO:tasks.workunit.client.1.smithi097.stdout:0/587: truncate da/d1a/d26/d47/d97/f8f 852228 0 2024-09-16T08:57:13.062 INFO:tasks.workunit.client.1.smithi097.stdout:7/529: creat d4/d7/d19/fb3 x:0 0 0 2024-09-16T08:57:13.063 INFO:tasks.workunit.client.1.smithi097.stdout:0/588: creat da/d1a/d26/d47/db3/fd4 x:0 0 0 2024-09-16T08:57:13.065 INFO:tasks.workunit.client.1.smithi097.stdout:8/566: dwrite d8/dd/d15/f34 [0,4194304] 0 2024-09-16T08:57:13.065 INFO:tasks.workunit.client.1.smithi097.stdout:8/567: chown d8/dd/d15/d16/fa1 91529 1 2024-09-16T08:57:13.065 INFO:tasks.workunit.client.0.smithi031.stdout:1/168: sync 2024-09-16T08:57:13.066 INFO:tasks.workunit.client.0.smithi031.stdout:0/184: sync 2024-09-16T08:57:13.066 INFO:tasks.workunit.client.0.smithi031.stdout:3/217: sync 2024-09-16T08:57:13.066 INFO:tasks.workunit.client.0.smithi031.stdout:6/225: sync 2024-09-16T08:57:13.067 INFO:tasks.workunit.client.1.smithi097.stdout:2/580: dwrite d3/d4/d68/d8b/f4d [0,4194304] 0 2024-09-16T08:57:13.067 INFO:tasks.workunit.client.1.smithi097.stdout:2/581: read - d3/d4/d68/d8b/d7c/d7d/d53/f50 zero size 2024-09-16T08:57:13.067 INFO:tasks.workunit.client.0.smithi031.stdout:0/185: write d6/d12/f17 [1641155,57851] 0 2024-09-16T08:57:13.067 INFO:tasks.workunit.client.0.smithi031.stdout:0/186: truncate d6/f27 432268 0 2024-09-16T08:57:13.070 INFO:tasks.workunit.client.0.smithi031.stdout:1/169: truncate d2/d4/d1b/f2b 29405 0 2024-09-16T08:57:13.071 INFO:tasks.workunit.client.0.smithi031.stdout:0/187: symlink d6/d12/l40 0 2024-09-16T08:57:13.073 INFO:tasks.workunit.client.0.smithi031.stdout:0/188: creat d6/d12/d13/f41 x:0 0 0 2024-09-16T08:57:13.073 INFO:tasks.workunit.client.0.smithi031.stdout:0/189: creat d6/f42 x:0 0 0 2024-09-16T08:57:13.073 INFO:tasks.workunit.client.0.smithi031.stdout:0/190: write d6/f42 [740594,69136] 0 2024-09-16T08:57:13.073 INFO:tasks.workunit.client.0.smithi031.stdout:0/191: fsync d6/f31 0 2024-09-16T08:57:13.074 INFO:tasks.workunit.client.0.smithi031.stdout:8/125: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:13.074 INFO:tasks.workunit.client.1.smithi097.stdout:2/582: mknod d3/d4/d68/d8b/d7c/d7d/d53/ca8 0 2024-09-16T08:57:13.074 INFO:tasks.workunit.client.1.smithi097.stdout:2/583: chown d3/d4/d68/d8b 28032 1 2024-09-16T08:57:13.074 INFO:tasks.workunit.client.0.smithi031.stdout:3/218: read d3/d14/f2f [663452,46217] 0 2024-09-16T08:57:13.075 INFO:tasks.workunit.client.0.smithi031.stdout:3/219: stat d3/d14/d37/d1a/d1d 0 2024-09-16T08:57:13.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/192: creat d6/d12/d13/d36/f43 x:0 0 0 2024-09-16T08:57:13.078 INFO:tasks.workunit.client.1.smithi097.stdout:8/568: dread d8/f24 [0,4194304] 0 2024-09-16T08:57:13.078 INFO:tasks.workunit.client.1.smithi097.stdout:8/569: fsync d8/dd/d15/d2c/d4a/d4d/d3d/f5b 0 2024-09-16T08:57:13.078 INFO:tasks.workunit.client.1.smithi097.stdout:2/584: creat d3/d4/d68/d8b/d7c/d7d/d53/da3/fa9 x:0 0 0 2024-09-16T08:57:13.079 INFO:tasks.workunit.client.0.smithi031.stdout:6/226: write d0/d17/d19/f29 [1173875,17944] 0 2024-09-16T08:57:13.079 INFO:tasks.workunit.client.0.smithi031.stdout:6/227: truncate d0/f15 1787895 0 2024-09-16T08:57:13.080 INFO:tasks.workunit.client.1.smithi097.stdout:8/570: rename d8/dd/d15/d2c/d4a/d4d/d3d/f5b to d8/dd/d71/d9c/fae 0 2024-09-16T08:57:13.080 INFO:tasks.workunit.client.1.smithi097.stdout:0/589: dread da/d1a/d26/d47/db3/d60/d73/f77 [0,4194304] 0 2024-09-16T08:57:13.080 INFO:tasks.workunit.client.1.smithi097.stdout:2/585: mknod d3/d4/d68/d8b/d7c/d7d/d53/d47/caa 0 2024-09-16T08:57:13.082 INFO:tasks.workunit.client.0.smithi031.stdout:0/193: symlink d6/d12/d13/l44 0 2024-09-16T08:57:13.082 INFO:tasks.workunit.client.1.smithi097.stdout:8/571: mknod d8/dd/d15/caf 0 2024-09-16T08:57:13.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/126: write d4/f8 [12836713,60003] 0 2024-09-16T08:57:13.084 INFO:tasks.workunit.client.0.smithi031.stdout:8/127: rename d4/d7/d21/d24 to d4/d7/d21/d24/d26 22 2024-09-16T08:57:13.084 INFO:tasks.workunit.client.0.smithi031.stdout:6/228: mkdir d0/d10/d14/d3f 0 2024-09-16T08:57:13.084 INFO:tasks.workunit.client.0.smithi031.stdout:6/229: chown d0/d10/d14 2712791 1 2024-09-16T08:57:13.085 INFO:tasks.workunit.client.1.smithi097.stdout:2/586: mkdir d3/d4/d8/dab 0 2024-09-16T08:57:13.085 INFO:tasks.workunit.client.0.smithi031.stdout:1/170: dread d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:13.085 INFO:tasks.workunit.client.0.smithi031.stdout:1/171: read d2/f14 [7532390,113510] 0 2024-09-16T08:57:13.085 INFO:tasks.workunit.client.0.smithi031.stdout:1/172: write d2/d4/d22/f23 [523645,124115] 0 2024-09-16T08:57:13.086 INFO:tasks.workunit.client.1.smithi097.stdout:8/572: truncate d8/dd/d71/d9c/fae 2601711 0 2024-09-16T08:57:13.089 INFO:tasks.workunit.client.1.smithi097.stdout:7/530: dwrite d4/d12/d1f/f3b [0,4194304] 0 2024-09-16T08:57:13.089 INFO:tasks.workunit.client.1.smithi097.stdout:7/531: readlink d4/d1b/d41/l62 0 2024-09-16T08:57:13.089 INFO:tasks.workunit.client.1.smithi097.stdout:7/532: truncate d4/d1b/d41/f95 347176 0 2024-09-16T08:57:13.090 INFO:tasks.workunit.client.1.smithi097.stdout:9/546: dwrite d2/d5/d8d/f9f [0,4194304] 0 2024-09-16T08:57:13.090 INFO:tasks.workunit.client.0.smithi031.stdout:6/230: creat d0/d17/f40 x:0 0 0 2024-09-16T08:57:13.093 INFO:tasks.workunit.client.0.smithi031.stdout:3/220: write d3/fb [3896353,83363] 0 2024-09-16T08:57:13.094 INFO:tasks.workunit.client.1.smithi097.stdout:8/573: creat d8/dd/d15/d2c/d4a/d4d/d3f/d99/fb0 x:0 0 0 2024-09-16T08:57:13.095 INFO:tasks.workunit.client.1.smithi097.stdout:8/574: write d8/dd/d15/f29 [2055137,29708] 0 2024-09-16T08:57:13.097 INFO:tasks.workunit.client.0.smithi031.stdout:8/128: dread d4/f6 [0,4194304] 0 2024-09-16T08:57:13.097 INFO:tasks.workunit.client.1.smithi097.stdout:9/547: dread d2/d30/d63/f77 [0,4194304] 0 2024-09-16T08:57:13.103 INFO:tasks.workunit.client.1.smithi097.stdout:9/548: dread d2/fe [0,4194304] 0 2024-09-16T08:57:13.103 INFO:tasks.workunit.client.1.smithi097.stdout:9/549: truncate d2/d3f/f55 66162 0 2024-09-16T08:57:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/129: getdents d4/d7 0 2024-09-16T08:57:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/130: fsync d4/f6 0 2024-09-16T08:57:13.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/131: write d4/d7/f1a [1153442,25702] 0 2024-09-16T08:57:13.107 INFO:tasks.workunit.client.1.smithi097.stdout:8/575: symlink d8/dd/d42/lb1 0 2024-09-16T08:57:13.109 INFO:tasks.workunit.client.0.smithi031.stdout:3/221: dread d3/d14/d37/f10 [4194304,4194304] 0 2024-09-16T08:57:13.119 INFO:tasks.workunit.client.1.smithi097.stdout:4/573: sync 2024-09-16T08:57:13.120 INFO:tasks.workunit.client.1.smithi097.stdout:4/574: creat d0/d7/d52/dbf/fc6 x:0 0 0 2024-09-16T08:57:13.123 INFO:tasks.workunit.client.1.smithi097.stdout:0/590: dwrite da/d24/d76/f9a [0,4194304] 0 2024-09-16T08:57:13.124 INFO:tasks.workunit.client.0.smithi031.stdout:6/231: fsync d0/d17/d19/f29 0 2024-09-16T08:57:13.124 INFO:tasks.workunit.client.0.smithi031.stdout:6/232: chown d0/dd/d2b 434 1 2024-09-16T08:57:13.129 INFO:tasks.workunit.client.1.smithi097.stdout:0/591: mkdir da/d1a/d26/d94/dd5 0 2024-09-16T08:57:13.129 INFO:tasks.workunit.client.1.smithi097.stdout:4/575: dread d0/f30 [0,4194304] 0 2024-09-16T08:57:13.129 INFO:tasks.workunit.client.1.smithi097.stdout:0/592: creat da/d1a/d41/db2/fd6 x:0 0 0 2024-09-16T08:57:13.130 INFO:tasks.workunit.client.1.smithi097.stdout:0/593: creat da/d1a/d26/d94/dd5/fd7 x:0 0 0 2024-09-16T08:57:13.130 INFO:tasks.workunit.client.1.smithi097.stdout:0/594: chown da/df/d3d/f66 372089102 1 2024-09-16T08:57:13.130 INFO:tasks.workunit.client.1.smithi097.stdout:4/576: dread d0/d7/df/d83/fa5 [0,4194304] 0 2024-09-16T08:57:13.131 INFO:tasks.workunit.client.1.smithi097.stdout:0/595: mkdir da/df/dd8 0 2024-09-16T08:57:13.131 INFO:tasks.workunit.client.1.smithi097.stdout:4/577: truncate d0/d7/f75 528420 0 2024-09-16T08:57:13.131 INFO:tasks.workunit.client.0.smithi031.stdout:3/222: dread d3/f5 [0,4194304] 0 2024-09-16T08:57:13.132 INFO:tasks.workunit.client.0.smithi031.stdout:0/194: dwrite d6/f27 [0,4194304] 0 2024-09-16T08:57:13.133 INFO:tasks.workunit.client.1.smithi097.stdout:2/587: dwrite d3/d4/d9/fa1 [0,4194304] 0 2024-09-16T08:57:13.133 INFO:tasks.workunit.client.1.smithi097.stdout:2/588: chown d3/d4/d68/d8b/d7c/d7d/d53/da3 8608022 1 2024-09-16T08:57:13.134 INFO:tasks.workunit.client.1.smithi097.stdout:4/578: rename d0/d7/df/lad to d0/d35/d6c/lc7 0 2024-09-16T08:57:13.135 INFO:tasks.workunit.client.1.smithi097.stdout:2/589: dread d3/d4/d68/d8b/d7c/d7d/f6b [0,4194304] 0 2024-09-16T08:57:13.135 INFO:tasks.workunit.client.1.smithi097.stdout:2/590: write d3/d4/d68/d8b/f60 [475168,104115] 0 2024-09-16T08:57:13.136 INFO:tasks.workunit.client.1.smithi097.stdout:2/591: chown d3/d4/d9/d78/f9e 3410402 1 2024-09-16T08:57:13.136 INFO:tasks.workunit.client.0.smithi031.stdout:0/195: creat d6/d12/d13/d3c/f45 x:0 0 0 2024-09-16T08:57:13.139 INFO:tasks.workunit.client.1.smithi097.stdout:7/533: dwrite d4/f5 [0,4194304] 0 2024-09-16T08:57:13.140 INFO:tasks.workunit.client.0.smithi031.stdout:0/196: creat d6/d12/d13/f46 x:0 0 0 2024-09-16T08:57:13.140 INFO:tasks.workunit.client.0.smithi031.stdout:0/197: dread - d6/d12/f2a zero size 2024-09-16T08:57:13.142 INFO:tasks.workunit.client.1.smithi097.stdout:4/579: mknod d0/d35/da4/dc5/cc8 0 2024-09-16T08:57:13.142 INFO:tasks.workunit.client.1.smithi097.stdout:4/580: truncate d0/d7/df/f8f 1058885 0 2024-09-16T08:57:13.143 INFO:tasks.workunit.client.1.smithi097.stdout:2/592: mkdir d3/d4/d68/d8b/d61/dac 0 2024-09-16T08:57:13.144 INFO:tasks.workunit.client.1.smithi097.stdout:2/593: creat d3/d4/d8/fad x:0 0 0 2024-09-16T08:57:13.145 INFO:tasks.workunit.client.0.smithi031.stdout:0/198: mkdir d6/d12/d47 0 2024-09-16T08:57:13.150 INFO:tasks.workunit.client.0.smithi031.stdout:3/223: write d3/d14/d37/d33/f38 [2757129,1367] 0 2024-09-16T08:57:13.150 INFO:tasks.workunit.client.0.smithi031.stdout:0/199: unlink d6/d12/f2a 0 2024-09-16T08:57:13.152 INFO:tasks.workunit.client.1.smithi097.stdout:4/581: creat d0/d23/d5f/fc9 x:0 0 0 2024-09-16T08:57:13.154 INFO:tasks.workunit.client.1.smithi097.stdout:7/534: mknod d4/d12/d1f/cb4 0 2024-09-16T08:57:13.155 INFO:tasks.workunit.client.0.smithi031.stdout:3/224: symlink d3/d14/d37/d1a/d1d/d1e/l3e 0 2024-09-16T08:57:13.155 INFO:tasks.workunit.client.1.smithi097.stdout:1/632: sync 2024-09-16T08:57:13.156 INFO:tasks.workunit.client.1.smithi097.stdout:9/550: dwrite d2/d5/f11 [0,4194304] 0 2024-09-16T08:57:13.157 INFO:tasks.workunit.client.1.smithi097.stdout:4/582: creat d0/d7/d52/d26/d8c/db0/fca x:0 0 0 2024-09-16T08:57:13.157 INFO:tasks.workunit.client.1.smithi097.stdout:2/594: write d3/d4/d9/f36 [1172711,101508] 0 2024-09-16T08:57:13.157 INFO:tasks.workunit.client.1.smithi097.stdout:7/535: truncate d4/d9/d2b/f4d 1778613 0 2024-09-16T08:57:13.158 INFO:tasks.workunit.client.1.smithi097.stdout:7/536: chown d4/d7/d7b/d43/l87 0 1 2024-09-16T08:57:13.158 INFO:tasks.workunit.client.1.smithi097.stdout:7/537: write d4/f5 [5101083,59710] 0 2024-09-16T08:57:13.158 INFO:tasks.workunit.client.0.smithi031.stdout:1/173: dwrite d2/de/f2a [4194304,4194304] 0 2024-09-16T08:57:13.158 INFO:tasks.workunit.client.0.smithi031.stdout:1/174: chown d2/d4/f30 236854 1 2024-09-16T08:57:13.159 INFO:tasks.workunit.client.0.smithi031.stdout:1/175: dread - d2/f2c zero size 2024-09-16T08:57:13.159 INFO:tasks.workunit.client.0.smithi031.stdout:3/225: truncate d3/d14/d37/d1a/d1d/f2d 6256249 0 2024-09-16T08:57:13.159 INFO:tasks.workunit.client.1.smithi097.stdout:1/633: mkdir d5/db/dd4 0 2024-09-16T08:57:13.159 INFO:tasks.workunit.client.1.smithi097.stdout:1/634: dread - d5/d2c/fa6 zero size 2024-09-16T08:57:13.161 INFO:tasks.workunit.client.1.smithi097.stdout:2/595: dread d3/d4/d68/d8b/d7c/d7d/f8d [0,4194304] 0 2024-09-16T08:57:13.161 INFO:tasks.workunit.client.1.smithi097.stdout:2/596: creat d3/fae x:0 0 0 2024-09-16T08:57:13.163 INFO:tasks.workunit.client.1.smithi097.stdout:6/591: sync 2024-09-16T08:57:13.163 INFO:tasks.workunit.client.1.smithi097.stdout:3/606: sync 2024-09-16T08:57:13.163 INFO:tasks.workunit.client.1.smithi097.stdout:5/644: sync 2024-09-16T08:57:13.163 INFO:tasks.workunit.client.1.smithi097.stdout:3/607: fdatasync d7/de/d3c/f4d 0 2024-09-16T08:57:13.163 INFO:tasks.workunit.client.1.smithi097.stdout:3/608: chown d7/de/d29/d35/d3d/fb7 13047 1 2024-09-16T08:57:13.163 INFO:tasks.workunit.client.1.smithi097.stdout:1/635: mknod d5/d17/d54/d6b/d71/d94/cd5 0 2024-09-16T08:57:13.163 INFO:tasks.workunit.client.1.smithi097.stdout:6/592: write d1/d3/d81/dad/f62 [1665139,53107] 0 2024-09-16T08:57:13.164 INFO:tasks.workunit.client.1.smithi097.stdout:1/636: creat d5/d17/d54/d6b/d71/d94/fd6 x:0 0 0 2024-09-16T08:57:13.164 INFO:tasks.workunit.client.1.smithi097.stdout:1/637: fsync d5/d70/d9d/fbd 0 2024-09-16T08:57:13.164 INFO:tasks.workunit.client.1.smithi097.stdout:8/576: dwrite d8/fad [0,4194304] 0 2024-09-16T08:57:13.165 INFO:tasks.workunit.client.0.smithi031.stdout:1/176: getdents d2/d4/d1b 0 2024-09-16T08:57:13.165 INFO:tasks.workunit.client.0.smithi031.stdout:1/177: write d2/f24 [64295,14306] 0 2024-09-16T08:57:13.169 INFO:tasks.workunit.client.1.smithi097.stdout:8/577: dread d8/dd/d15/d2c/d4a/d4d/d62/f76 [0,4194304] 0 2024-09-16T08:57:13.169 INFO:tasks.workunit.client.1.smithi097.stdout:2/597: mknod d3/d4/d68/d8b/d7c/d7d/d53/da3/caf 0 2024-09-16T08:57:13.171 INFO:tasks.workunit.client.0.smithi031.stdout:6/233: dwrite d0/d17/d19/f34 [0,4194304] 0 2024-09-16T08:57:13.171 INFO:tasks.workunit.client.0.smithi031.stdout:3/226: fdatasync d3/fb 0 2024-09-16T08:57:13.171 INFO:tasks.workunit.client.0.smithi031.stdout:3/227: dread - d3/d14/d37/d33/f3c zero size 2024-09-16T08:57:13.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/228: chown d3/d14/d37/d33/c12 2537 1 2024-09-16T08:57:13.173 INFO:tasks.workunit.client.1.smithi097.stdout:0/596: dwrite da/d24/d76/f8b [0,4194304] 0 2024-09-16T08:57:13.173 INFO:tasks.workunit.client.1.smithi097.stdout:0/597: write da/d1a/d26/d47/db3/d60/d73/f77 [4949896,123870] 0 2024-09-16T08:57:13.174 INFO:tasks.workunit.client.1.smithi097.stdout:3/609: mkdir d7/de/d9c/dca 0 2024-09-16T08:57:13.177 INFO:tasks.workunit.client.0.smithi031.stdout:8/132: dwrite d4/d7/f1a [0,4194304] 0 2024-09-16T08:57:13.177 INFO:tasks.workunit.client.1.smithi097.stdout:4/583: write d0/fe [1592360,119477] 0 2024-09-16T08:57:13.178 INFO:tasks.workunit.client.1.smithi097.stdout:1/638: rename d5/db/dc/d19/f38 to d5/d70/d9d/fd7 0 2024-09-16T08:57:13.179 INFO:tasks.workunit.client.1.smithi097.stdout:5/645: write db/f2e [4141187,70989] 0 2024-09-16T08:57:13.181 INFO:tasks.workunit.client.0.smithi031.stdout:1/178: rmdir d2 39 2024-09-16T08:57:13.182 INFO:tasks.workunit.client.1.smithi097.stdout:9/551: dread d2/f31 [0,4194304] 0 2024-09-16T08:57:13.185 INFO:tasks.workunit.client.1.smithi097.stdout:8/578: mkdir d8/dd/d15/d2c/d4a/d4e/db2 0 2024-09-16T08:57:13.185 INFO:tasks.workunit.client.1.smithi097.stdout:2/598: dread d3/d4/d9/f9c [0,4194304] 0 2024-09-16T08:57:13.185 INFO:tasks.workunit.client.1.smithi097.stdout:2/599: readlink d3/d4/d8/l89 0 2024-09-16T08:57:13.186 INFO:tasks.workunit.client.1.smithi097.stdout:7/538: dwrite d4/f9d [0,4194304] 0 2024-09-16T08:57:13.194 INFO:tasks.workunit.client.0.smithi031.stdout:3/229: mkdir d3/d14/d37/d1a/d1d/d3b/d3f 0 2024-09-16T08:57:13.194 INFO:tasks.workunit.client.1.smithi097.stdout:2/600: read d3/d4/d68/d8b/d7c/d7d/f8c [2636038,3977] 0 2024-09-16T08:57:13.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/133: creat d4/f27 x:0 0 0 2024-09-16T08:57:13.196 INFO:tasks.workunit.client.0.smithi031.stdout:8/134: chown d4/f18 69 1 2024-09-16T08:57:13.199 INFO:tasks.workunit.client.1.smithi097.stdout:1/639: truncate d5/d2c/d3b/f9a 1896456 0 2024-09-16T08:57:13.200 INFO:tasks.workunit.client.0.smithi031.stdout:3/230: creat d3/d14/d37/d1a/d24/f40 x:0 0 0 2024-09-16T08:57:13.201 INFO:tasks.workunit.client.0.smithi031.stdout:3/231: read d3/d14/f2f [263214,96965] 0 2024-09-16T08:57:13.201 INFO:tasks.workunit.client.1.smithi097.stdout:4/584: dwrite d0/d36/d4c/fac [4194304,4194304] 0 2024-09-16T08:57:13.202 INFO:tasks.workunit.client.0.smithi031.stdout:8/135: creat d4/d7/f28 x:0 0 0 2024-09-16T08:57:13.202 INFO:tasks.workunit.client.0.smithi031.stdout:8/136: creat d4/f29 x:0 0 0 2024-09-16T08:57:13.203 INFO:tasks.workunit.client.1.smithi097.stdout:5/646: mkdir db/d6e/d79/dd5 0 2024-09-16T08:57:13.204 INFO:tasks.workunit.client.0.smithi031.stdout:1/179: symlink d2/d4/l3a 0 2024-09-16T08:57:13.205 INFO:tasks.workunit.client.0.smithi031.stdout:1/180: truncate d2/f17 662106 0 2024-09-16T08:57:13.205 INFO:tasks.workunit.client.0.smithi031.stdout:1/181: readlink d2/l2d 0 2024-09-16T08:57:13.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/234: dwrite d0/dd/d2b/f33 [0,4194304] 0 2024-09-16T08:57:13.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/235: fdatasync d0/d17/d19/f2e 0 2024-09-16T08:57:13.206 INFO:tasks.workunit.client.0.smithi031.stdout:3/232: mknod d3/d14/d37/d1a/d1d/d3b/d3f/c41 0 2024-09-16T08:57:13.208 INFO:tasks.workunit.client.1.smithi097.stdout:9/552: mkdir d2/d30/d74/d8c/dad 0 2024-09-16T08:57:13.211 INFO:tasks.workunit.client.1.smithi097.stdout:8/579: link d8/dd/d15/d2c/d48/d56/f6d d8/dd/d15/d2c/d4a/d4e/db2/fb3 0 2024-09-16T08:57:13.211 INFO:tasks.workunit.client.1.smithi097.stdout:8/580: chown d8/dd/d15/d16/fa1 25 1 2024-09-16T08:57:13.211 INFO:tasks.workunit.client.1.smithi097.stdout:8/581: readlink d8/dd/d15/d2c/l3b 0 2024-09-16T08:57:13.212 INFO:tasks.workunit.client.0.smithi031.stdout:8/137: creat d4/d7/d21/d24/f2a x:0 0 0 2024-09-16T08:57:13.217 INFO:tasks.workunit.client.1.smithi097.stdout:7/539: rename d4/d1b/f35 to d4/d1b/fb5 0 2024-09-16T08:57:13.218 INFO:tasks.workunit.client.1.smithi097.stdout:1/640: read d5/d2c/dcb/f53 [773651,24854] 0 2024-09-16T08:57:13.218 INFO:tasks.workunit.client.1.smithi097.stdout:1/641: creat d5/d14/d95/fd8 x:0 0 0 2024-09-16T08:57:13.218 INFO:tasks.workunit.client.1.smithi097.stdout:9/553: write d2/d5/d56/f88 [905592,95343] 0 2024-09-16T08:57:13.218 INFO:tasks.workunit.client.1.smithi097.stdout:9/554: chown d2 2 1 2024-09-16T08:57:13.219 INFO:tasks.workunit.client.1.smithi097.stdout:7/540: write d4/d7/d77/f9e [179124,103254] 0 2024-09-16T08:57:13.221 INFO:tasks.workunit.client.0.smithi031.stdout:6/236: link d0/d17/f18 d0/dd/f41 0 2024-09-16T08:57:13.221 INFO:tasks.workunit.client.0.smithi031.stdout:6/237: dread - d0/d10/d14/f24 zero size 2024-09-16T08:57:13.221 INFO:tasks.workunit.client.0.smithi031.stdout:6/238: creat d0/d17/f42 x:0 0 0 2024-09-16T08:57:13.225 INFO:tasks.workunit.client.0.smithi031.stdout:8/138: fsync d4/f27 0 2024-09-16T08:57:13.228 INFO:tasks.workunit.client.1.smithi097.stdout:2/601: symlink d3/d4/d68/d8b/d7c/lb0 0 2024-09-16T08:57:13.232 INFO:tasks.workunit.client.1.smithi097.stdout:1/642: read d5/d2c/f9b [1814049,40142] 0 2024-09-16T08:57:13.232 INFO:tasks.workunit.client.1.smithi097.stdout:1/643: creat d5/fd9 x:0 0 0 2024-09-16T08:57:13.232 INFO:tasks.workunit.client.1.smithi097.stdout:1/644: truncate d5/db/dc/d19/f88 1226824 0 2024-09-16T08:57:13.232 INFO:tasks.workunit.client.1.smithi097.stdout:1/645: chown d5/d70/d85/db5/lc6 520 1 2024-09-16T08:57:13.232 INFO:tasks.workunit.client.1.smithi097.stdout:1/646: truncate d5/d17/d54/d6b/f9c 724146 0 2024-09-16T08:57:13.232 INFO:tasks.workunit.client.1.smithi097.stdout:1/647: readlink d5/l3e 0 2024-09-16T08:57:13.233 INFO:tasks.workunit.client.1.smithi097.stdout:1/648: stat d5/db/dc/c83 0 2024-09-16T08:57:13.233 INFO:tasks.workunit.client.0.smithi031.stdout:1/182: dwrite d2/de/f2f [0,4194304] 0 2024-09-16T08:57:13.234 INFO:tasks.workunit.client.0.smithi031.stdout:8/139: getdents d4/d7 0 2024-09-16T08:57:13.234 INFO:tasks.workunit.client.0.smithi031.stdout:8/140: read - d4/f27 zero size 2024-09-16T08:57:13.236 INFO:tasks.workunit.client.1.smithi097.stdout:1/649: dread d5/db/dc/d19/f4d [0,4194304] 0 2024-09-16T08:57:13.236 INFO:tasks.workunit.client.1.smithi097.stdout:1/650: creat d5/db/dc/fda x:0 0 0 2024-09-16T08:57:13.237 INFO:tasks.workunit.client.1.smithi097.stdout:1/651: truncate d5/d2c/f5e 1381132 0 2024-09-16T08:57:13.237 INFO:tasks.workunit.client.1.smithi097.stdout:1/652: chown d5/d2c/dcb/f45 71629 1 2024-09-16T08:57:13.237 INFO:tasks.workunit.client.1.smithi097.stdout:1/653: dread - d5/d2c/f92 zero size 2024-09-16T08:57:13.237 INFO:tasks.workunit.client.1.smithi097.stdout:1/654: chown d5/d17/d54/d6b/da8/dc5 540238 1 2024-09-16T08:57:13.237 INFO:tasks.workunit.client.1.smithi097.stdout:1/655: dread - d5/d2c/d3b/fb7 zero size 2024-09-16T08:57:13.237 INFO:tasks.workunit.client.1.smithi097.stdout:1/656: write d5/db/dc/f41 [4431244,121250] 0 2024-09-16T08:57:13.239 INFO:tasks.workunit.client.0.smithi031.stdout:1/183: creat d2/d4/d1b/f3b x:0 0 0 2024-09-16T08:57:13.239 INFO:tasks.workunit.client.0.smithi031.stdout:1/184: readlink d2/l2d 0 2024-09-16T08:57:13.240 INFO:tasks.workunit.client.0.smithi031.stdout:6/239: dwrite d0/d17/f1e [0,4194304] 0 2024-09-16T08:57:13.240 INFO:tasks.workunit.client.0.smithi031.stdout:6/240: creat d0/dd/f43 x:0 0 0 2024-09-16T08:57:13.241 INFO:tasks.workunit.client.0.smithi031.stdout:8/141: mknod d4/c2b 0 2024-09-16T08:57:13.247 INFO:tasks.workunit.client.1.smithi097.stdout:4/585: mknod d0/ccb 0 2024-09-16T08:57:13.247 INFO:tasks.workunit.client.1.smithi097.stdout:4/586: readlink d0/d7/l61 0 2024-09-16T08:57:13.249 INFO:tasks.workunit.client.1.smithi097.stdout:5/647: mknod db/d1a/cd6 0 2024-09-16T08:57:13.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/185: dread d2/de/f2f [0,4194304] 0 2024-09-16T08:57:13.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/186: fsync f0 0 2024-09-16T08:57:13.251 INFO:tasks.workunit.client.0.smithi031.stdout:6/241: rename d0/d17/d19/l3c to d0/d10/l44 0 2024-09-16T08:57:13.251 INFO:tasks.workunit.client.0.smithi031.stdout:6/242: truncate d0/d17/d19/f32 86536 0 2024-09-16T08:57:13.253 INFO:tasks.workunit.client.0.smithi031.stdout:1/187: creat d2/f3c x:0 0 0 2024-09-16T08:57:13.255 INFO:tasks.workunit.client.1.smithi097.stdout:1/657: dwrite d5/d17/d54/d6b/d71/d94/f99 [4194304,4194304] 0 2024-09-16T08:57:13.255 INFO:tasks.workunit.client.0.smithi031.stdout:6/243: creat d0/dd/d2b/f45 x:0 0 0 2024-09-16T08:57:13.255 INFO:tasks.workunit.client.0.smithi031.stdout:6/244: stat d0/d17 0 2024-09-16T08:57:13.256 INFO:tasks.workunit.client.0.smithi031.stdout:6/245: getdents d0/dd/d35 0 2024-09-16T08:57:13.256 INFO:tasks.workunit.client.0.smithi031.stdout:6/246: fsync d0/dd/f23 0 2024-09-16T08:57:13.256 INFO:tasks.workunit.client.0.smithi031.stdout:6/247: stat d0/dd/f43 0 2024-09-16T08:57:13.259 INFO:tasks.workunit.client.1.smithi097.stdout:1/658: dread d5/db/f3c [0,4194304] 0 2024-09-16T08:57:13.259 INFO:tasks.workunit.client.0.smithi031.stdout:1/188: creat d2/d4/d22/f3d x:0 0 0 2024-09-16T08:57:13.259 INFO:tasks.workunit.client.0.smithi031.stdout:1/189: chown d2/d4/f28 0 1 2024-09-16T08:57:13.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/190: truncate d2/de/f13 4752078 0 2024-09-16T08:57:13.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/191: read d2/f14 [6709190,44926] 0 2024-09-16T08:57:13.260 INFO:tasks.workunit.client.0.smithi031.stdout:8/142: dread d4/d7/f1a [0,4194304] 0 2024-09-16T08:57:13.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/192: creat d2/f3e x:0 0 0 2024-09-16T08:57:13.261 INFO:tasks.workunit.client.1.smithi097.stdout:9/555: dwrite d2/f31 [4194304,4194304] 0 2024-09-16T08:57:13.261 INFO:tasks.workunit.client.1.smithi097.stdout:7/541: creat d4/d12/d1f/fb6 x:0 0 0 2024-09-16T08:57:13.261 INFO:tasks.workunit.client.1.smithi097.stdout:7/542: chown d4/d12/d1f/d22/f3d 9317864 1 2024-09-16T08:57:13.261 INFO:tasks.workunit.client.1.smithi097.stdout:7/543: dread - d4/d9/d2b/fa9 zero size 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.0.smithi031.stdout:6/248: mknod d0/c46 0 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.0.smithi031.stdout:6/249: truncate d0/d10/d14/f24 217972 0 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.0.smithi031.stdout:6/250: write d0/dd/f3e [1094703,70404] 0 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.0.smithi031.stdout:6/251: chown d0/d17/d19/f2e 56 1 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.1.smithi097.stdout:8/582: dread d8/dd/f2f [4194304,4194304] 0 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.1.smithi097.stdout:8/583: write d8/dd/d15/d2c/d4a/d4d/fa5 [388590,108423] 0 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.1.smithi097.stdout:2/602: truncate d3/d4/d8/f16 3465530 0 2024-09-16T08:57:13.263 INFO:tasks.workunit.client.1.smithi097.stdout:2/603: write d3/d4/d8/f2b [1624775,83484] 0 2024-09-16T08:57:13.264 INFO:tasks.workunit.client.0.smithi031.stdout:8/143: mknod d4/d7/d21/d24/c2c 0 2024-09-16T08:57:13.268 INFO:tasks.workunit.client.0.smithi031.stdout:1/193: mknod d2/de/c3f 0 2024-09-16T08:57:13.270 INFO:tasks.workunit.client.1.smithi097.stdout:3/610: rename d7/de/d29 to d7/de/d3c/d3f/d60/dcb 0 2024-09-16T08:57:13.270 INFO:tasks.workunit.client.1.smithi097.stdout:3/611: fdatasync d7/de/f23 0 2024-09-16T08:57:13.271 INFO:tasks.workunit.client.0.smithi031.stdout:3/233: truncate d3/d14/d37/d1a/d1d/f34 1324757 0 2024-09-16T08:57:13.272 INFO:tasks.workunit.client.0.smithi031.stdout:5/230: sync 2024-09-16T08:57:13.273 INFO:tasks.workunit.client.1.smithi097.stdout:4/587: mkdir d0/d23/d5f/d7d/dbc/dcc 0 2024-09-16T08:57:13.275 INFO:tasks.workunit.client.1.smithi097.stdout:4/588: read d0/d7/df/d83/fa5 [359628,91672] 0 2024-09-16T08:57:13.275 INFO:tasks.workunit.client.1.smithi097.stdout:4/589: creat d0/d39/fcd x:0 0 0 2024-09-16T08:57:13.277 INFO:tasks.workunit.client.1.smithi097.stdout:1/659: truncate f4 2656125 0 2024-09-16T08:57:13.277 INFO:tasks.workunit.client.1.smithi097.stdout:1/660: readlink d5/db/dc/d19/l4f 0 2024-09-16T08:57:13.277 INFO:tasks.workunit.client.0.smithi031.stdout:5/231: creat d2/dc/f44 x:0 0 0 2024-09-16T08:57:13.277 INFO:tasks.workunit.client.0.smithi031.stdout:5/232: chown d2/dc/d3d/l3e 620495 1 2024-09-16T08:57:13.277 INFO:tasks.workunit.client.0.smithi031.stdout:5/233: truncate d2/dc/f1f 4590846 0 2024-09-16T08:57:13.278 INFO:tasks.workunit.client.1.smithi097.stdout:7/544: write d4/d12/d1f/f21 [3590329,55114] 0 2024-09-16T08:57:13.278 INFO:tasks.workunit.client.1.smithi097.stdout:5/648: dwrite db/d18/d1c/d42/d46/f9b [0,4194304] 0 2024-09-16T08:57:13.280 INFO:tasks.workunit.client.0.smithi031.stdout:5/234: creat d2/dc/d25/d21/f45 x:0 0 0 2024-09-16T08:57:13.280 INFO:tasks.workunit.client.0.smithi031.stdout:5/235: creat d2/dc/d25/d21/f46 x:0 0 0 2024-09-16T08:57:13.281 INFO:tasks.workunit.client.0.smithi031.stdout:5/236: fdatasync d2/dc/f3b 0 2024-09-16T08:57:13.281 INFO:tasks.workunit.client.0.smithi031.stdout:5/237: fsync d2/f34 0 2024-09-16T08:57:13.281 INFO:tasks.workunit.client.0.smithi031.stdout:5/238: chown d2/dc/d25/d21/d2e/f33 11 1 2024-09-16T08:57:13.281 INFO:tasks.workunit.client.0.smithi031.stdout:6/252: dwrite d0/dd/f2f [0,4194304] 0 2024-09-16T08:57:13.290 INFO:tasks.workunit.client.1.smithi097.stdout:9/556: symlink d2/d5/d3a/d3c/lae 0 2024-09-16T08:57:13.292 INFO:tasks.workunit.client.0.smithi031.stdout:6/253: dread d0/fb [0,4194304] 0 2024-09-16T08:57:13.293 INFO:tasks.workunit.client.0.smithi031.stdout:1/194: dwrite d2/d4/f30 [0,4194304] 0 2024-09-16T08:57:13.293 INFO:tasks.workunit.client.0.smithi031.stdout:1/195: dread - d2/d4/f31 zero size 2024-09-16T08:57:13.293 INFO:tasks.workunit.client.1.smithi097.stdout:2/604: rmdir d3/d4/d8/dab 0 2024-09-16T08:57:13.293 INFO:tasks.workunit.client.1.smithi097.stdout:2/605: write d3/d4/d68/d8b/d7c/d7d/d53/d47/f49 [3058916,75857] 0 2024-09-16T08:57:13.293 INFO:tasks.workunit.client.1.smithi097.stdout:2/606: read d3/d4/d8/f2b [1919783,15181] 0 2024-09-16T08:57:13.294 INFO:tasks.workunit.client.1.smithi097.stdout:8/584: rename d8/dd/d15/d2c/d4a/d4d/d3f/l40 to d8/dd/d15/d16/d7c/lb4 0 2024-09-16T08:57:13.296 INFO:tasks.workunit.client.1.smithi097.stdout:3/612: mkdir d7/de/d3c/dcc 0 2024-09-16T08:57:13.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/144: rmdir d4/d7/d21 39 2024-09-16T08:57:13.297 INFO:tasks.workunit.client.0.smithi031.stdout:8/145: write d4/f18 [537245,129741] 0 2024-09-16T08:57:13.297 INFO:tasks.workunit.client.1.smithi097.stdout:5/649: dwrite fa [0,4194304] 0 2024-09-16T08:57:13.298 INFO:tasks.workunit.client.1.smithi097.stdout:5/650: dread - db/d6e/fd4 zero size 2024-09-16T08:57:13.300 INFO:tasks.workunit.client.1.smithi097.stdout:4/590: mkdir d0/d7/dce 0 2024-09-16T08:57:13.302 INFO:tasks.workunit.client.1.smithi097.stdout:1/661: creat d5/d2c/d3b/d7f/fdb x:0 0 0 2024-09-16T08:57:13.303 INFO:tasks.workunit.client.1.smithi097.stdout:9/557: read d2/d3f/f43 [3294121,95922] 0 2024-09-16T08:57:13.303 INFO:tasks.workunit.client.1.smithi097.stdout:9/558: chown d2/d5/d72/f89 2190467 1 2024-09-16T08:57:13.303 INFO:tasks.workunit.client.1.smithi097.stdout:9/559: creat d2/d5/d56/faf x:0 0 0 2024-09-16T08:57:13.304 INFO:tasks.workunit.client.0.smithi031.stdout:8/146: write d4/d7/f15 [2520624,61215] 0 2024-09-16T08:57:13.304 INFO:tasks.workunit.client.0.smithi031.stdout:3/234: fsync d3/d14/d37/d1a/d1d/f2d 0 2024-09-16T08:57:13.304 INFO:tasks.workunit.client.0.smithi031.stdout:3/235: stat d3/c7 0 2024-09-16T08:57:13.304 INFO:tasks.workunit.client.0.smithi031.stdout:3/236: stat d3/d14/d37/d1a/d1d/d1e 0 2024-09-16T08:57:13.304 INFO:tasks.workunit.client.0.smithi031.stdout:3/237: fsync d3/fb 0 2024-09-16T08:57:13.304 INFO:tasks.workunit.client.0.smithi031.stdout:3/238: read - d3/d14/d37/d33/f3c zero size 2024-09-16T08:57:13.305 INFO:tasks.workunit.client.1.smithi097.stdout:7/545: mkdir d4/d9/d51/db7 0 2024-09-16T08:57:13.311 INFO:tasks.workunit.client.1.smithi097.stdout:2/607: truncate d3/d4/d68/d8b/f9d 908235 0 2024-09-16T08:57:13.311 INFO:tasks.workunit.client.1.smithi097.stdout:2/608: write d3/f71 [873150,87007] 0 2024-09-16T08:57:13.312 INFO:tasks.workunit.client.1.smithi097.stdout:8/585: getdents d8/dd/d15/d2c/d4a/d4d/d3d/d7b 0 2024-09-16T08:57:13.314 INFO:tasks.workunit.client.0.smithi031.stdout:6/254: rename d0/d10/f11 to d0/d10/d14/f47 0 2024-09-16T08:57:13.315 INFO:tasks.workunit.client.1.smithi097.stdout:3/613: creat d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa/fcd x:0 0 0 2024-09-16T08:57:13.316 INFO:tasks.workunit.client.1.smithi097.stdout:3/614: write d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/f78 [675995,96905] 0 2024-09-16T08:57:13.316 INFO:tasks.workunit.client.1.smithi097.stdout:3/615: creat d7/de/d3c/dbe/fce x:0 0 0 2024-09-16T08:57:13.316 INFO:tasks.workunit.client.0.smithi031.stdout:5/239: rmdir d2/dc 39 2024-09-16T08:57:13.316 INFO:tasks.workunit.client.0.smithi031.stdout:5/240: dread - d2/dc/d1d/f3f zero size 2024-09-16T08:57:13.316 INFO:tasks.workunit.client.0.smithi031.stdout:1/196: mknod d2/c40 0 2024-09-16T08:57:13.317 INFO:tasks.workunit.client.1.smithi097.stdout:7/546: dread d4/d7/f11 [0,4194304] 0 2024-09-16T08:57:13.317 INFO:tasks.workunit.client.1.smithi097.stdout:5/651: creat db/d18/d1c/d42/fd7 x:0 0 0 2024-09-16T08:57:13.318 INFO:tasks.workunit.client.1.smithi097.stdout:5/652: truncate db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fb6 140837 0 2024-09-16T08:57:13.318 INFO:tasks.workunit.client.1.smithi097.stdout:5/653: dread - db/d1a/f86 zero size 2024-09-16T08:57:13.318 INFO:tasks.workunit.client.1.smithi097.stdout:5/654: creat db/d18/d1c/d42/d46/d7e/d8f/fd8 x:0 0 0 2024-09-16T08:57:13.318 INFO:tasks.workunit.client.1.smithi097.stdout:5/655: dread - db/d18/d1c/d42/d46/d7e/fb8 zero size 2024-09-16T08:57:13.319 INFO:tasks.workunit.client.0.smithi031.stdout:8/147: mkdir d4/d2d 0 2024-09-16T08:57:13.322 INFO:tasks.workunit.client.1.smithi097.stdout:4/591: rename d0/d39/d71/f72 to d0/d23/d5f/d7d/fcf 0 2024-09-16T08:57:13.325 INFO:tasks.workunit.client.1.smithi097.stdout:1/662: truncate d5/db/f34 6314394 0 2024-09-16T08:57:13.325 INFO:tasks.workunit.client.1.smithi097.stdout:1/663: chown d5/l27 138632 1 2024-09-16T08:57:13.327 INFO:tasks.workunit.client.1.smithi097.stdout:6/593: sync 2024-09-16T08:57:13.328 INFO:tasks.workunit.client.1.smithi097.stdout:0/598: sync 2024-09-16T08:57:13.328 INFO:tasks.workunit.client.1.smithi097.stdout:0/599: dread da/d24/d4c/d6d/f6e [0,4194304] 0 2024-09-16T08:57:13.333 INFO:tasks.workunit.client.1.smithi097.stdout:9/560: mkdir d2/d5/db0 0 2024-09-16T08:57:13.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/239: mknod d3/d14/d37/d1a/d1d/d1e/c42 0 2024-09-16T08:57:13.337 INFO:tasks.workunit.client.0.smithi031.stdout:3/240: dread - d3/f2a zero size 2024-09-16T08:57:13.338 INFO:tasks.workunit.client.1.smithi097.stdout:8/586: mknod d8/dd/d15/d16/d7c/cb5 0 2024-09-16T08:57:13.339 INFO:tasks.workunit.client.0.smithi031.stdout:1/197: dwrite d2/d4/f30 [0,4194304] 0 2024-09-16T08:57:13.340 INFO:tasks.workunit.client.0.smithi031.stdout:6/255: link d0/dd/f43 d0/d10/d14/d3f/f48 0 2024-09-16T08:57:13.342 INFO:tasks.workunit.client.1.smithi097.stdout:3/616: mkdir d7/de/d6c/dcf 0 2024-09-16T08:57:13.343 INFO:tasks.workunit.client.0.smithi031.stdout:5/241: mknod d2/dc/c47 0 2024-09-16T08:57:13.343 INFO:tasks.workunit.client.0.smithi031.stdout:5/242: fsync d2/dc/d1d/f3f 0 2024-09-16T08:57:13.344 INFO:tasks.workunit.client.0.smithi031.stdout:5/243: read d2/f13 [591964,71517] 0 2024-09-16T08:57:13.345 INFO:tasks.workunit.client.0.smithi031.stdout:8/148: symlink d4/d2d/l2e 0 2024-09-16T08:57:13.345 INFO:tasks.workunit.client.0.smithi031.stdout:8/149: chown d4/f29 71067 1 2024-09-16T08:57:13.347 INFO:tasks.workunit.client.0.smithi031.stdout:3/241: link d3/d14/d37/d1a/f28 d3/d14/d37/d1a/d1d/d3b/f43 0 2024-09-16T08:57:13.347 INFO:tasks.workunit.client.0.smithi031.stdout:3/242: fdatasync d3/d14/d37/d1a/d1d/d3b/f43 0 2024-09-16T08:57:13.347 INFO:tasks.workunit.client.0.smithi031.stdout:3/243: read d3/d14/f2f [652375,84262] 0 2024-09-16T08:57:13.347 INFO:tasks.workunit.client.0.smithi031.stdout:3/244: chown d3/d14 131821512 1 2024-09-16T08:57:13.347 INFO:tasks.workunit.client.0.smithi031.stdout:3/245: chown d3/d14/d37/d1a/f28 0 1 2024-09-16T08:57:13.347 INFO:tasks.workunit.client.0.smithi031.stdout:3/246: truncate d3/d14/f2e 810919 0 2024-09-16T08:57:13.350 INFO:tasks.workunit.client.1.smithi097.stdout:1/664: creat d5/d2c/fdc x:0 0 0 2024-09-16T08:57:13.351 INFO:tasks.workunit.client.0.smithi031.stdout:7/273: sync 2024-09-16T08:57:13.352 INFO:tasks.workunit.client.1.smithi097.stdout:6/594: symlink d1/dc/d1e/db0/lc3 0 2024-09-16T08:57:13.353 INFO:tasks.workunit.client.1.smithi097.stdout:7/547: dwrite d4/d7/d7b/f7c [0,4194304] 0 2024-09-16T08:57:13.353 INFO:tasks.workunit.client.0.smithi031.stdout:2/128: sync 2024-09-16T08:57:13.353 INFO:tasks.workunit.client.0.smithi031.stdout:2/129: creat d6/d20/f2a x:0 0 0 2024-09-16T08:57:13.353 INFO:tasks.workunit.client.1.smithi097.stdout:5/656: dwrite db/d18/d1c/d42/d46/d7e/f53 [0,4194304] 0 2024-09-16T08:57:13.354 INFO:tasks.workunit.client.0.smithi031.stdout:6/256: symlink d0/dd/d35/l49 0 2024-09-16T08:57:13.354 INFO:tasks.workunit.client.0.smithi031.stdout:6/257: dread d0/d10/d14/f24 [0,4194304] 0 2024-09-16T08:57:13.355 INFO:tasks.workunit.client.0.smithi031.stdout:6/258: stat d0/d17/d19/f26 0 2024-09-16T08:57:13.356 INFO:tasks.workunit.client.1.smithi097.stdout:0/600: symlink da/d1a/d41/ld9 0 2024-09-16T08:57:13.357 INFO:tasks.workunit.client.1.smithi097.stdout:1/665: dread d5/f77 [0,4194304] 0 2024-09-16T08:57:13.357 INFO:tasks.workunit.client.0.smithi031.stdout:5/244: creat d2/dc/d25/d21/d2e/f48 x:0 0 0 2024-09-16T08:57:13.359 INFO:tasks.workunit.client.1.smithi097.stdout:7/548: dread d4/d12/f4f [0,4194304] 0 2024-09-16T08:57:13.361 INFO:tasks.workunit.client.0.smithi031.stdout:8/150: mkdir d4/d2f 0 2024-09-16T08:57:13.361 INFO:tasks.workunit.client.0.smithi031.stdout:8/151: chown d4/f27 1229 1 2024-09-16T08:57:13.362 INFO:tasks.workunit.client.1.smithi097.stdout:9/561: mkdir d2/d5/d8d/db1 0 2024-09-16T08:57:13.364 INFO:tasks.workunit.client.0.smithi031.stdout:7/274: truncate d2/d8/d11/f31 2488294 0 2024-09-16T08:57:13.366 INFO:tasks.workunit.client.0.smithi031.stdout:1/198: symlink d2/d4/l41 0 2024-09-16T08:57:13.367 INFO:tasks.workunit.client.1.smithi097.stdout:7/549: dread d4/d7/d7b/f8f [0,4194304] 0 2024-09-16T08:57:13.368 INFO:tasks.workunit.client.1.smithi097.stdout:8/587: symlink d8/dd/d15/d2c/d4a/d4e/d97/lb6 0 2024-09-16T08:57:13.369 INFO:tasks.workunit.client.1.smithi097.stdout:8/588: dread d8/dd/d15/d2c/d4a/d4d/d62/f76 [0,4194304] 0 2024-09-16T08:57:13.369 INFO:tasks.workunit.client.0.smithi031.stdout:4/98: sync 2024-09-16T08:57:13.369 INFO:tasks.workunit.client.0.smithi031.stdout:0/200: sync 2024-09-16T08:57:13.369 INFO:tasks.workunit.client.0.smithi031.stdout:4/99: truncate d0/f18 486017 0 2024-09-16T08:57:13.369 INFO:tasks.workunit.client.0.smithi031.stdout:4/100: chown d0/d2/cf 15415 1 2024-09-16T08:57:13.370 INFO:tasks.workunit.client.0.smithi031.stdout:6/259: dwrite d0/dd/f36 [0,4194304] 0 2024-09-16T08:57:13.370 INFO:tasks.workunit.client.0.smithi031.stdout:6/260: creat d0/dd/d2b/f4a x:0 0 0 2024-09-16T08:57:13.370 INFO:tasks.workunit.client.0.smithi031.stdout:6/261: write d0/dd/f41 [5093220,57995] 0 2024-09-16T08:57:13.371 INFO:tasks.workunit.client.1.smithi097.stdout:8/589: dread d8/dd/d15/d2c/d4a/d4d/f35 [4194304,4194304] 0 2024-09-16T08:57:13.371 INFO:tasks.workunit.client.1.smithi097.stdout:8/590: chown d8/dd/d15/d2c/d4a/d4e/db2 28770536 1 2024-09-16T08:57:13.372 INFO:tasks.workunit.client.0.smithi031.stdout:2/130: creat d6/d20/d23/f2b x:0 0 0 2024-09-16T08:57:13.372 INFO:tasks.workunit.client.0.smithi031.stdout:2/131: chown d6/f16 1 1 2024-09-16T08:57:13.373 INFO:tasks.workunit.client.1.smithi097.stdout:7/550: dread d4/d9/d1c/f28 [0,4194304] 0 2024-09-16T08:57:13.373 INFO:tasks.workunit.client.1.smithi097.stdout:7/551: fdatasync d4/d9/dc/f49 0 2024-09-16T08:57:13.373 INFO:tasks.workunit.client.1.smithi097.stdout:7/552: chown d4/d9/dc/c58 51315 1 2024-09-16T08:57:13.373 INFO:tasks.workunit.client.0.smithi031.stdout:9/156: sync 2024-09-16T08:57:13.374 INFO:tasks.workunit.client.0.smithi031.stdout:9/157: write d3/d5/da/f14 [1028658,105312] 0 2024-09-16T08:57:13.374 INFO:tasks.workunit.client.0.smithi031.stdout:9/158: dread - d3/d5/da/f2c zero size 2024-09-16T08:57:13.374 INFO:tasks.workunit.client.0.smithi031.stdout:9/159: readlink d3/d5/da/l28 0 2024-09-16T08:57:13.374 INFO:tasks.workunit.client.0.smithi031.stdout:5/245: mkdir d2/dc/d25/d21/d49 0 2024-09-16T08:57:13.375 INFO:tasks.workunit.client.1.smithi097.stdout:3/617: symlink d7/de/d3c/d3f/ld0 0 2024-09-16T08:57:13.378 INFO:tasks.workunit.client.0.smithi031.stdout:8/152: symlink d4/l30 0 2024-09-16T08:57:13.380 INFO:tasks.workunit.client.1.smithi097.stdout:6/595: mkdir d1/d3/d81/dc4 0 2024-09-16T08:57:13.382 INFO:tasks.workunit.client.0.smithi031.stdout:7/275: mkdir d2/d8/d61 0 2024-09-16T08:57:13.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/199: symlink d2/d15/l42 0 2024-09-16T08:57:13.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/200: readlink d2/d4/l41 0 2024-09-16T08:57:13.386 INFO:tasks.workunit.client.1.smithi097.stdout:5/657: mknod db/d22/cd9 0 2024-09-16T08:57:13.386 INFO:tasks.workunit.client.1.smithi097.stdout:5/658: truncate db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 243747 0 2024-09-16T08:57:13.386 INFO:tasks.workunit.client.1.smithi097.stdout:5/659: creat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fda x:0 0 0 2024-09-16T08:57:13.386 INFO:tasks.workunit.client.1.smithi097.stdout:5/660: fsync db/d18/d20/fc9 0 2024-09-16T08:57:13.389 INFO:tasks.workunit.client.0.smithi031.stdout:4/101: symlink d0/d2/l26 0 2024-09-16T08:57:13.389 INFO:tasks.workunit.client.0.smithi031.stdout:4/102: write d0/d2/f10 [575790,118622] 0 2024-09-16T08:57:13.389 INFO:tasks.workunit.client.0.smithi031.stdout:4/103: read d0/f18 [60526,24106] 0 2024-09-16T08:57:13.391 INFO:tasks.workunit.client.1.smithi097.stdout:2/609: truncate d3/d4/d9/f4f 2793149 0 2024-09-16T08:57:13.392 INFO:tasks.workunit.client.0.smithi031.stdout:9/160: dwrite d3/d5/da/f2b [4194304,4194304] 0 2024-09-16T08:57:13.393 INFO:tasks.workunit.client.0.smithi031.stdout:9/161: dread f2 [0,4194304] 0 2024-09-16T08:57:13.396 INFO:tasks.workunit.client.1.smithi097.stdout:2/610: dread d3/d4/d68/d8b/d7c/d7d/d53/f6c [0,4194304] 0 2024-09-16T08:57:13.397 INFO:tasks.workunit.client.1.smithi097.stdout:0/601: creat da/df/d8a/fda x:0 0 0 2024-09-16T08:57:13.397 INFO:tasks.workunit.client.0.smithi031.stdout:7/276: dwrite d2/d8/d11/f45 [0,4194304] 0 2024-09-16T08:57:13.397 INFO:tasks.workunit.client.0.smithi031.stdout:7/277: getdents d2/d8/d61 0 2024-09-16T08:57:13.397 INFO:tasks.workunit.client.0.smithi031.stdout:7/278: creat d2/f62 x:0 0 0 2024-09-16T08:57:13.397 INFO:tasks.workunit.client.0.smithi031.stdout:7/279: write d2/d8/f57 [1571513,25405] 0 2024-09-16T08:57:13.398 INFO:tasks.workunit.client.0.smithi031.stdout:7/280: creat d2/d8/f63 x:0 0 0 2024-09-16T08:57:13.398 INFO:tasks.workunit.client.1.smithi097.stdout:1/666: rmdir d5/d70/d85 39 2024-09-16T08:57:13.399 INFO:tasks.workunit.client.1.smithi097.stdout:0/602: dread da/d24/d4c/d80/f2a [0,4194304] 0 2024-09-16T08:57:13.400 INFO:tasks.workunit.client.1.smithi097.stdout:9/562: creat d2/d30/d74/d8c/dad/fb2 x:0 0 0 2024-09-16T08:57:13.401 INFO:tasks.workunit.client.0.smithi031.stdout:5/246: creat d2/dc/d36/f4a x:0 0 0 2024-09-16T08:57:13.403 INFO:tasks.workunit.client.0.smithi031.stdout:8/153: creat d4/d7/d21/d24/f31 x:0 0 0 2024-09-16T08:57:13.403 INFO:tasks.workunit.client.1.smithi097.stdout:8/591: truncate d8/dd/d15/d2c/f74 4364940 0 2024-09-16T08:57:13.405 INFO:tasks.workunit.client.0.smithi031.stdout:1/201: mkdir d2/d15/d1f/d43 0 2024-09-16T08:57:13.405 INFO:tasks.workunit.client.0.smithi031.stdout:1/202: fdatasync d2/f3e 0 2024-09-16T08:57:13.406 INFO:tasks.workunit.client.0.smithi031.stdout:0/201: mknod d6/d12/d13/c48 0 2024-09-16T08:57:13.407 INFO:tasks.workunit.client.1.smithi097.stdout:7/553: symlink d4/d7/d7b/d43/daf/lb8 0 2024-09-16T08:57:13.407 INFO:tasks.workunit.client.1.smithi097.stdout:1/667: dread d5/d2c/f4e [0,4194304] 0 2024-09-16T08:57:13.410 INFO:tasks.workunit.client.1.smithi097.stdout:3/618: mknod d7/de/d3c/d3f/d60/d6b/cd1 0 2024-09-16T08:57:13.414 INFO:tasks.workunit.client.1.smithi097.stdout:7/554: dread d4/d9/d1c/f2c [0,4194304] 0 2024-09-16T08:57:13.414 INFO:tasks.workunit.client.1.smithi097.stdout:6/596: link d1/d3/l97 d1/d3/d16/lc5 0 2024-09-16T08:57:13.418 INFO:tasks.workunit.client.0.smithi031.stdout:9/162: mkdir d3/d5/da/d34 0 2024-09-16T08:57:13.420 INFO:tasks.workunit.client.1.smithi097.stdout:2/611: creat d3/d4/d68/d8b/d61/fb1 x:0 0 0 2024-09-16T08:57:13.421 INFO:tasks.workunit.client.1.smithi097.stdout:7/555: dread d4/f23 [0,4194304] 0 2024-09-16T08:57:13.425 INFO:tasks.workunit.client.1.smithi097.stdout:0/603: symlink da/d24/d4c/d6d/ldb 0 2024-09-16T08:57:13.425 INFO:tasks.workunit.client.1.smithi097.stdout:0/604: write da/d7e/f96 [834434,32861] 0 2024-09-16T08:57:13.427 INFO:tasks.workunit.client.1.smithi097.stdout:0/605: dread da/d24/d4b/fa3 [0,4194304] 0 2024-09-16T08:57:13.428 INFO:tasks.workunit.client.1.smithi097.stdout:9/563: stat d2/d30/l49 0 2024-09-16T08:57:13.431 INFO:tasks.workunit.client.1.smithi097.stdout:8/592: link d8/dd/d15/d2c/d4a/d4d/f5c d8/dd/d15/d16/d7c/fb7 0 2024-09-16T08:57:13.431 INFO:tasks.workunit.client.0.smithi031.stdout:7/281: mknod d2/d8/d11/c64 0 2024-09-16T08:57:13.432 INFO:tasks.workunit.client.0.smithi031.stdout:7/282: chown d2/d8/l2b 4150157 1 2024-09-16T08:57:13.432 INFO:tasks.workunit.client.1.smithi097.stdout:7/556: dread d4/d9/d51/d88/f9b [0,4194304] 0 2024-09-16T08:57:13.432 INFO:tasks.workunit.client.1.smithi097.stdout:7/557: write d4/d9/dc/f93 [493709,68231] 0 2024-09-16T08:57:13.432 INFO:tasks.workunit.client.1.smithi097.stdout:7/558: dread - d4/d12/d1f/d22/f8c zero size 2024-09-16T08:57:13.435 INFO:tasks.workunit.client.1.smithi097.stdout:1/668: mkdir d5/ddd 0 2024-09-16T08:57:13.435 INFO:tasks.workunit.client.1.smithi097.stdout:6/597: dwrite d1/d3/f4b [0,4194304] 0 2024-09-16T08:57:13.435 INFO:tasks.workunit.client.1.smithi097.stdout:6/598: readlink d1/dc/d1e/d9e/la1 0 2024-09-16T08:57:13.436 INFO:tasks.workunit.client.1.smithi097.stdout:3/619: symlink d7/d28/d97/ld2 0 2024-09-16T08:57:13.436 INFO:tasks.workunit.client.0.smithi031.stdout:6/262: rmdir d0/dd 39 2024-09-16T08:57:13.437 INFO:tasks.workunit.client.0.smithi031.stdout:6/263: dread - d0/d17/d19/f26 zero size 2024-09-16T08:57:13.437 INFO:tasks.workunit.client.0.smithi031.stdout:6/264: read d0/dd/f41 [764215,28434] 0 2024-09-16T08:57:13.438 INFO:tasks.workunit.client.1.smithi097.stdout:2/612: mkdir d3/d4/d9/d67/db2 0 2024-09-16T08:57:13.440 INFO:tasks.workunit.client.0.smithi031.stdout:5/247: mkdir d2/d4b 0 2024-09-16T08:57:13.443 INFO:tasks.workunit.client.1.smithi097.stdout:0/606: unlink da/f3e 0 2024-09-16T08:57:13.445 INFO:tasks.workunit.client.0.smithi031.stdout:7/283: write d2/d8/f4a [1656741,80047] 0 2024-09-16T08:57:13.445 INFO:tasks.workunit.client.0.smithi031.stdout:7/284: chown d2/d8/d11/f40 62894 1 2024-09-16T08:57:13.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/154: symlink d4/l32 0 2024-09-16T08:57:13.447 INFO:tasks.workunit.client.1.smithi097.stdout:9/564: dwrite d2/d5/fb [4194304,4194304] 0 2024-09-16T08:57:13.447 INFO:tasks.workunit.client.1.smithi097.stdout:8/593: truncate d8/dd/d15/f8f 1955763 0 2024-09-16T08:57:13.449 INFO:tasks.workunit.client.0.smithi031.stdout:6/265: dread d0/f4 [0,4194304] 0 2024-09-16T08:57:13.450 INFO:tasks.workunit.client.0.smithi031.stdout:1/203: symlink d2/l44 0 2024-09-16T08:57:13.453 INFO:tasks.workunit.client.1.smithi097.stdout:1/669: rename d5/d2c/fa6 to d5/db/dc/fde 0 2024-09-16T08:57:13.453 INFO:tasks.workunit.client.1.smithi097.stdout:1/670: chown d5/d17/d54/d6b/da8/dc5 2 1 2024-09-16T08:57:13.453 INFO:tasks.workunit.client.1.smithi097.stdout:1/671: chown d5/d17/d54/d6b/l75 53822538 1 2024-09-16T08:57:13.453 INFO:tasks.workunit.client.1.smithi097.stdout:1/672: chown d5/d17/d54/d6b/da8/fc8 206510402 1 2024-09-16T08:57:13.454 INFO:tasks.workunit.client.0.smithi031.stdout:0/202: creat d6/d12/f49 x:0 0 0 2024-09-16T08:57:13.457 INFO:tasks.workunit.client.0.smithi031.stdout:7/285: dread d2/d8/d11/f31 [0,4194304] 0 2024-09-16T08:57:13.458 INFO:tasks.workunit.client.1.smithi097.stdout:6/599: mkdir d1/d3/d81/dc4/dc6 0 2024-09-16T08:57:13.460 INFO:tasks.workunit.client.1.smithi097.stdout:3/620: creat d7/d28/d97/fd3 x:0 0 0 2024-09-16T08:57:13.461 INFO:tasks.workunit.client.1.smithi097.stdout:2/613: mknod d3/d4/d68/cb3 0 2024-09-16T08:57:13.463 INFO:tasks.workunit.client.1.smithi097.stdout:2/614: write d3/d4/d68/d8b/d7c/d7d/f8d [446112,100416] 0 2024-09-16T08:57:13.463 INFO:tasks.workunit.client.1.smithi097.stdout:2/615: write d3/d4/d68/d8b/d7c/d7d/d53/d47/f98 [373862,39703] 0 2024-09-16T08:57:13.463 INFO:tasks.workunit.client.1.smithi097.stdout:2/616: chown d3/d4/d68/d8b/d7c/d7d/d53/f58 6 1 2024-09-16T08:57:13.464 INFO:tasks.workunit.client.1.smithi097.stdout:2/617: creat d3/d4/d9/fb4 x:0 0 0 2024-09-16T08:57:13.464 INFO:tasks.workunit.client.1.smithi097.stdout:0/607: unlink da/d1a/f3a 0 2024-09-16T08:57:13.464 INFO:tasks.workunit.client.0.smithi031.stdout:4/104: rmdir d0/d2 39 2024-09-16T08:57:13.464 INFO:tasks.workunit.client.0.smithi031.stdout:4/105: chown d0/f1 7484 1 2024-09-16T08:57:13.465 INFO:tasks.workunit.client.0.smithi031.stdout:9/163: creat d3/d5/da/d34/f35 x:0 0 0 2024-09-16T08:57:13.467 INFO:tasks.workunit.client.0.smithi031.stdout:5/248: truncate d2/dc/d25/f22 1979952 0 2024-09-16T08:57:13.468 INFO:tasks.workunit.client.1.smithi097.stdout:7/559: dwrite d4/f9d [4194304,4194304] 0 2024-09-16T08:57:13.468 INFO:tasks.workunit.client.1.smithi097.stdout:9/565: mkdir d2/d30/d74/d8c/d94/d96/dac/db3 0 2024-09-16T08:57:13.470 INFO:tasks.workunit.client.1.smithi097.stdout:8/594: mknod d8/dd/d15/d2c/d4a/cb8 0 2024-09-16T08:57:13.473 INFO:tasks.workunit.client.0.smithi031.stdout:4/106: dread d0/fa [0,4194304] 0 2024-09-16T08:57:13.473 INFO:tasks.workunit.client.0.smithi031.stdout:4/107: stat d0/fb 0 2024-09-16T08:57:13.473 INFO:tasks.workunit.client.0.smithi031.stdout:7/286: dread d2/f56 [0,4194304] 0 2024-09-16T08:57:13.473 INFO:tasks.workunit.client.0.smithi031.stdout:8/155: dwrite d4/d7/d21/d24/f31 [0,4194304] 0 2024-09-16T08:57:13.473 INFO:tasks.workunit.client.1.smithi097.stdout:1/673: rename d5/d17/f26 to d5/d17/d54/fdf 0 2024-09-16T08:57:13.474 INFO:tasks.workunit.client.1.smithi097.stdout:1/674: write d5/d2c/d3b/fb7 [864504,19240] 0 2024-09-16T08:57:13.474 INFO:tasks.workunit.client.1.smithi097.stdout:8/595: fsync d8/dd/d15/d2c/d4a/d4d/f5c 0 2024-09-16T08:57:13.474 INFO:tasks.workunit.client.1.smithi097.stdout:6/600: truncate d1/d3/d16/f7b 677994 0 2024-09-16T08:57:13.475 INFO:tasks.workunit.client.1.smithi097.stdout:6/601: read d1/d3/d16/d4f/f92 [136789,86513] 0 2024-09-16T08:57:13.477 INFO:tasks.workunit.client.1.smithi097.stdout:3/621: link d7/d28/d30/d69/f6a d7/d28/fd4 0 2024-09-16T08:57:13.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/204: dwrite d2/de/f2f [0,4194304] 0 2024-09-16T08:57:13.479 INFO:tasks.workunit.client.0.smithi031.stdout:1/205: truncate d2/d4/d22/f23 861900 0 2024-09-16T08:57:13.483 INFO:tasks.workunit.client.1.smithi097.stdout:2/618: creat d3/d4/d68/d8b/d7c/d7d/fb5 x:0 0 0 2024-09-16T08:57:13.485 INFO:tasks.workunit.client.1.smithi097.stdout:0/608: mknod da/d24/d4c/d6d/cdc 0 2024-09-16T08:57:13.485 INFO:tasks.workunit.client.1.smithi097.stdout:0/609: creat da/d7e/fdd x:0 0 0 2024-09-16T08:57:13.487 INFO:tasks.workunit.client.0.smithi031.stdout:6/266: rename d0/dd/f36 to d0/d10/d14/d3f/f4b 0 2024-09-16T08:57:13.487 INFO:tasks.workunit.client.0.smithi031.stdout:6/267: fdatasync d0/dd/d2b/f31 0 2024-09-16T08:57:13.487 INFO:tasks.workunit.client.0.smithi031.stdout:6/268: dread - d0/dd/d2b/f31 zero size 2024-09-16T08:57:13.487 INFO:tasks.workunit.client.0.smithi031.stdout:0/203: dwrite d6/d12/d13/f23 [0,4194304] 0 2024-09-16T08:57:13.488 INFO:tasks.workunit.client.0.smithi031.stdout:9/164: creat d3/d5/da/db/f36 x:0 0 0 2024-09-16T08:57:13.488 INFO:tasks.workunit.client.0.smithi031.stdout:9/165: chown d3/d5/da/db/c25 47 1 2024-09-16T08:57:13.490 INFO:tasks.workunit.client.0.smithi031.stdout:5/249: getdents d2/dc/d1d 0 2024-09-16T08:57:13.496 INFO:tasks.workunit.client.0.smithi031.stdout:1/206: dwrite d2/de/f13 [0,4194304] 0 2024-09-16T08:57:13.496 INFO:tasks.workunit.client.0.smithi031.stdout:1/207: truncate d2/d4/d22/f23 1424697 0 2024-09-16T08:57:13.496 INFO:tasks.workunit.client.0.smithi031.stdout:1/208: creat d2/d4/d22/f45 x:0 0 0 2024-09-16T08:57:13.496 INFO:tasks.workunit.client.0.smithi031.stdout:7/287: symlink d2/d8/d61/l65 0 2024-09-16T08:57:13.497 INFO:tasks.workunit.client.0.smithi031.stdout:9/166: write d3/d5/da/f2b [4587493,44242] 0 2024-09-16T08:57:13.499 INFO:tasks.workunit.client.0.smithi031.stdout:8/156: creat d4/d2d/f33 x:0 0 0 2024-09-16T08:57:13.504 INFO:tasks.workunit.client.1.smithi097.stdout:7/560: mknod d4/d7/dab/cb9 0 2024-09-16T08:57:13.506 INFO:tasks.workunit.client.0.smithi031.stdout:6/269: creat d0/d10/d14/d3f/f4c x:0 0 0 2024-09-16T08:57:13.508 INFO:tasks.workunit.client.0.smithi031.stdout:0/204: mknod d6/c4a 0 2024-09-16T08:57:13.508 INFO:tasks.workunit.client.0.smithi031.stdout:0/205: write d6/d12/f2d [240324,54267] 0 2024-09-16T08:57:13.508 INFO:tasks.workunit.client.0.smithi031.stdout:0/206: stat d6/l25 0 2024-09-16T08:57:13.508 INFO:tasks.workunit.client.0.smithi031.stdout:0/207: truncate d6/d12/f22 133537 0 2024-09-16T08:57:13.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/250: mkdir d2/dc/d3d/d4c 0 2024-09-16T08:57:13.510 INFO:tasks.workunit.client.1.smithi097.stdout:1/675: truncate d5/db/dc/d19/f3f 5933238 0 2024-09-16T08:57:13.513 INFO:tasks.workunit.client.1.smithi097.stdout:1/676: dread d5/d14/d95/fb8 [0,4194304] 0 2024-09-16T08:57:13.514 INFO:tasks.workunit.client.1.smithi097.stdout:8/596: mkdir d8/dd/d15/d2c/d4a/d4d/d3d/db9 0 2024-09-16T08:57:13.515 INFO:tasks.workunit.client.0.smithi031.stdout:9/167: dwrite d3/d5/da/d34/f35 [0,4194304] 0 2024-09-16T08:57:13.516 INFO:tasks.workunit.client.0.smithi031.stdout:1/209: creat d2/d4/f46 x:0 0 0 2024-09-16T08:57:13.518 INFO:tasks.workunit.client.1.smithi097.stdout:6/602: link d1/d3/d16/d6a/f99 d1/d3/d81/dad/d5a/fc7 0 2024-09-16T08:57:13.518 INFO:tasks.workunit.client.1.smithi097.stdout:6/603: write d1/d3/d81/dad/d5a/f32 [1762507,51966] 0 2024-09-16T08:57:13.520 INFO:tasks.workunit.client.1.smithi097.stdout:3/622: creat d7/de/d3c/d3f/d60/dcb/fd5 x:0 0 0 2024-09-16T08:57:13.522 INFO:tasks.workunit.client.0.smithi031.stdout:8/157: mkdir d4/d34 0 2024-09-16T08:57:13.524 INFO:tasks.workunit.client.0.smithi031.stdout:6/270: mknod d0/dd/d35/c4d 0 2024-09-16T08:57:13.525 INFO:tasks.workunit.client.0.smithi031.stdout:6/271: dread - d0/d17/d19/f2e zero size 2024-09-16T08:57:13.527 INFO:tasks.workunit.client.1.smithi097.stdout:2/619: creat d3/d4/d9/d67/fb6 x:0 0 0 2024-09-16T08:57:13.527 INFO:tasks.workunit.client.1.smithi097.stdout:2/620: chown d3/d4/fa2 2942626 1 2024-09-16T08:57:13.527 INFO:tasks.workunit.client.1.smithi097.stdout:2/621: write d3/d4/d77/f96 [733176,82195] 0 2024-09-16T08:57:13.527 INFO:tasks.workunit.client.1.smithi097.stdout:2/622: getdents d3/d4/d68/d8b/d61/dac 0 2024-09-16T08:57:13.527 INFO:tasks.workunit.client.0.smithi031.stdout:0/208: link d6/f1e d6/f4b 0 2024-09-16T08:57:13.527 INFO:tasks.workunit.client.0.smithi031.stdout:0/209: stat d6/f33 0 2024-09-16T08:57:13.531 INFO:tasks.workunit.client.1.smithi097.stdout:0/610: truncate da/d24/d4c/d80/d3b/f4e 176124 0 2024-09-16T08:57:13.537 INFO:tasks.workunit.client.0.smithi031.stdout:1/210: rename d2/f24 to d2/d15/d38/f47 0 2024-09-16T08:57:13.540 INFO:tasks.workunit.client.1.smithi097.stdout:6/604: rename d1/d3/db6/fb8 to d1/d3/d81/dc4/fc8 0 2024-09-16T08:57:13.540 INFO:tasks.workunit.client.0.smithi031.stdout:9/168: getdents d3/d5/da/db 0 2024-09-16T08:57:13.542 INFO:tasks.workunit.client.1.smithi097.stdout:7/561: dwrite d4/d9/d1c/d5f/f6c [0,4194304] 0 2024-09-16T08:57:13.542 INFO:tasks.workunit.client.1.smithi097.stdout:7/562: write d4/d7/d19/f71 [2772606,102553] 0 2024-09-16T08:57:13.543 INFO:tasks.workunit.client.1.smithi097.stdout:7/563: creat d4/d9/d1c/fba x:0 0 0 2024-09-16T08:57:13.543 INFO:tasks.workunit.client.1.smithi097.stdout:2/623: creat d3/d4/d68/d8b/fb7 x:0 0 0 2024-09-16T08:57:13.543 INFO:tasks.workunit.client.1.smithi097.stdout:2/624: write d3/d4/d8/d51/f59 [422326,78789] 0 2024-09-16T08:57:13.544 INFO:tasks.workunit.client.1.smithi097.stdout:0/611: mkdir da/d7e/dde 0 2024-09-16T08:57:13.544 INFO:tasks.workunit.client.1.smithi097.stdout:0/612: truncate da/d7e/fdd 692293 0 2024-09-16T08:57:13.544 INFO:tasks.workunit.client.1.smithi097.stdout:0/613: chown da/d1a/d26/d47/d97/fa2 13990102 1 2024-09-16T08:57:13.545 INFO:tasks.workunit.client.0.smithi031.stdout:0/210: rename d6/d12/d13/c1f to d6/d12/d13/d36/c4c 0 2024-09-16T08:57:13.546 INFO:tasks.workunit.client.1.smithi097.stdout:3/623: rename d7/de/d3c/d4f/f7f to d7/d28/d97/fd6 0 2024-09-16T08:57:13.546 INFO:tasks.workunit.client.1.smithi097.stdout:3/624: creat d7/d28/d63/d70/fd7 x:0 0 0 2024-09-16T08:57:13.546 INFO:tasks.workunit.client.0.smithi031.stdout:1/211: symlink d2/d4/d22/l48 0 2024-09-16T08:57:13.547 INFO:tasks.workunit.client.1.smithi097.stdout:6/605: link d1/d3/d81/dad/d5a/l63 d1/d3/d81/dad/d5a/lc9 0 2024-09-16T08:57:13.547 INFO:tasks.workunit.client.1.smithi097.stdout:6/606: fsync d1/dc/d1e/f68 0 2024-09-16T08:57:13.547 INFO:tasks.workunit.client.0.smithi031.stdout:0/211: mkdir d6/d12/d47/d4d 0 2024-09-16T08:57:13.548 INFO:tasks.workunit.client.0.smithi031.stdout:1/212: mkdir d2/d49 0 2024-09-16T08:57:13.548 INFO:tasks.workunit.client.0.smithi031.stdout:1/213: read d2/d4/f12 [153042,4547] 0 2024-09-16T08:57:13.549 INFO:tasks.workunit.client.1.smithi097.stdout:5/661: sync 2024-09-16T08:57:13.551 INFO:tasks.workunit.client.0.smithi031.stdout:0/212: rename d6/d12/d13/f2e to d6/d12/d47/f4e 0 2024-09-16T08:57:13.556 INFO:tasks.workunit.client.0.smithi031.stdout:0/213: rename d6/d12/f2c to d6/f4f 0 2024-09-16T08:57:13.557 INFO:tasks.workunit.client.1.smithi097.stdout:0/614: creat da/d24/fdf x:0 0 0 2024-09-16T08:57:13.559 INFO:tasks.workunit.client.1.smithi097.stdout:3/625: truncate d7/de/d3c/f45 2801962 0 2024-09-16T08:57:13.560 INFO:tasks.workunit.client.1.smithi097.stdout:3/626: dread - d7/de/d3c/d3f/d60/dcb/fd5 zero size 2024-09-16T08:57:13.560 INFO:tasks.workunit.client.0.smithi031.stdout:9/169: write d3/f9 [2699870,77852] 0 2024-09-16T08:57:13.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/251: dwrite d2/f1a [0,4194304] 0 2024-09-16T08:57:13.562 INFO:tasks.workunit.client.1.smithi097.stdout:7/564: rename d4/d9/d2b/f7e to d4/d12/d1f/fbb 0 2024-09-16T08:57:13.563 INFO:tasks.workunit.client.1.smithi097.stdout:7/565: creat d4/d9/d1c/d5f/fbc x:0 0 0 2024-09-16T08:57:13.563 INFO:tasks.workunit.client.1.smithi097.stdout:4/592: sync 2024-09-16T08:57:13.563 INFO:tasks.workunit.client.1.smithi097.stdout:5/662: dread db/d18/d1c/d42/d46/f9b [0,4194304] 0 2024-09-16T08:57:13.564 INFO:tasks.workunit.client.1.smithi097.stdout:0/615: dread da/f1b [0,4194304] 0 2024-09-16T08:57:13.564 INFO:tasks.workunit.client.0.smithi031.stdout:9/170: mknod d3/d5/c37 0 2024-09-16T08:57:13.564 INFO:tasks.workunit.client.0.smithi031.stdout:9/171: chown d3/d5/da/db/d21/f2f 12 1 2024-09-16T08:57:13.564 INFO:tasks.workunit.client.1.smithi097.stdout:8/597: dwrite d8/dd/d71/f7d [0,4194304] 0 2024-09-16T08:57:13.566 INFO:tasks.workunit.client.1.smithi097.stdout:3/627: mkdir d7/d28/d63/dd8 0 2024-09-16T08:57:13.568 INFO:tasks.workunit.client.1.smithi097.stdout:2/625: rename d3/d4/fa2 to d3/d4/d9/d67/db2/fb8 0 2024-09-16T08:57:13.568 INFO:tasks.workunit.client.1.smithi097.stdout:2/626: fdatasync d3/f71 0 2024-09-16T08:57:13.568 INFO:tasks.workunit.client.0.smithi031.stdout:8/158: dwrite d4/d7/d21/d24/f31 [0,4194304] 0 2024-09-16T08:57:13.568 INFO:tasks.workunit.client.0.smithi031.stdout:8/159: write d4/f29 [496746,66272] 0 2024-09-16T08:57:13.568 INFO:tasks.workunit.client.0.smithi031.stdout:6/272: dwrite d0/d17/f40 [0,4194304] 0 2024-09-16T08:57:13.572 INFO:tasks.workunit.client.0.smithi031.stdout:9/172: rename d3/d5/da/l28 to d3/d5/da/db/l38 0 2024-09-16T08:57:13.573 INFO:tasks.workunit.client.1.smithi097.stdout:5/663: link db/d18/c27 db/d18/d1c/d42/d46/d7e/d64/d7f/d89/dbc/cdb 0 2024-09-16T08:57:13.573 INFO:tasks.workunit.client.1.smithi097.stdout:5/664: fsync db/d18/d20/d2b/f87 0 2024-09-16T08:57:13.574 INFO:tasks.workunit.client.1.smithi097.stdout:5/665: write db/d18/d1c/d42/d46/f47 [2031779,73077] 0 2024-09-16T08:57:13.577 INFO:tasks.workunit.client.1.smithi097.stdout:0/616: creat da/d7e/dc4/fe0 x:0 0 0 2024-09-16T08:57:13.578 INFO:tasks.workunit.client.1.smithi097.stdout:6/607: dwrite d1/dc/d3c/f6d [0,4194304] 0 2024-09-16T08:57:13.578 INFO:tasks.workunit.client.1.smithi097.stdout:6/608: truncate d1/dc/d3c/f6d 4464590 0 2024-09-16T08:57:13.579 INFO:tasks.workunit.client.1.smithi097.stdout:8/598: symlink d8/dd/d15/d2c/d4a/d4d/d3d/lba 0 2024-09-16T08:57:13.580 INFO:tasks.workunit.client.0.smithi031.stdout:6/273: link d0/d17/d19/c3a d0/d17/d19/c4e 0 2024-09-16T08:57:13.582 INFO:tasks.workunit.client.0.smithi031.stdout:9/173: creat d3/d5/da/db/f39 x:0 0 0 2024-09-16T08:57:13.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/274: symlink d0/d17/l4f 0 2024-09-16T08:57:13.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/275: truncate d0/dd/f43 375706 0 2024-09-16T08:57:13.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/276: write d0/dd/d2b/f31 [188462,6322] 0 2024-09-16T08:57:13.585 INFO:tasks.workunit.client.1.smithi097.stdout:6/609: dread d1/dc/d3c/f44 [0,4194304] 0 2024-09-16T08:57:13.585 INFO:tasks.workunit.client.0.smithi031.stdout:1/214: dwrite d2/de/f2f [0,4194304] 0 2024-09-16T08:57:13.586 INFO:tasks.workunit.client.1.smithi097.stdout:2/627: symlink d3/d4/d68/d8b/d7c/lb9 0 2024-09-16T08:57:13.588 INFO:tasks.workunit.client.0.smithi031.stdout:9/174: unlink d3/d5/da/db/f36 0 2024-09-16T08:57:13.591 INFO:tasks.workunit.client.1.smithi097.stdout:4/593: rename d0/d35/f38 to d0/d23/fd0 0 2024-09-16T08:57:13.591 INFO:tasks.workunit.client.1.smithi097.stdout:4/594: chown d0/d35/d6c/cc4 1158 1 2024-09-16T08:57:13.591 INFO:tasks.workunit.client.1.smithi097.stdout:4/595: write d0/d7/df/d83/fb5 [712904,71860] 0 2024-09-16T08:57:13.591 INFO:tasks.workunit.client.1.smithi097.stdout:4/596: readlink d0/d7/d52/d26/d8c/lb6 0 2024-09-16T08:57:13.597 INFO:tasks.workunit.client.1.smithi097.stdout:7/566: dwrite d4/d9/dc/f93 [0,4194304] 0 2024-09-16T08:57:13.602 INFO:tasks.workunit.client.1.smithi097.stdout:0/617: symlink da/df/d3d/le1 0 2024-09-16T08:57:13.602 INFO:tasks.workunit.client.1.smithi097.stdout:0/618: truncate da/d1a/d26/d47/db3/d60/d73/d85/fcd 451581 0 2024-09-16T08:57:13.607 INFO:tasks.workunit.client.0.smithi031.stdout:1/215: mkdir d2/d15/d4a 0 2024-09-16T08:57:13.609 INFO:tasks.workunit.client.1.smithi097.stdout:8/599: creat d8/dd/d15/d16/d7c/da2/fbb x:0 0 0 2024-09-16T08:57:13.609 INFO:tasks.workunit.client.1.smithi097.stdout:8/600: stat d8/dd/d15/d2c/l3b 0 2024-09-16T08:57:13.609 INFO:tasks.workunit.client.1.smithi097.stdout:3/628: mknod d7/cd9 0 2024-09-16T08:57:13.611 INFO:tasks.workunit.client.0.smithi031.stdout:0/214: dwrite d6/d12/d13/f3e [0,4194304] 0 2024-09-16T08:57:13.611 INFO:tasks.workunit.client.1.smithi097.stdout:2/628: link d3/d4/d77/f3c d3/d4/d68/d8b/fba 0 2024-09-16T08:57:13.612 INFO:tasks.workunit.client.0.smithi031.stdout:8/160: dwrite d4/d7/d21/d24/f31 [0,4194304] 0 2024-09-16T08:57:13.613 INFO:tasks.workunit.client.1.smithi097.stdout:5/666: dwrite db/d1a/f92 [0,4194304] 0 2024-09-16T08:57:13.614 INFO:tasks.workunit.client.0.smithi031.stdout:6/277: dwrite d0/d10/d14/d3f/f48 [0,4194304] 0 2024-09-16T08:57:13.614 INFO:tasks.workunit.client.1.smithi097.stdout:4/597: mkdir d0/d35/da4/dd1 0 2024-09-16T08:57:13.615 INFO:tasks.workunit.client.1.smithi097.stdout:4/598: readlink d0/d35/d6c/lc7 0 2024-09-16T08:57:13.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/215: mknod d6/d12/d47/c50 0 2024-09-16T08:57:13.619 INFO:tasks.workunit.client.0.smithi031.stdout:0/216: fdatasync d6/fa 0 2024-09-16T08:57:13.619 INFO:tasks.workunit.client.0.smithi031.stdout:9/175: dwrite d3/d5/da/f14 [0,4194304] 0 2024-09-16T08:57:13.623 INFO:tasks.workunit.client.0.smithi031.stdout:8/161: mkdir d4/d7/d21/d35 0 2024-09-16T08:57:13.626 INFO:tasks.workunit.client.1.smithi097.stdout:0/619: symlink da/d1a/d9c/le2 0 2024-09-16T08:57:13.626 INFO:tasks.workunit.client.1.smithi097.stdout:0/620: truncate da/d1a/d26/d47/d97/f8f 1100930 0 2024-09-16T08:57:13.626 INFO:tasks.workunit.client.1.smithi097.stdout:0/621: read da/d24/d4b/fa3 [454811,119204] 0 2024-09-16T08:57:13.627 INFO:tasks.workunit.client.1.smithi097.stdout:8/601: write d8/dd/d15/d2c/d4a/d4e/db2/fb3 [1332594,100424] 0 2024-09-16T08:57:13.627 INFO:tasks.workunit.client.1.smithi097.stdout:8/602: stat d8/dd/d15/d2c/d4a/d4d/d3f/fa6 0 2024-09-16T08:57:13.629 INFO:tasks.workunit.client.0.smithi031.stdout:0/217: read d6/f9 [766073,76703] 0 2024-09-16T08:57:13.629 INFO:tasks.workunit.client.0.smithi031.stdout:0/218: creat d6/d12/d13/d36/f51 x:0 0 0 2024-09-16T08:57:13.630 INFO:tasks.workunit.client.0.smithi031.stdout:6/278: dread d0/ff [0,4194304] 0 2024-09-16T08:57:13.631 INFO:tasks.workunit.client.0.smithi031.stdout:6/279: dread d0/dd/f23 [0,4194304] 0 2024-09-16T08:57:13.634 INFO:tasks.workunit.client.1.smithi097.stdout:6/610: rename d1/d3 to d1/dc/d1e/d9e/dca 0 2024-09-16T08:57:13.638 INFO:tasks.workunit.client.0.smithi031.stdout:9/176: rename d3/f1f to d3/f3a 0 2024-09-16T08:57:13.641 INFO:tasks.workunit.client.0.smithi031.stdout:8/162: mkdir d4/d7/d36 0 2024-09-16T08:57:13.642 INFO:tasks.workunit.client.0.smithi031.stdout:1/216: dwrite d2/f17 [0,4194304] 0 2024-09-16T08:57:13.645 INFO:tasks.workunit.client.0.smithi031.stdout:0/219: symlink d6/d12/l52 0 2024-09-16T08:57:13.648 INFO:tasks.workunit.client.0.smithi031.stdout:9/177: unlink d3/d5/f13 0 2024-09-16T08:57:13.650 INFO:tasks.workunit.client.1.smithi097.stdout:5/667: dwrite db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/f99 [0,4194304] 0 2024-09-16T08:57:13.652 INFO:tasks.workunit.client.1.smithi097.stdout:2/629: dwrite d3/f9b [0,4194304] 0 2024-09-16T08:57:13.652 INFO:tasks.workunit.client.0.smithi031.stdout:8/163: mkdir d4/d7/d21/d37 0 2024-09-16T08:57:13.652 INFO:tasks.workunit.client.0.smithi031.stdout:6/280: dwrite d0/d10/d14/f47 [0,4194304] 0 2024-09-16T08:57:13.653 INFO:tasks.workunit.client.1.smithi097.stdout:4/599: link d0/d36/c8e d0/d23/d5f/d7d/dbc/cd2 0 2024-09-16T08:57:13.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/164: dread d4/d7/f1a [0,4194304] 0 2024-09-16T08:57:13.655 INFO:tasks.workunit.client.0.smithi031.stdout:8/165: chown d4/c9 254780967 1 2024-09-16T08:57:13.656 INFO:tasks.workunit.client.1.smithi097.stdout:0/622: mknod da/d7e/dc4/ce3 0 2024-09-16T08:57:13.657 INFO:tasks.workunit.client.1.smithi097.stdout:0/623: chown da/d24/d4c/d80/lb8 5801549 1 2024-09-16T08:57:13.660 INFO:tasks.workunit.client.0.smithi031.stdout:0/220: dwrite d6/d12/d13/f3e [0,4194304] 0 2024-09-16T08:57:13.661 INFO:tasks.workunit.client.1.smithi097.stdout:3/629: rename d7/l13 to d7/de/d3c/dbe/lda 0 2024-09-16T08:57:13.662 INFO:tasks.workunit.client.1.smithi097.stdout:3/630: chown d7/d28/d63/d70/fd7 130 1 2024-09-16T08:57:13.662 INFO:tasks.workunit.client.1.smithi097.stdout:3/631: stat d7/de/d3c/d3f/f7b 0 2024-09-16T08:57:13.662 INFO:tasks.workunit.client.1.smithi097.stdout:3/632: readlink d7/de/d3c/d3f/d60/d7a/l7e 0 2024-09-16T08:57:13.663 INFO:tasks.workunit.client.1.smithi097.stdout:6/611: creat d1/dc/fcb x:0 0 0 2024-09-16T08:57:13.664 INFO:tasks.workunit.client.1.smithi097.stdout:3/633: dread d7/de/d3c/d3f/d60/dcb/fa7 [0,4194304] 0 2024-09-16T08:57:13.664 INFO:tasks.workunit.client.1.smithi097.stdout:3/634: chown d7/de/d3c/d3f/d60 708 1 2024-09-16T08:57:13.667 INFO:tasks.workunit.client.0.smithi031.stdout:1/217: creat d2/d15/d1f/d43/f4b x:0 0 0 2024-09-16T08:57:13.669 INFO:tasks.workunit.client.1.smithi097.stdout:5/668: creat db/d18/fdc x:0 0 0 2024-09-16T08:57:13.670 INFO:tasks.workunit.client.0.smithi031.stdout:9/178: mkdir d3/d5/da/db/d21/d3b 0 2024-09-16T08:57:13.670 INFO:tasks.workunit.client.0.smithi031.stdout:9/179: truncate f2 622170 0 2024-09-16T08:57:13.677 INFO:tasks.workunit.client.0.smithi031.stdout:6/281: mknod d0/dd/c50 0 2024-09-16T08:57:13.677 INFO:tasks.workunit.client.1.smithi097.stdout:4/600: unlink d0/d23/d5f/f2b 0 2024-09-16T08:57:13.677 INFO:tasks.workunit.client.1.smithi097.stdout:4/601: fsync d0/d23/f7b 0 2024-09-16T08:57:13.681 INFO:tasks.workunit.client.1.smithi097.stdout:6/612: creat d1/dc/d1e/d9e/dca/d81/dc4/fcc x:0 0 0 2024-09-16T08:57:13.681 INFO:tasks.workunit.client.1.smithi097.stdout:6/613: readlink d1/dc/d1e/d9e/dca/d81/dad/d5a/l50 0 2024-09-16T08:57:13.681 INFO:tasks.workunit.client.1.smithi097.stdout:6/614: readlink d1/dc/lac 0 2024-09-16T08:57:13.681 INFO:tasks.workunit.client.1.smithi097.stdout:6/615: truncate d1/dc/d1e/d9e/db3/fbd 758872 0 2024-09-16T08:57:13.682 INFO:tasks.workunit.client.0.smithi031.stdout:1/218: rename d2/d15/d1f/d43/f4b to d2/d15/d1f/d43/f4c 0 2024-09-16T08:57:13.686 INFO:tasks.workunit.client.1.smithi097.stdout:3/635: link d7/de/d3c/d3f/d60/dcb/f76 d7/d28/d30/d69/fdb 0 2024-09-16T08:57:13.686 INFO:tasks.workunit.client.1.smithi097.stdout:6/616: dread d1/dc/d1e/fc1 [0,4194304] 0 2024-09-16T08:57:13.687 INFO:tasks.workunit.client.0.smithi031.stdout:6/282: truncate d0/fb 792134 0 2024-09-16T08:57:13.687 INFO:tasks.workunit.client.1.smithi097.stdout:6/617: dread d1/dc/d3c/f6f [0,4194304] 0 2024-09-16T08:57:13.688 INFO:tasks.workunit.client.1.smithi097.stdout:0/624: dwrite da/d24/d4b/fa3 [0,4194304] 0 2024-09-16T08:57:13.690 INFO:tasks.workunit.client.1.smithi097.stdout:5/669: rename db/d6e/d81/lb1 to db/d18/d20/ldd 0 2024-09-16T08:57:13.693 INFO:tasks.workunit.client.1.smithi097.stdout:3/636: link d7/d28/d30/f53 d7/d28/d63/d70/fdc 0 2024-09-16T08:57:13.693 INFO:tasks.workunit.client.1.smithi097.stdout:3/637: creat d7/de/d8d/d90/d92/fdd x:0 0 0 2024-09-16T08:57:13.693 INFO:tasks.workunit.client.1.smithi097.stdout:3/638: write d7/de/d3c/d3f/d60/dcb/d35/d3d/f59 [1243962,59832] 0 2024-09-16T08:57:13.694 INFO:tasks.workunit.client.0.smithi031.stdout:8/166: dwrite d4/f6 [8388608,4194304] 0 2024-09-16T08:57:13.694 INFO:tasks.workunit.client.1.smithi097.stdout:9/566: sync 2024-09-16T08:57:13.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/219: rename d2/d4/l8 to d2/d15/d1f/d43/l4d 0 2024-09-16T08:57:13.697 INFO:tasks.workunit.client.0.smithi031.stdout:1/220: creat d2/d4/d22/f4e x:0 0 0 2024-09-16T08:57:13.697 INFO:tasks.workunit.client.0.smithi031.stdout:6/283: dread d0/dd/f43 [0,4194304] 0 2024-09-16T08:57:13.697 INFO:tasks.workunit.client.0.smithi031.stdout:6/284: chown d0/d10/l44 1024 1 2024-09-16T08:57:13.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/285: stat d0/d10/d14/d3f 0 2024-09-16T08:57:13.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/286: chown d0/d10/d14/d3f 233680 1 2024-09-16T08:57:13.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/167: mkdir d4/d7/d36/d38 0 2024-09-16T08:57:13.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/168: write d4/d7/f28 [938099,111330] 0 2024-09-16T08:57:13.703 INFO:tasks.workunit.client.1.smithi097.stdout:0/625: mknod da/d1a/d41/ce4 0 2024-09-16T08:57:13.703 INFO:tasks.workunit.client.1.smithi097.stdout:0/626: dread - da/df/d8a/fa9 zero size 2024-09-16T08:57:13.706 INFO:tasks.workunit.client.1.smithi097.stdout:6/618: rename d1/dc/d1e/d9e/dca/d16/c57 to d1/dc/d1e/d9e/dca/d81/ccd 0 2024-09-16T08:57:13.706 INFO:tasks.workunit.client.0.smithi031.stdout:0/221: dwrite d6/d12/d13/d36/f38 [0,4194304] 0 2024-09-16T08:57:13.708 INFO:tasks.workunit.client.1.smithi097.stdout:5/670: write db/d18/d1c/d42/d46/d7e/f51 [500407,48748] 0 2024-09-16T08:57:13.708 INFO:tasks.workunit.client.1.smithi097.stdout:5/671: write db/d18/d1c/d42/d46/d7e/d8f/fb4 [692597,15343] 0 2024-09-16T08:57:13.708 INFO:tasks.workunit.client.1.smithi097.stdout:5/672: dread - db/d18/d20/d2b/f87 zero size 2024-09-16T08:57:13.708 INFO:tasks.workunit.client.1.smithi097.stdout:5/673: write db/d18/d20/d2b/f87 [447871,76651] 0 2024-09-16T08:57:13.709 INFO:tasks.workunit.client.0.smithi031.stdout:6/287: mknod d0/c51 0 2024-09-16T08:57:13.710 INFO:tasks.workunit.client.1.smithi097.stdout:4/602: dwrite d0/d7/df/f2c [4194304,4194304] 0 2024-09-16T08:57:13.710 INFO:tasks.workunit.client.1.smithi097.stdout:4/603: fdatasync d0/d7/df/f37 0 2024-09-16T08:57:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:9/180: dwrite d3/d5/da/f14 [0,4194304] 0 2024-09-16T08:57:13.711 INFO:tasks.workunit.client.0.smithi031.stdout:9/181: chown d3/d5/da/f27 1715627292 1 2024-09-16T08:57:13.712 INFO:tasks.workunit.client.0.smithi031.stdout:8/169: link d4/l30 d4/d2d/l39 0 2024-09-16T08:57:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:8/170: truncate d4/f29 1205820 0 2024-09-16T08:57:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:8/171: readlink d4/l13 0 2024-09-16T08:57:13.713 INFO:tasks.workunit.client.0.smithi031.stdout:8/172: creat d4/d2d/f3a x:0 0 0 2024-09-16T08:57:13.715 INFO:tasks.workunit.client.0.smithi031.stdout:0/222: symlink d6/l53 0 2024-09-16T08:57:13.715 INFO:tasks.workunit.client.0.smithi031.stdout:0/223: write d6/f26 [1069919,48393] 0 2024-09-16T08:57:13.718 INFO:tasks.workunit.client.1.smithi097.stdout:9/567: rename d2/d3f/f9c to d2/d3f/fb4 0 2024-09-16T08:57:13.718 INFO:tasks.workunit.client.1.smithi097.stdout:9/568: write d2/d30/d74/d8c/d94/d96/fa5 [334306,97002] 0 2024-09-16T08:57:13.721 INFO:tasks.workunit.client.0.smithi031.stdout:6/288: rename d0/dd/f41 to d0/dd/d35/f52 0 2024-09-16T08:57:13.722 INFO:tasks.workunit.client.0.smithi031.stdout:3/247: sync 2024-09-16T08:57:13.722 INFO:tasks.workunit.client.0.smithi031.stdout:2/132: sync 2024-09-16T08:57:13.722 INFO:tasks.workunit.client.0.smithi031.stdout:2/133: write d6/da/f1d [236491,126932] 0 2024-09-16T08:57:13.723 INFO:tasks.workunit.client.1.smithi097.stdout:0/627: symlink da/d1a/d26/d47/db3/d60/d73/d85/le5 0 2024-09-16T08:57:13.724 INFO:tasks.workunit.client.0.smithi031.stdout:6/289: write d0/d17/f18 [5011915,19350] 0 2024-09-16T08:57:13.724 INFO:tasks.workunit.client.0.smithi031.stdout:6/290: write d0/d17/f42 [517246,27781] 0 2024-09-16T08:57:13.724 INFO:tasks.workunit.client.0.smithi031.stdout:6/291: chown d0/d17/f30 1215 1 2024-09-16T08:57:13.725 INFO:tasks.workunit.client.1.smithi097.stdout:3/639: dwrite d7/de/d3c/d3f/f7b [4194304,4194304] 0 2024-09-16T08:57:13.725 INFO:tasks.workunit.client.1.smithi097.stdout:5/674: mknod db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/cde 0 2024-09-16T08:57:13.725 INFO:tasks.workunit.client.1.smithi097.stdout:1/677: sync 2024-09-16T08:57:13.726 INFO:tasks.workunit.client.1.smithi097.stdout:1/678: fdatasync d5/d17/d54/fdf 0 2024-09-16T08:57:13.726 INFO:tasks.workunit.client.0.smithi031.stdout:9/182: mkdir d3/d5/da/d34/d3c 0 2024-09-16T08:57:13.727 INFO:tasks.workunit.client.0.smithi031.stdout:8/173: creat d4/d7/d21/f3b x:0 0 0 2024-09-16T08:57:13.729 INFO:tasks.workunit.client.0.smithi031.stdout:0/224: unlink d6/d12/d13/d36/f38 0 2024-09-16T08:57:13.729 INFO:tasks.workunit.client.0.smithi031.stdout:0/225: fdatasync d6/d12/d13/d36/f43 0 2024-09-16T08:57:13.730 INFO:tasks.workunit.client.1.smithi097.stdout:1/679: write d5/db/dc/d19/f3f [5346648,55986] 0 2024-09-16T08:57:13.732 INFO:tasks.workunit.client.0.smithi031.stdout:1/221: dwrite d2/d4/d22/f3d [0,4194304] 0 2024-09-16T08:57:13.735 INFO:tasks.workunit.client.1.smithi097.stdout:1/680: write d5/f77 [2280925,22035] 0 2024-09-16T08:57:13.735 INFO:tasks.workunit.client.1.smithi097.stdout:1/681: read - d5/db/dc/d19/f8c zero size 2024-09-16T08:57:13.735 INFO:tasks.workunit.client.1.smithi097.stdout:1/682: read - d5/d2c/d3b/d7f/fdb zero size 2024-09-16T08:57:13.736 INFO:tasks.workunit.client.1.smithi097.stdout:1/683: creat d5/db/dc/fe0 x:0 0 0 2024-09-16T08:57:13.736 INFO:tasks.workunit.client.1.smithi097.stdout:6/619: dwrite d1/dc/d1e/d9e/dca/d16/d4f/f92 [0,4194304] 0 2024-09-16T08:57:13.737 INFO:tasks.workunit.client.0.smithi031.stdout:3/248: mknod d3/c44 0 2024-09-16T08:57:13.737 INFO:tasks.workunit.client.0.smithi031.stdout:2/134: write d6/f8 [2641257,53318] 0 2024-09-16T08:57:13.737 INFO:tasks.workunit.client.0.smithi031.stdout:2/135: truncate d6/da/f27 1246475 0 2024-09-16T08:57:13.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/174: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:13.739 INFO:tasks.workunit.client.1.smithi097.stdout:9/569: symlink d2/d30/d63/lb5 0 2024-09-16T08:57:13.739 INFO:tasks.workunit.client.1.smithi097.stdout:9/570: fdatasync d2/d30/f5a 0 2024-09-16T08:57:13.739 INFO:tasks.workunit.client.1.smithi097.stdout:4/604: dwrite d0/d23/d5f/d7d/dbc/fbd [0,4194304] 0 2024-09-16T08:57:13.741 INFO:tasks.workunit.client.0.smithi031.stdout:6/292: symlink d0/d10/d14/d3f/l53 0 2024-09-16T08:57:13.741 INFO:tasks.workunit.client.0.smithi031.stdout:6/293: truncate d0/dd/f23 1382924 0 2024-09-16T08:57:13.743 INFO:tasks.workunit.client.1.smithi097.stdout:1/684: dread d5/f9 [0,4194304] 0 2024-09-16T08:57:13.744 INFO:tasks.workunit.client.0.smithi031.stdout:9/183: link d3/d5/da/db/f1e d3/d5/da/db/d21/f3d 0 2024-09-16T08:57:13.744 INFO:tasks.workunit.client.0.smithi031.stdout:9/184: creat d3/d5/da/f3e x:0 0 0 2024-09-16T08:57:13.744 INFO:tasks.workunit.client.0.smithi031.stdout:9/185: write d3/f9 [4235559,3864] 0 2024-09-16T08:57:13.744 INFO:tasks.workunit.client.0.smithi031.stdout:9/186: stat d3 0 2024-09-16T08:57:13.749 INFO:tasks.workunit.client.0.smithi031.stdout:0/226: dwrite d6/d12/d13/d36/f51 [0,4194304] 0 2024-09-16T08:57:13.750 INFO:tasks.workunit.client.0.smithi031.stdout:1/222: write d2/de/f2a [1377149,126922] 0 2024-09-16T08:57:13.750 INFO:tasks.workunit.client.0.smithi031.stdout:1/223: fdatasync d2/d4/f46 0 2024-09-16T08:57:13.751 INFO:tasks.workunit.client.1.smithi097.stdout:3/640: mkdir d7/de/d3c/dde 0 2024-09-16T08:57:13.751 INFO:tasks.workunit.client.1.smithi097.stdout:3/641: truncate d7/d28/d97/fd3 297648 0 2024-09-16T08:57:13.752 INFO:tasks.workunit.client.1.smithi097.stdout:5/675: symlink db/d18/d1c/d42/d46/d7e/ldf 0 2024-09-16T08:57:13.753 INFO:tasks.workunit.client.0.smithi031.stdout:9/187: dread d3/f9 [0,4194304] 0 2024-09-16T08:57:13.754 INFO:tasks.workunit.client.0.smithi031.stdout:9/188: write d3/d5/da/f3e [444782,36566] 0 2024-09-16T08:57:13.754 INFO:tasks.workunit.client.0.smithi031.stdout:9/189: creat d3/d5/da/d34/f3f x:0 0 0 2024-09-16T08:57:13.757 INFO:tasks.workunit.client.0.smithi031.stdout:0/227: write d6/f33 [3278129,110249] 0 2024-09-16T08:57:13.762 INFO:tasks.workunit.client.1.smithi097.stdout:3/642: write d7/d28/f2f [1906827,84726] 0 2024-09-16T08:57:13.765 INFO:tasks.workunit.client.0.smithi031.stdout:3/249: symlink d3/d14/d37/d1a/d24/l45 0 2024-09-16T08:57:13.767 INFO:tasks.workunit.client.1.smithi097.stdout:6/620: rmdir d1/dc/d1e/d9e/dca/d81/dad 39 2024-09-16T08:57:13.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/190: dwrite d3/d5/da/d34/f3f [0,4194304] 0 2024-09-16T08:57:13.792 INFO:tasks.workunit.client.1.smithi097.stdout:6/621: dwrite d1/dc/d1e/d9e/dca/f7 [0,4194304] 0 2024-09-16T08:57:13.792 INFO:tasks.workunit.client.1.smithi097.stdout:3/643: dwrite d7/de/d3c/d3f/d60/d7a/fa1 [0,4194304] 0 2024-09-16T08:57:13.807 INFO:tasks.workunit.client.0.smithi031.stdout:8/175: creat d4/d7/d36/d38/f3c x:0 0 0 2024-09-16T08:57:13.807 INFO:tasks.workunit.client.0.smithi031.stdout:8/176: stat d4/l12 0 2024-09-16T08:57:13.808 INFO:tasks.workunit.client.1.smithi097.stdout:6/622: dwrite d1/f37 [0,4194304] 0 2024-09-16T08:57:13.808 INFO:tasks.workunit.client.1.smithi097.stdout:6/623: fdatasync d1/dc/d1e/d9e/dca/f11 0 2024-09-16T08:57:13.808 INFO:tasks.workunit.client.1.smithi097.stdout:9/571: symlink d2/d30/d74/d8c/d94/d96/dac/d79/lb6 0 2024-09-16T08:57:13.809 INFO:tasks.workunit.client.1.smithi097.stdout:9/572: write d2/d30/d74/d8c/d94/d96/fa5 [27041,9505] 0 2024-09-16T08:57:13.811 INFO:tasks.workunit.client.0.smithi031.stdout:6/294: mkdir d0/d17/d19/d54 0 2024-09-16T08:57:13.818 INFO:tasks.workunit.client.1.smithi097.stdout:6/624: write d1/dc/d1e/d9e/dca/d16/f79 [3006101,81346] 0 2024-09-16T08:57:13.821 INFO:tasks.workunit.client.1.smithi097.stdout:0/628: getdents da/df/d3d 0 2024-09-16T08:57:13.822 INFO:tasks.workunit.client.1.smithi097.stdout:7/567: sync 2024-09-16T08:57:13.822 INFO:tasks.workunit.client.1.smithi097.stdout:7/568: write d4/d1b/d41/fb0 [990609,34698] 0 2024-09-16T08:57:13.824 INFO:tasks.workunit.client.1.smithi097.stdout:1/685: mknod d5/dce/ce1 0 2024-09-16T08:57:13.829 INFO:tasks.workunit.client.0.smithi031.stdout:3/250: symlink d3/d14/d37/d1a/d1d/d3b/d3f/l46 0 2024-09-16T08:57:13.832 INFO:tasks.workunit.client.0.smithi031.stdout:2/136: getdents d6/d20/d23 0 2024-09-16T08:57:13.832 INFO:tasks.workunit.client.0.smithi031.stdout:2/137: dread - d6/d20/f26 zero size 2024-09-16T08:57:13.832 INFO:tasks.workunit.client.0.smithi031.stdout:2/138: fsync f1 0 2024-09-16T08:57:13.835 INFO:tasks.workunit.client.0.smithi031.stdout:2/139: dread d6/fc [0,4194304] 0 2024-09-16T08:57:13.835 INFO:tasks.workunit.client.0.smithi031.stdout:2/140: fdatasync d6/f9 0 2024-09-16T08:57:13.837 INFO:tasks.workunit.client.1.smithi097.stdout:0/629: dwrite da/df/f16 [0,4194304] 0 2024-09-16T08:57:13.837 INFO:tasks.workunit.client.1.smithi097.stdout:0/630: truncate da/d7e/fdd 768352 0 2024-09-16T08:57:13.837 INFO:tasks.workunit.client.1.smithi097.stdout:0/631: write da/d24/d4c/d80/f3c [1180482,45895] 0 2024-09-16T08:57:13.837 INFO:tasks.workunit.client.0.smithi031.stdout:2/141: read d6/fb [552679,12925] 0 2024-09-16T08:57:13.837 INFO:tasks.workunit.client.0.smithi031.stdout:2/142: chown d6/da 3655 1 2024-09-16T08:57:13.839 INFO:tasks.workunit.client.0.smithi031.stdout:4/108: sync 2024-09-16T08:57:13.839 INFO:tasks.workunit.client.0.smithi031.stdout:4/109: write d0/d2/fe [551984,89979] 0 2024-09-16T08:57:13.865 INFO:tasks.workunit.client.0.smithi031.stdout:9/191: symlink d3/d5/da/l40 0 2024-09-16T08:57:13.897 INFO:tasks.workunit.client.0.smithi031.stdout:8/177: creat d4/d7/f3d x:0 0 0 2024-09-16T08:57:13.897 INFO:tasks.workunit.client.0.smithi031.stdout:8/178: truncate d4/d7/d21/d24/f2a 375266 0 2024-09-16T08:57:13.900 INFO:tasks.workunit.client.0.smithi031.stdout:6/295: rename d0/fb to d0/d17/f55 0 2024-09-16T08:57:13.902 INFO:tasks.workunit.client.0.smithi031.stdout:6/296: dread d0/f15 [0,4194304] 0 2024-09-16T08:57:13.917 INFO:tasks.workunit.client.0.smithi031.stdout:0/228: symlink d6/d12/d13/l54 0 2024-09-16T08:57:13.920 INFO:tasks.workunit.client.0.smithi031.stdout:3/251: mkdir d3/d14/d37/d47 0 2024-09-16T08:57:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:2/143: mkdir d6/d20/d23/d2c 0 2024-09-16T08:57:13.923 INFO:tasks.workunit.client.0.smithi031.stdout:2/144: creat d6/f2d x:0 0 0 2024-09-16T08:57:13.927 INFO:tasks.workunit.client.0.smithi031.stdout:0/229: dread d6/f4b [0,4194304] 0 2024-09-16T08:57:13.928 INFO:tasks.workunit.client.0.smithi031.stdout:2/145: dread d6/f13 [0,4194304] 0 2024-09-16T08:57:13.931 INFO:tasks.workunit.client.0.smithi031.stdout:4/110: getdents d0/d2/d16 0 2024-09-16T08:57:13.936 INFO:tasks.workunit.client.0.smithi031.stdout:9/192: truncate d3/d5/da/d34/f35 3968404 0 2024-09-16T08:57:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:8/179: link d4/d7/d21/l22 d4/d7/d36/d38/l3e 0 2024-09-16T08:57:13.941 INFO:tasks.workunit.client.0.smithi031.stdout:8/180: rename d4/d7/d21/d24 to d4/d7/d21/d24/d3f 22 2024-09-16T08:57:13.943 INFO:tasks.workunit.client.0.smithi031.stdout:6/297: mkdir d0/d10/d14/d56 0 2024-09-16T08:57:13.947 INFO:tasks.workunit.client.1.smithi097.stdout:3/644: mknod d7/de/d8d/cdf 0 2024-09-16T08:57:13.947 INFO:tasks.workunit.client.1.smithi097.stdout:3/645: truncate d7/d28/d30/d88/fc1 583583 0 2024-09-16T08:57:13.947 INFO:tasks.workunit.client.1.smithi097.stdout:3/646: truncate d7/de/d3c/d4f/f7c 167306 0 2024-09-16T08:57:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:0/230: creat d6/d12/d47/f55 x:0 0 0 2024-09-16T08:57:13.953 INFO:tasks.workunit.client.0.smithi031.stdout:2/146: rmdir d6/d20/d23 39 2024-09-16T08:57:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:2/147: write d6/f2d [14450,124876] 0 2024-09-16T08:57:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:0/231: dread d6/f4f [0,4194304] 0 2024-09-16T08:57:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:0/232: fdatasync d6/d12/d13/f24 0 2024-09-16T08:57:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:0/233: fsync d6/d12/d13/f39 0 2024-09-16T08:57:13.954 INFO:tasks.workunit.client.0.smithi031.stdout:0/234: write d6/f9 [3593856,42441] 0 2024-09-16T08:57:13.955 INFO:tasks.workunit.client.0.smithi031.stdout:9/193: link d3/d5/da/f33 d3/d5/da/db/d21/f41 0 2024-09-16T08:57:13.957 INFO:tasks.workunit.client.0.smithi031.stdout:8/181: mkdir d4/d7/d21/d35/d40 0 2024-09-16T08:57:13.960 INFO:tasks.workunit.client.0.smithi031.stdout:9/194: dread d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:4/111: dwrite d0/f24 [0,4194304] 0 2024-09-16T08:57:13.963 INFO:tasks.workunit.client.0.smithi031.stdout:3/252: getdents d3/d14/d37/d1a/d1d 0 2024-09-16T08:57:13.965 INFO:tasks.workunit.client.0.smithi031.stdout:0/235: dread d6/fc [0,4194304] 0 2024-09-16T08:57:13.968 INFO:tasks.workunit.client.0.smithi031.stdout:2/148: mkdir d6/d2e 0 2024-09-16T08:57:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:8/182: creat d4/d2f/f41 x:0 0 0 2024-09-16T08:57:13.971 INFO:tasks.workunit.client.0.smithi031.stdout:8/183: readlink d4/l30 0 2024-09-16T08:57:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:6/298: rmdir d0/d10/d14/d3f 39 2024-09-16T08:57:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:6/299: fdatasync d0/dd/f43 0 2024-09-16T08:57:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:6/300: dread - d0/dd/d2b/f45 zero size 2024-09-16T08:57:13.973 INFO:tasks.workunit.client.0.smithi031.stdout:6/301: readlink d0/d17/l4f 0 2024-09-16T08:57:13.977 INFO:tasks.workunit.client.0.smithi031.stdout:9/195: mkdir d3/d5/da/d34/d3c/d42 0 2024-09-16T08:57:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.980+0000 7f5d182b2640 1 -- 172.21.15.31:0/1444561479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 msgr2=0x7f5d100fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.981+0000 7f5d182b2640 1 --2- 172.21.15.31:0/1444561479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 0x7f5d100fffc0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009880 tx=0x7f5d0c02a4b0 comp rx=0 tx=0).stop 2024-09-16T08:57:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.981+0000 7f5d182b2640 1 -- 172.21.15.31:0/1444561479 shutdown_connections 2024-09-16T08:57:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.981+0000 7f5d182b2640 1 --2- 172.21.15.31:0/1444561479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 0x7f5d10101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.981+0000 7f5d182b2640 1 --2- 172.21.15.31:0/1444561479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 0x7f5d100fffc0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.981+0000 7f5d182b2640 1 -- 172.21.15.31:0/1444561479 >> 172.21.15.31:0/1444561479 conn(0x7f5d100fb350 msgr2=0x7f5d100fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.981+0000 7f5d182b2640 1 -- 172.21.15.31:0/1444561479 shutdown_connections 2024-09-16T08:57:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.981+0000 7f5d182b2640 1 -- 172.21.15.31:0/1444561479 wait complete. 2024-09-16T08:57:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.982+0000 7f5d182b2640 1 Processor -- start 2024-09-16T08:57:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.982+0000 7f5d182b2640 1 -- start start 2024-09-16T08:57:13.986 INFO:tasks.workunit.client.0.smithi031.stdout:8/184: dwrite d4/d7/d36/d38/f3c [0,4194304] 0 2024-09-16T08:57:13.986 INFO:tasks.workunit.client.1.smithi097.stdout:9/573: creat d2/d5/d56/d6b/fb7 x:0 0 0 2024-09-16T08:57:13.986 INFO:tasks.workunit.client.1.smithi097.stdout:9/574: chown d2/d5/d3a/d3c/f53 1 1 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.983+0000 7f5d182b2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 0x7f5d10195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.983+0000 7f5d182b2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 0x7f5d10195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.983+0000 7f5d182b2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10196120 con 0x7f5d100ffbc0 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.983+0000 7f5d182b2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10196290 con 0x7f5d10100dc0 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.984+0000 7f5d1584d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 0x7f5d10195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.984+0000 7f5d1584d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 0x7f5d10195b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:40628/0 (socket says 172.21.15.31:40628) 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.984+0000 7f5d1584d640 1 -- 172.21.15.31:0/2655786152 learned_addr learned my addr 172.21.15.31:0/2655786152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.984+0000 7f5d1604e640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 0x7f5d10195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.984+0000 7f5d1584d640 1 -- 172.21.15.31:0/2655786152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 msgr2=0x7f5d10195610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.984+0000 7f5d1584d640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 0x7f5d10195610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.984+0000 7f5d1584d640 1 -- 172.21.15.31:0/2655786152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c009560 con 0x7f5d10100dc0 2024-09-16T08:57:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.985+0000 7f5d1584d640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 0x7f5d10195b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800eb40 tx=0x7f5d0800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.985+0000 7f5d037fe640 1 -- 172.21.15.31:0/2655786152 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d08010430 con 0x7f5d10100dc0 2024-09-16T08:57:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.985+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1019ad30 con 0x7f5d10100dc0 2024-09-16T08:57:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.985+0000 7f5d037fe640 1 -- 172.21.15.31:0/2655786152 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d08010a50 con 0x7f5d10100dc0 2024-09-16T08:57:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.985+0000 7f5d037fe640 1 -- 172.21.15.31:0/2655786152 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d08018b60 con 0x7f5d10100dc0 2024-09-16T08:57:13.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.985+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d1019b300 con 0x7f5d10100dc0 2024-09-16T08:57:13.990 INFO:tasks.workunit.client.0.smithi031.stdout:3/253: mknod d3/d14/d37/d1a/d1d/d3b/d3f/c48 0 2024-09-16T08:57:13.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.987+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d101087b0 con 0x7f5d10100dc0 2024-09-16T08:57:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.987+0000 7f5d037fe640 1 -- 172.21.15.31:0/2655786152 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f5d08018d10 con 0x7f5d10100dc0 2024-09-16T08:57:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.988+0000 7f5d037fe640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f5cfc073dd0 0x7f5cfc076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.988+0000 7f5d1604e640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f5cfc073dd0 0x7f5cfc076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.989+0000 7f5d037fe640 1 -- 172.21.15.31:0/2655786152 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f5d08014070 con 0x7f5d10100dc0 2024-09-16T08:57:13.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.989+0000 7f5d1604e640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f5cfc073dd0 0x7f5cfc076290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c02aaa0 tx=0x7f5d0c0314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:13.992 INFO:tasks.workunit.client.0.smithi031.stdout:2/149: creat d6/d2e/f2f x:0 0 0 2024-09-16T08:57:13.992 INFO:tasks.workunit.client.1.smithi097.stdout:1/686: rmdir d5/db 39 2024-09-16T08:57:13.992 INFO:tasks.workunit.client.1.smithi097.stdout:1/687: dread - d5/db/dc/fad zero size 2024-09-16T08:57:13.994 INFO:tasks.workunit.client.0.smithi031.stdout:9/196: mkdir d3/d5/da/db/d21/d3b/d43 0 2024-09-16T08:57:13.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:13.993+0000 7f5d037fe640 1 -- 172.21.15.31:0/2655786152 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f5d08055760 con 0x7f5d10100dc0 2024-09-16T08:57:13.996 INFO:tasks.workunit.client.1.smithi097.stdout:5/676: getdents db/d18/d1c/d42/d46/d7e/d8f 0 2024-09-16T08:57:13.997 INFO:tasks.workunit.client.1.smithi097.stdout:5/677: fdatasync db/d18/d1c/d42/f4c 0 2024-09-16T08:57:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:2/150: dread d6/f16 [0,4194304] 0 2024-09-16T08:57:14.000 INFO:tasks.workunit.client.0.smithi031.stdout:2/151: fdatasync f1 0 2024-09-16T08:57:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:3/254: rmdir d3/d14/d37/d1a/d1d/d1e 39 2024-09-16T08:57:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:3/255: fdatasync d3/d14/f2e 0 2024-09-16T08:57:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:2/152: dread d6/f13 [0,4194304] 0 2024-09-16T08:57:14.006 INFO:tasks.workunit.client.0.smithi031.stdout:2/153: chown d6/f8 142 1 2024-09-16T08:57:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:3/256: mkdir d3/d14/d37/d1a/d1d/d3b/d3f/d49 0 2024-09-16T08:57:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:3/257: chown d3/d14/d37/d1a/d1d/f2d 19694231 1 2024-09-16T08:57:14.009 INFO:tasks.workunit.client.0.smithi031.stdout:3/258: fdatasync d3/d14/d37/d33/f13 0 2024-09-16T08:57:14.010 INFO:tasks.workunit.client.1.smithi097.stdout:0/632: fdatasync da/d24/d4c/d80/d3b/f6a 0 2024-09-16T08:57:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:2/154: dread d6/da/f1d [0,4194304] 0 2024-09-16T08:57:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:2/155: read - d6/d20/f26 zero size 2024-09-16T08:57:14.011 INFO:tasks.workunit.client.0.smithi031.stdout:2/156: fsync f1 0 2024-09-16T08:57:14.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:13 smithi097 ceph-mon[35478]: pgmap v8: 65 pgs: 65 active+clean; 1.0 GiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 110 MiB/s rd, 149 MiB/s wr, 322 op/s 2024-09-16T08:57:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:13 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:13 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:14.017 INFO:tasks.workunit.client.0.smithi031.stdout:9/197: dwrite d3/d5/f6 [0,4194304] 0 2024-09-16T08:57:14.018 INFO:tasks.workunit.client.1.smithi097.stdout:1/688: mkdir d5/d2c/dcb/de2 0 2024-09-16T08:57:14.019 INFO:tasks.workunit.client.1.smithi097.stdout:1/689: dread - d5/d14/d95/fac zero size 2024-09-16T08:57:14.019 INFO:tasks.workunit.client.1.smithi097.stdout:1/690: dread - d5/d17/d54/d6b/d71/d94/fd6 zero size 2024-09-16T08:57:14.019 INFO:tasks.workunit.client.1.smithi097.stdout:1/691: readlink d5/d2c/l5f 0 2024-09-16T08:57:14.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/259: dread d3/f4 [0,4194304] 0 2024-09-16T08:57:14.026 INFO:tasks.workunit.client.1.smithi097.stdout:5/678: symlink db/d6e/d79/dd5/le0 0 2024-09-16T08:57:14.027 INFO:tasks.workunit.client.0.smithi031.stdout:9/198: creat d3/d5/da/d34/d3c/d42/f44 x:0 0 0 2024-09-16T08:57:14.028 INFO:tasks.workunit.client.0.smithi031.stdout:9/199: dread d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:14.030 INFO:tasks.workunit.client.1.smithi097.stdout:0/633: mknod da/d1a/d26/ce6 0 2024-09-16T08:57:14.033 INFO:tasks.workunit.client.0.smithi031.stdout:9/200: symlink d3/d5/da/l45 0 2024-09-16T08:57:14.035 INFO:tasks.workunit.client.0.smithi031.stdout:2/157: dwrite f1 [0,4194304] 0 2024-09-16T08:57:14.036 INFO:tasks.workunit.client.1.smithi097.stdout:5/679: unlink db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/f9c 0 2024-09-16T08:57:14.036 INFO:tasks.workunit.client.1.smithi097.stdout:0/634: write da/d24/d4c/d80/d3b/d86/faa [64889,66014] 0 2024-09-16T08:57:14.038 INFO:tasks.workunit.client.0.smithi031.stdout:2/158: mknod d6/d20/d23/d2c/c30 0 2024-09-16T08:57:14.039 INFO:tasks.workunit.client.1.smithi097.stdout:1/692: rmdir d5/d17/d54 39 2024-09-16T08:57:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:2/159: mknod d6/d20/d23/d2c/c31 0 2024-09-16T08:57:14.042 INFO:tasks.workunit.client.0.smithi031.stdout:2/160: truncate d6/f9 4273351 0 2024-09-16T08:57:14.043 INFO:tasks.workunit.client.1.smithi097.stdout:1/693: symlink d5/d70/le3 0 2024-09-16T08:57:14.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/161: rename d6/d20/f2a to d6/d20/f32 0 2024-09-16T08:57:14.046 INFO:tasks.workunit.client.1.smithi097.stdout:3/647: rmdir d7/de/d6c 39 2024-09-16T08:57:14.047 INFO:tasks.workunit.client.1.smithi097.stdout:1/694: symlink d5/d17/d54/d6b/da8/dc5/le4 0 2024-09-16T08:57:14.049 INFO:tasks.workunit.client.1.smithi097.stdout:3/648: creat d7/de/d3c/d3f/d60/dcb/fe0 x:0 0 0 2024-09-16T08:57:14.050 INFO:tasks.workunit.client.1.smithi097.stdout:3/649: truncate d7/de/d3c/f99 811499 0 2024-09-16T08:57:14.051 INFO:tasks.workunit.client.1.smithi097.stdout:3/650: creat d7/de/d3c/d3f/d60/fe1 x:0 0 0 2024-09-16T08:57:14.054 INFO:tasks.workunit.client.1.smithi097.stdout:3/651: dread d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/f78 [0,4194304] 0 2024-09-16T08:57:14.057 INFO:tasks.workunit.client.1.smithi097.stdout:3/652: link d7/de/la9 d7/de/d3c/d3f/d60/dcb/d35/le2 0 2024-09-16T08:57:14.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/653: mknod d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/ce3 0 2024-09-16T08:57:14.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/654: chown d7/de/l10 11599 1 2024-09-16T08:57:14.062 INFO:tasks.workunit.client.0.smithi031.stdout:1/224: rmdir d2/d49 0 2024-09-16T08:57:14.067 INFO:tasks.workunit.client.1.smithi097.stdout:1/695: dwrite d5/db/f74 [0,4194304] 0 2024-09-16T08:57:14.069 INFO:tasks.workunit.client.1.smithi097.stdout:1/696: creat d5/d2c/dcb/de2/fe5 x:0 0 0 2024-09-16T08:57:14.071 INFO:tasks.workunit.client.0.smithi031.stdout:4/112: unlink d0/l9 0 2024-09-16T08:57:14.074 INFO:tasks.workunit.client.0.smithi031.stdout:4/113: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:4/114: dread d0/f18 [0,4194304] 0 2024-09-16T08:57:14.075 INFO:tasks.workunit.client.0.smithi031.stdout:4/115: dread - d0/d2/f1a zero size 2024-09-16T08:57:14.076 INFO:tasks.workunit.client.1.smithi097.stdout:1/697: link d5/d70/c8a d5/d14/ce6 0 2024-09-16T08:57:14.076 INFO:tasks.workunit.client.1.smithi097.stdout:4/605: rename d0/d23/c90 to d0/d23/d5f/cd3 0 2024-09-16T08:57:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/116: creat d0/d2/f27 x:0 0 0 2024-09-16T08:57:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/117: fdatasync d0/d2/d16/f19 0 2024-09-16T08:57:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/118: write d0/d2/d16/f19 [969958,76095] 0 2024-09-16T08:57:14.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/119: write d0/d2/d16/f22 [58218,7487] 0 2024-09-16T08:57:14.078 INFO:tasks.workunit.client.1.smithi097.stdout:1/698: rmdir d5/db/dc 39 2024-09-16T08:57:14.078 INFO:tasks.workunit.client.1.smithi097.stdout:1/699: chown d5/d17/d54/d6b/lbf 6360212 1 2024-09-16T08:57:14.082 INFO:tasks.workunit.client.1.smithi097.stdout:7/569: rename f2 to d4/d7/fbd 0 2024-09-16T08:57:14.082 INFO:tasks.workunit.client.1.smithi097.stdout:7/570: fdatasync d4/d1b/f91 0 2024-09-16T08:57:14.084 INFO:tasks.workunit.client.1.smithi097.stdout:1/700: unlink d5/l3e 0 2024-09-16T08:57:14.085 INFO:tasks.workunit.client.1.smithi097.stdout:7/571: mknod d4/d7/d79/cbe 0 2024-09-16T08:57:14.086 INFO:tasks.workunit.client.1.smithi097.stdout:7/572: write d4/d9/d1c/f2c [4251395,124273] 0 2024-09-16T08:57:14.087 INFO:tasks.workunit.client.1.smithi097.stdout:1/701: symlink d5/d2c/dcb/de2/le7 0 2024-09-16T08:57:14.088 INFO:tasks.workunit.client.1.smithi097.stdout:1/702: fdatasync d5/d14/d95/fd8 0 2024-09-16T08:57:14.088 INFO:tasks.workunit.client.1.smithi097.stdout:1/703: readlink d5/db/dc/l39 0 2024-09-16T08:57:14.088 INFO:tasks.workunit.client.1.smithi097.stdout:1/704: readlink d5/d2c/d3b/l5c 0 2024-09-16T08:57:14.088 INFO:tasks.workunit.client.1.smithi097.stdout:1/705: chown d5/d70/d9d 8853178 1 2024-09-16T08:57:14.090 INFO:tasks.workunit.client.1.smithi097.stdout:1/706: link d5/d14/ce6 d5/d17/d54/d6b/d71/dd3/ce8 0 2024-09-16T08:57:14.091 INFO:tasks.workunit.client.1.smithi097.stdout:1/707: read d5/db/dc/d19/f4d [217265,7095] 0 2024-09-16T08:57:14.091 INFO:tasks.workunit.client.1.smithi097.stdout:1/708: dread - d5/f35 zero size 2024-09-16T08:57:14.093 INFO:tasks.workunit.client.1.smithi097.stdout:1/709: read d5/d2c/d3b/fb7 [157176,44170] 0 2024-09-16T08:57:14.093 INFO:tasks.workunit.client.1.smithi097.stdout:1/710: chown d5/d17/d54/ca9 123048 1 2024-09-16T08:57:14.093 INFO:tasks.workunit.client.1.smithi097.stdout:1/711: truncate d5/d17/d54/d6b/f9c 947425 0 2024-09-16T08:57:14.093 INFO:tasks.workunit.client.0.smithi031.stdout:0/236: rename d6/f4b to d6/d12/d13/d36/f56 0 2024-09-16T08:57:14.094 INFO:tasks.workunit.client.1.smithi097.stdout:1/712: truncate d5/db/dc/d19/f3f 2832042 0 2024-09-16T08:57:14.094 INFO:tasks.workunit.client.1.smithi097.stdout:1/713: chown d5/d70/c8a 9053146 1 2024-09-16T08:57:14.094 INFO:tasks.workunit.client.1.smithi097.stdout:1/714: readlink d5/d2c/l73 0 2024-09-16T08:57:14.095 INFO:tasks.workunit.client.0.smithi031.stdout:6/302: rename d0/d17/d19/c3a to d0/d10/d14/c57 0 2024-09-16T08:57:14.096 INFO:tasks.workunit.client.1.smithi097.stdout:1/715: mknod d5/ddd/ce9 0 2024-09-16T08:57:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:8/185: rename d4/c2b to d4/d34/c42 0 2024-09-16T08:57:14.098 INFO:tasks.workunit.client.0.smithi031.stdout:3/260: rename d3/d14/d37 to d3/d14/d37/d1a/d1d/d3b/d3f/d49/d4a 22 2024-09-16T08:57:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:6/303: creat d0/d10/f58 x:0 0 0 2024-09-16T08:57:14.099 INFO:tasks.workunit.client.0.smithi031.stdout:6/304: chown d0/dd/f22 724352 1 2024-09-16T08:57:14.099 INFO:tasks.workunit.client.1.smithi097.stdout:4/606: dwrite d0/d23/d5f/d7d/dbc/fbd [0,4194304] 0 2024-09-16T08:57:14.099 INFO:tasks.workunit.client.1.smithi097.stdout:4/607: truncate d0/d7/d52/d26/d8c/fb2 142664 0 2024-09-16T08:57:14.102 INFO:tasks.workunit.client.0.smithi031.stdout:3/261: link d3/d14/d37/d1a/d1d/d3b/d3f/l46 d3/d14/d37/d33/l4b 0 2024-09-16T08:57:14.104 INFO:tasks.workunit.client.0.smithi031.stdout:0/237: read d6/d12/f16 [5681588,72149] 0 2024-09-16T08:57:14.106 INFO:tasks.workunit.client.0.smithi031.stdout:9/201: rename d3/d5/da/f27 to d3/d5/da/d34/f46 0 2024-09-16T08:57:14.106 INFO:tasks.workunit.client.1.smithi097.stdout:4/608: rmdir d0/d7/d52/d26/d3d 39 2024-09-16T08:57:14.106 INFO:tasks.workunit.client.1.smithi097.stdout:4/609: fdatasync d0/d7/d52/d26/d3d/f44 0 2024-09-16T08:57:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:8/186: rmdir d4/d7/d36/d38 39 2024-09-16T08:57:14.107 INFO:tasks.workunit.client.0.smithi031.stdout:8/187: fsync d4/f29 0 2024-09-16T08:57:14.109 INFO:tasks.workunit.client.1.smithi097.stdout:4/610: mknod d0/d23/d5f/d7d/dbc/cd4 0 2024-09-16T08:57:14.111 INFO:tasks.workunit.client.1.smithi097.stdout:4/611: stat d0/d23/c69 0 2024-09-16T08:57:14.113 INFO:tasks.workunit.client.1.smithi097.stdout:4/612: mknod d0/d23/d5f/cd5 0 2024-09-16T08:57:14.115 INFO:tasks.workunit.client.0.smithi031.stdout:3/262: read d3/d14/d37/d1a/f26 [3027184,51423] 0 2024-09-16T08:57:14.116 INFO:tasks.workunit.client.0.smithi031.stdout:2/162: getdents d6/d20/d23/d2c 0 2024-09-16T08:57:14.116 INFO:tasks.workunit.client.1.smithi097.stdout:4/613: rmdir d0/d7/df/d15 39 2024-09-16T08:57:14.120 INFO:tasks.workunit.client.1.smithi097.stdout:4/614: creat d0/d23/d5f/d7d/fd6 x:0 0 0 2024-09-16T08:57:14.122 INFO:tasks.workunit.client.0.smithi031.stdout:2/163: link d6/f13 d6/d20/f33 0 2024-09-16T08:57:14.122 INFO:tasks.workunit.client.1.smithi097.stdout:4/615: dread d0/d7/df/d83/fa5 [0,4194304] 0 2024-09-16T08:57:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:13 smithi031 ceph-mon[27844]: pgmap v8: 65 pgs: 65 active+clean; 1.0 GiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 110 MiB/s rd, 149 MiB/s wr, 322 op/s 2024-09-16T08:57:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:13 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:13 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:14.124 INFO:tasks.workunit.client.0.smithi031.stdout:2/164: creat d6/d20/d23/d2c/f34 x:0 0 0 2024-09-16T08:57:14.125 INFO:tasks.workunit.client.1.smithi097.stdout:4/616: creat d0/d36/d4c/fd7 x:0 0 0 2024-09-16T08:57:14.127 INFO:tasks.workunit.client.1.smithi097.stdout:5/680: symlink db/d22/le1 0 2024-09-16T08:57:14.128 INFO:tasks.workunit.client.1.smithi097.stdout:5/681: write db/d18/d20/d2b/f87 [706396,130173] 0 2024-09-16T08:57:14.130 INFO:tasks.workunit.client.1.smithi097.stdout:6/625: sync 2024-09-16T08:57:14.130 INFO:tasks.workunit.client.1.smithi097.stdout:8/603: sync 2024-09-16T08:57:14.131 INFO:tasks.workunit.client.1.smithi097.stdout:2/630: sync 2024-09-16T08:57:14.131 INFO:tasks.workunit.client.1.smithi097.stdout:2/631: chown d3/d4/d68/d8b/d7c/d7d/l86 124 1 2024-09-16T08:57:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:0/238: dwrite d6/f42 [0,4194304] 0 2024-09-16T08:57:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:0/239: write d6/d12/d47/f4e [4298877,127561] 0 2024-09-16T08:57:14.132 INFO:tasks.workunit.client.0.smithi031.stdout:0/240: readlink d6/d12/l28 0 2024-09-16T08:57:14.134 INFO:tasks.workunit.client.0.smithi031.stdout:4/120: rmdir d0 39 2024-09-16T08:57:14.135 INFO:tasks.workunit.client.1.smithi097.stdout:8/604: mknod d8/dd/d71/cbc 0 2024-09-16T08:57:14.135 INFO:tasks.workunit.client.1.smithi097.stdout:0/635: creat da/d24/d4c/fe7 x:0 0 0 2024-09-16T08:57:14.135 INFO:tasks.workunit.client.1.smithi097.stdout:2/632: truncate d3/d4/d68/d8b/d7c/d7d/d53/d47/f4b 3809771 0 2024-09-16T08:57:14.135 INFO:tasks.workunit.client.1.smithi097.stdout:2/633: chown d3/d4/d9/f4f 0 1 2024-09-16T08:57:14.136 INFO:tasks.workunit.client.1.smithi097.stdout:8/605: readlink d8/dd/d15/d2c/d4a/d4d/d3d/lba 0 2024-09-16T08:57:14.136 INFO:tasks.workunit.client.1.smithi097.stdout:8/606: write d8/dd/d15/d2c/d4a/d4d/f49 [3476820,19164] 0 2024-09-16T08:57:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:0/241: dread d6/f9 [0,4194304] 0 2024-09-16T08:57:14.139 INFO:tasks.workunit.client.0.smithi031.stdout:0/242: dread - d6/f31 zero size 2024-09-16T08:57:14.140 INFO:tasks.workunit.client.1.smithi097.stdout:0/636: mknod da/d24/d4c/d80/d3b/d86/ce8 0 2024-09-16T08:57:14.140 INFO:tasks.workunit.client.1.smithi097.stdout:0/637: stat da/d1a/d26/d47/db3/fd4 0 2024-09-16T08:57:14.140 INFO:tasks.workunit.client.1.smithi097.stdout:6/626: dread d1/dc/d1e/d9e/dca/d81/dad/d5a/f47 [0,4194304] 0 2024-09-16T08:57:14.140 INFO:tasks.workunit.client.1.smithi097.stdout:6/627: truncate d1/dc/d1e/f84 197781 0 2024-09-16T08:57:14.145 INFO:tasks.workunit.client.0.smithi031.stdout:4/121: symlink d0/d2/l28 0 2024-09-16T08:57:14.146 INFO:tasks.workunit.client.1.smithi097.stdout:2/634: rename d3/d4/d9/f4c to d3/d4/d68/d8b/d7c/d7d/d53/da3/fbb 0 2024-09-16T08:57:14.146 INFO:tasks.workunit.client.1.smithi097.stdout:2/635: stat d3/d4/d68/d8b/d7c/d7d/fb5 0 2024-09-16T08:57:14.146 INFO:tasks.workunit.client.1.smithi097.stdout:2/636: dread - d3/d4/d68/d8b/d7c/d7d/d53/f50 zero size 2024-09-16T08:57:14.147 INFO:tasks.workunit.client.1.smithi097.stdout:2/637: read d3/d4/d8/d51/f63 [330877,72993] 0 2024-09-16T08:57:14.147 INFO:tasks.workunit.client.1.smithi097.stdout:2/638: stat d3/d4/d68/d8b/d7c/d7d/f8c 0 2024-09-16T08:57:14.148 INFO:tasks.workunit.client.0.smithi031.stdout:4/122: symlink d0/d2/l29 0 2024-09-16T08:57:14.151 INFO:tasks.workunit.client.1.smithi097.stdout:6/628: creat d1/db7/fce x:0 0 0 2024-09-16T08:57:14.151 INFO:tasks.workunit.client.1.smithi097.stdout:6/629: chown d1/lb1 11 1 2024-09-16T08:57:14.151 INFO:tasks.workunit.client.1.smithi097.stdout:1/716: write d5/d17/f24 [1815406,55321] 0 2024-09-16T08:57:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:3/263: dwrite d3/d14/d37/d33/f38 [0,4194304] 0 2024-09-16T08:57:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:6/305: rename d0/d10/d14 to d0/dd/d35/d59 0 2024-09-16T08:57:14.157 INFO:tasks.workunit.client.0.smithi031.stdout:6/306: creat d0/d10/f5a x:0 0 0 2024-09-16T08:57:14.157 INFO:tasks.workunit.client.1.smithi097.stdout:8/607: rename d8/dd/f80 to d8/dd/d15/d2c/d4a/d4e/db2/fbd 0 2024-09-16T08:57:14.157 INFO:tasks.workunit.client.1.smithi097.stdout:8/608: dread - d8/dd/d42/f9e zero size 2024-09-16T08:57:14.158 INFO:tasks.workunit.client.1.smithi097.stdout:8/609: readlink d8/dd/d15/d2c/l9f 0 2024-09-16T08:57:14.159 INFO:tasks.workunit.client.0.smithi031.stdout:2/165: dwrite d6/da/f27 [0,4194304] 0 2024-09-16T08:57:14.159 INFO:tasks.workunit.client.1.smithi097.stdout:2/639: link d3/d4/d68/d8b/d7c/d7d/f6d d3/d4/d9/d67/db2/fbc 0 2024-09-16T08:57:14.159 INFO:tasks.workunit.client.1.smithi097.stdout:4/617: dwrite d0/d7/fb [8388608,4194304] 0 2024-09-16T08:57:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:8/188: rmdir d4 39 2024-09-16T08:57:14.161 INFO:tasks.workunit.client.0.smithi031.stdout:8/189: fdatasync d4/d7/f15 0 2024-09-16T08:57:14.161 INFO:tasks.workunit.client.1.smithi097.stdout:5/682: dwrite db/d18/d1c/d42/f4c [4194304,4194304] 0 2024-09-16T08:57:14.162 INFO:tasks.workunit.client.1.smithi097.stdout:0/638: rmdir da/d1a/d26/d47/d97/d84 39 2024-09-16T08:57:14.162 INFO:tasks.workunit.client.1.smithi097.stdout:0/639: stat da/d1a/d26/d47/db3/cb1 0 2024-09-16T08:57:14.162 INFO:tasks.workunit.client.1.smithi097.stdout:0/640: fdatasync da/d1a/d26/d47/db3/d60/d68/f5d 0 2024-09-16T08:57:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:3/264: symlink d3/d14/d37/d1a/d24/l4c 0 2024-09-16T08:57:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:3/265: stat d3/d14/f2e 0 2024-09-16T08:57:14.167 INFO:tasks.workunit.client.0.smithi031.stdout:3/266: creat d3/d14/d37/d33/f4d x:0 0 0 2024-09-16T08:57:14.168 INFO:tasks.workunit.client.0.smithi031.stdout:4/123: write d0/f24 [2403657,80762] 0 2024-09-16T08:57:14.170 INFO:tasks.workunit.client.1.smithi097.stdout:6/630: unlink d1/dc/d1e/d9e/dca/d81/dad/d5a/f32 0 2024-09-16T08:57:14.172 INFO:tasks.workunit.client.0.smithi031.stdout:9/202: rename d3/d5/l1b to d3/d5/da/db/l47 0 2024-09-16T08:57:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:9/203: chown d3/fc 553400 1 2024-09-16T08:57:14.173 INFO:tasks.workunit.client.0.smithi031.stdout:9/204: write f2 [1142325,41622] 0 2024-09-16T08:57:14.177 INFO:tasks.workunit.client.1.smithi097.stdout:1/717: symlink d5/d17/d54/d6b/da8/dc5/dc7/lea 0 2024-09-16T08:57:14.179 INFO:tasks.workunit.client.0.smithi031.stdout:6/307: mkdir d0/dd/d35/d5b 0 2024-09-16T08:57:14.181 INFO:tasks.workunit.client.1.smithi097.stdout:0/641: write da/d24/d4c/f9d [105279,103467] 0 2024-09-16T08:57:14.182 INFO:tasks.workunit.client.1.smithi097.stdout:1/718: dread d5/d2c/dcb/f53 [0,4194304] 0 2024-09-16T08:57:14.183 INFO:tasks.workunit.client.1.smithi097.stdout:5/683: rename db/d18/f34 to db/d1a/fe2 0 2024-09-16T08:57:14.185 INFO:tasks.workunit.client.1.smithi097.stdout:6/631: creat d1/dc/d1e/d9e/dca/d81/dc4/dc6/fcf x:0 0 0 2024-09-16T08:57:14.188 INFO:tasks.workunit.client.0.smithi031.stdout:8/190: creat d4/f43 x:0 0 0 2024-09-16T08:57:14.188 INFO:tasks.workunit.client.0.smithi031.stdout:2/166: dwrite d6/fc [0,4194304] 0 2024-09-16T08:57:14.189 INFO:tasks.workunit.client.1.smithi097.stdout:8/610: dwrite d8/dd/d15/d2c/d4a/d4e/db2/fb3 [0,4194304] 0 2024-09-16T08:57:14.190 INFO:tasks.workunit.client.1.smithi097.stdout:2/640: dwrite d3/f23 [0,4194304] 0 2024-09-16T08:57:14.194 INFO:tasks.workunit.client.0.smithi031.stdout:3/267: link d3/d14/c22 d3/d14/d37/d1a/d24/c4e 0 2024-09-16T08:57:14.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/124: rename d0/d2/d16/c25 to d0/c2a 0 2024-09-16T08:57:14.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/125: dread - d0/d2/d16/f21 zero size 2024-09-16T08:57:14.201 INFO:tasks.workunit.client.1.smithi097.stdout:0/642: mknod da/d1a/d26/d94/dc9/dcb/ce9 0 2024-09-16T08:57:14.202 INFO:tasks.workunit.client.0.smithi031.stdout:4/126: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:14.202 INFO:tasks.workunit.client.0.smithi031.stdout:4/127: fdatasync d0/d2/d16/f21 0 2024-09-16T08:57:14.202 INFO:tasks.workunit.client.0.smithi031.stdout:4/128: readlink d0/l3 0 2024-09-16T08:57:14.203 INFO:tasks.workunit.client.0.smithi031.stdout:9/205: mkdir d3/d48 0 2024-09-16T08:57:14.204 INFO:tasks.workunit.client.1.smithi097.stdout:1/719: mkdir d5/db/deb 0 2024-09-16T08:57:14.204 INFO:tasks.workunit.client.1.smithi097.stdout:1/720: stat d5/d2c/d3b 0 2024-09-16T08:57:14.204 INFO:tasks.workunit.client.1.smithi097.stdout:1/721: chown d5/db/f34 60044647 1 2024-09-16T08:57:14.204 INFO:tasks.workunit.client.1.smithi097.stdout:1/722: dread - d5/db/dc/fe0 zero size 2024-09-16T08:57:14.204 INFO:tasks.workunit.client.1.smithi097.stdout:1/723: chown d5/dce 2188898 1 2024-09-16T08:57:14.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/308: symlink d0/l5c 0 2024-09-16T08:57:14.206 INFO:tasks.workunit.client.1.smithi097.stdout:5/684: mknod db/d6e/d79/dba/ce3 0 2024-09-16T08:57:14.206 INFO:tasks.workunit.client.1.smithi097.stdout:5/685: dread - db/d18/d1c/d42/d46/d7e/d8f/fd8 zero size 2024-09-16T08:57:14.206 INFO:tasks.workunit.client.1.smithi097.stdout:5/686: chown db/d18/d1c/d42 0 1 2024-09-16T08:57:14.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/167: creat d6/d20/d23/d28/f35 x:0 0 0 2024-09-16T08:57:14.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/168: creat d6/f36 x:0 0 0 2024-09-16T08:57:14.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/169: readlink d6/da/l14 0 2024-09-16T08:57:14.207 INFO:tasks.workunit.client.0.smithi031.stdout:2/170: write d6/da/f15 [933312,41235] 0 2024-09-16T08:57:14.208 INFO:tasks.workunit.client.1.smithi097.stdout:6/632: symlink d1/dc/d1e/d9e/dca/d81/dad/ld0 0 2024-09-16T08:57:14.209 INFO:tasks.workunit.client.0.smithi031.stdout:8/191: dwrite d4/f27 [0,4194304] 0 2024-09-16T08:57:14.209 INFO:tasks.workunit.client.0.smithi031.stdout:3/268: rename d3/d14/d37/d33/f13 to d3/d14/d37/d1a/d1d/d3b/f4f 0 2024-09-16T08:57:14.213 INFO:tasks.workunit.client.0.smithi031.stdout:4/129: mkdir d0/d2/d16/d2b 0 2024-09-16T08:57:14.216 INFO:tasks.workunit.client.0.smithi031.stdout:9/206: creat d3/d48/f49 x:0 0 0 2024-09-16T08:57:14.220 INFO:tasks.workunit.client.1.smithi097.stdout:8/611: creat d8/dd/d15/d2c/d4a/d4d/fbe x:0 0 0 2024-09-16T08:57:14.220 INFO:tasks.workunit.client.1.smithi097.stdout:8/612: chown d8/dd/d15/l6f 3537 1 2024-09-16T08:57:14.221 INFO:tasks.workunit.client.0.smithi031.stdout:2/171: mkdir d6/da/d37 0 2024-09-16T08:57:14.222 INFO:tasks.workunit.client.0.smithi031.stdout:2/172: chown d6/da/c21 1013102726 1 2024-09-16T08:57:14.226 INFO:tasks.workunit.client.1.smithi097.stdout:2/641: creat d3/d4/d68/d7f/fbd x:0 0 0 2024-09-16T08:57:14.228 INFO:tasks.workunit.client.1.smithi097.stdout:0/643: creat da/d1a/d41/fea x:0 0 0 2024-09-16T08:57:14.228 INFO:tasks.workunit.client.0.smithi031.stdout:7/288: sync 2024-09-16T08:57:14.228 INFO:tasks.workunit.client.0.smithi031.stdout:5/252: sync 2024-09-16T08:57:14.228 INFO:tasks.workunit.client.0.smithi031.stdout:5/253: chown d2/dc/f3b 10982910 1 2024-09-16T08:57:14.229 INFO:tasks.workunit.client.0.smithi031.stdout:8/192: mkdir d4/d34/d44 0 2024-09-16T08:57:14.230 INFO:tasks.workunit.client.1.smithi097.stdout:1/724: symlink d5/db/dc/d19/lec 0 2024-09-16T08:57:14.232 INFO:tasks.workunit.client.0.smithi031.stdout:9/207: symlink d3/l4a 0 2024-09-16T08:57:14.233 INFO:tasks.workunit.client.1.smithi097.stdout:5/687: truncate db/d18/d1c/d42/d46/f47 763416 0 2024-09-16T08:57:14.233 INFO:tasks.workunit.client.1.smithi097.stdout:5/688: stat db/d18/d1c/d42/fd7 0 2024-09-16T08:57:14.235 INFO:tasks.workunit.client.0.smithi031.stdout:4/130: dwrite d0/f1 [0,4194304] 0 2024-09-16T08:57:14.235 INFO:tasks.workunit.client.0.smithi031.stdout:4/131: dread - d0/d2/f27 zero size 2024-09-16T08:57:14.235 INFO:tasks.workunit.client.0.smithi031.stdout:3/269: dwrite d3/f35 [0,4194304] 0 2024-09-16T08:57:14.236 INFO:tasks.workunit.client.0.smithi031.stdout:6/309: dwrite d0/d17/f30 [0,4194304] 0 2024-09-16T08:57:14.236 INFO:tasks.workunit.client.0.smithi031.stdout:7/289: rename d2/d8/d61 to d2/d8/d11/d66 0 2024-09-16T08:57:14.238 INFO:tasks.workunit.client.1.smithi097.stdout:0/644: dread da/d1a/d26/d47/db3/d60/d73/f77 [0,4194304] 0 2024-09-16T08:57:14.238 INFO:tasks.workunit.client.1.smithi097.stdout:0/645: truncate da/df/d8a/fa9 104611 0 2024-09-16T08:57:14.239 INFO:tasks.workunit.client.0.smithi031.stdout:5/254: symlink d2/dc/d3d/l4d 0 2024-09-16T08:57:14.239 INFO:tasks.workunit.client.0.smithi031.stdout:9/208: dread d3/d5/da/db/d21/f3d [0,4194304] 0 2024-09-16T08:57:14.241 INFO:tasks.workunit.client.0.smithi031.stdout:8/193: link d4/cc d4/d2f/c45 0 2024-09-16T08:57:14.242 INFO:tasks.workunit.client.1.smithi097.stdout:6/633: creat d1/dc/d1e/d9e/dca/dbb/fd1 x:0 0 0 2024-09-16T08:57:14.242 INFO:tasks.workunit.client.1.smithi097.stdout:6/634: write d1/dc/d1e/d9e/dca/d16/d4f/d9f/fba [76845,94875] 0 2024-09-16T08:57:14.243 INFO:tasks.workunit.client.1.smithi097.stdout:8/613: mknod d8/dd/d15/d2c/d4a/cbf 0 2024-09-16T08:57:14.244 INFO:tasks.workunit.client.0.smithi031.stdout:2/173: getdents d6/d20/d23/d28 0 2024-09-16T08:57:14.246 INFO:tasks.workunit.client.1.smithi097.stdout:2/642: symlink d3/d1b/lbe 0 2024-09-16T08:57:14.247 INFO:tasks.workunit.client.1.smithi097.stdout:8/614: dread d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f8b [0,4194304] 0 2024-09-16T08:57:14.249 INFO:tasks.workunit.client.1.smithi097.stdout:1/725: rmdir d5/dce 39 2024-09-16T08:57:14.249 INFO:tasks.workunit.client.1.smithi097.stdout:1/726: chown d5/d14/c67 128708 1 2024-09-16T08:57:14.250 INFO:tasks.workunit.client.0.smithi031.stdout:4/132: mknod d0/d2/d16/c2c 0 2024-09-16T08:57:14.250 INFO:tasks.workunit.client.0.smithi031.stdout:4/133: write d0/d2/d16/d1e/f23 [597148,111053] 0 2024-09-16T08:57:14.251 INFO:tasks.workunit.client.1.smithi097.stdout:5/689: getdents db/d18/d20 0 2024-09-16T08:57:14.253 INFO:tasks.workunit.client.1.smithi097.stdout:5/690: dread db/d18/d1c/d42/d46/d7e/d64/d7f/f85 [0,4194304] 0 2024-09-16T08:57:14.253 INFO:tasks.workunit.client.1.smithi097.stdout:0/646: getdents da/d1a/d26/d94 0 2024-09-16T08:57:14.262 INFO:tasks.workunit.client.0.smithi031.stdout:3/270: mknod d3/d14/c50 0 2024-09-16T08:57:14.263 INFO:tasks.workunit.client.1.smithi097.stdout:2/643: mkdir d3/d1b/dbf 0 2024-09-16T08:57:14.263 INFO:tasks.workunit.client.1.smithi097.stdout:5/691: symlink db/d1a/d6d/dc6/le4 0 2024-09-16T08:57:14.266 INFO:tasks.workunit.client.0.smithi031.stdout:6/310: link d0/dd/ce d0/d17/d19/c5d 0 2024-09-16T08:57:14.266 INFO:tasks.workunit.client.0.smithi031.stdout:6/311: getdents d0/dd/d35/d59/d56 0 2024-09-16T08:57:14.266 INFO:tasks.workunit.client.1.smithi097.stdout:1/727: dread d5/d17/d54/d6b/d71/d94/f99 [0,4194304] 0 2024-09-16T08:57:14.266 INFO:tasks.workunit.client.1.smithi097.stdout:1/728: write d5/fd9 [663316,53995] 0 2024-09-16T08:57:14.266 INFO:tasks.workunit.client.1.smithi097.stdout:1/729: fsync d5/d70/fc4 0 2024-09-16T08:57:14.266 INFO:tasks.workunit.client.1.smithi097.stdout:9/575: sync 2024-09-16T08:57:14.268 INFO:tasks.workunit.client.1.smithi097.stdout:0/647: rename da/d24/d4b/l64 to da/df/leb 0 2024-09-16T08:57:14.272 INFO:tasks.workunit.client.1.smithi097.stdout:6/635: rmdir d1/dc/d1e/d9e/dca/d81/dc4/dc6 39 2024-09-16T08:57:14.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/271: dread d3/d14/d37/f10 [0,4194304] 0 2024-09-16T08:57:14.272 INFO:tasks.workunit.client.0.smithi031.stdout:3/272: chown d3 1082 1 2024-09-16T08:57:14.274 INFO:tasks.workunit.client.1.smithi097.stdout:7/573: sync 2024-09-16T08:57:14.274 INFO:tasks.workunit.client.1.smithi097.stdout:4/618: sync 2024-09-16T08:57:14.274 INFO:tasks.workunit.client.1.smithi097.stdout:3/655: sync 2024-09-16T08:57:14.276 INFO:tasks.workunit.client.0.smithi031.stdout:5/255: link d2/dc/d25/d21/d2e/f48 d2/dc/d25/d21/f4e 0 2024-09-16T08:57:14.279 INFO:tasks.workunit.client.0.smithi031.stdout:9/209: creat d3/d5/da/db/d21/d3b/f4b x:0 0 0 2024-09-16T08:57:14.279 INFO:tasks.workunit.client.0.smithi031.stdout:9/210: chown d3/d5/da/f33 1 1 2024-09-16T08:57:14.280 INFO:tasks.workunit.client.1.smithi097.stdout:2/644: rename d3/d4/d68/d8b/f60 to d3/d4/d68/d8b/d61/dac/fc0 0 2024-09-16T08:57:14.280 INFO:tasks.workunit.client.1.smithi097.stdout:2/645: write d3/d4/d8/d51/f63 [1755690,79500] 0 2024-09-16T08:57:14.281 INFO:tasks.workunit.client.0.smithi031.stdout:8/194: rmdir d4/d7/d21 39 2024-09-16T08:57:14.281 INFO:tasks.workunit.client.0.smithi031.stdout:8/195: readlink d4/l1f 0 2024-09-16T08:57:14.282 INFO:tasks.workunit.client.1.smithi097.stdout:0/648: creat da/d1a/d26/d47/db3/d60/d73/d85/dad/fec x:0 0 0 2024-09-16T08:57:14.283 INFO:tasks.workunit.client.0.smithi031.stdout:7/290: rmdir d2/d2d 39 2024-09-16T08:57:14.283 INFO:tasks.workunit.client.0.smithi031.stdout:7/291: chown d2/d8/d11/l20 27552968 1 2024-09-16T08:57:14.284 INFO:tasks.workunit.client.1.smithi097.stdout:6/636: link d1/dc/d1e/d9e/dca/c9c d1/dc/d3c/cd2 0 2024-09-16T08:57:14.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/273: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/l51 0 2024-09-16T08:57:14.285 INFO:tasks.workunit.client.0.smithi031.stdout:3/274: readlink d3/d14/d37/d1a/d24/l32 0 2024-09-16T08:57:14.287 INFO:tasks.workunit.client.0.smithi031.stdout:2/174: dwrite d6/f16 [0,4194304] 0 2024-09-16T08:57:14.289 INFO:tasks.workunit.client.0.smithi031.stdout:4/134: dwrite d0/d2/d16/f22 [0,4194304] 0 2024-09-16T08:57:14.289 INFO:tasks.workunit.client.1.smithi097.stdout:3/656: creat d7/de/d3c/fe4 x:0 0 0 2024-09-16T08:57:14.290 INFO:tasks.workunit.client.1.smithi097.stdout:8/615: dwrite d8/dd/d15/f34 [0,4194304] 0 2024-09-16T08:57:14.292 INFO:tasks.workunit.client.0.smithi031.stdout:5/256: rmdir d2/dc/d25/d21/d49 0 2024-09-16T08:57:14.293 INFO:tasks.workunit.client.0.smithi031.stdout:6/312: dwrite d0/dd/d35/f52 [0,4194304] 0 2024-09-16T08:57:14.294 INFO:tasks.workunit.client.1.smithi097.stdout:7/574: creat d4/d9/d1c/fbf x:0 0 0 2024-09-16T08:57:14.296 INFO:tasks.workunit.client.0.smithi031.stdout:9/211: rename d3/c4 to d3/d5/da/d34/d3c/d42/c4c 0 2024-09-16T08:57:14.298 INFO:tasks.workunit.client.1.smithi097.stdout:4/619: creat d0/d35/d6c/fd8 x:0 0 0 2024-09-16T08:57:14.298 INFO:tasks.workunit.client.1.smithi097.stdout:4/620: creat d0/d35/d6c/fd9 x:0 0 0 2024-09-16T08:57:14.299 INFO:tasks.workunit.client.1.smithi097.stdout:9/576: dwrite d2/d5/d72/fab [0,4194304] 0 2024-09-16T08:57:14.299 INFO:tasks.workunit.client.1.smithi097.stdout:9/577: chown d2/d5/d72/da2 32214 1 2024-09-16T08:57:14.299 INFO:tasks.workunit.client.0.smithi031.stdout:2/175: dread d6/fc [0,4194304] 0 2024-09-16T08:57:14.300 INFO:tasks.workunit.client.1.smithi097.stdout:4/621: dread d0/d7/d52/d26/d3d/f82 [0,4194304] 0 2024-09-16T08:57:14.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/292: creat d2/d2d/f67 x:0 0 0 2024-09-16T08:57:14.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/293: chown d2/d8/d11/c37 2946040 1 2024-09-16T08:57:14.301 INFO:tasks.workunit.client.0.smithi031.stdout:7/294: read d2/d8/d11/f24 [423101,3057] 0 2024-09-16T08:57:14.301 INFO:tasks.workunit.client.0.smithi031.stdout:7/295: dread - d2/d8/d11/f55 zero size 2024-09-16T08:57:14.301 INFO:tasks.workunit.client.0.smithi031.stdout:7/296: dread - d2/d2d/f43 zero size 2024-09-16T08:57:14.301 INFO:tasks.workunit.client.0.smithi031.stdout:7/297: fdatasync d2/f56 0 2024-09-16T08:57:14.301 INFO:tasks.workunit.client.0.smithi031.stdout:7/298: stat d2/d2d/f2e 0 2024-09-16T08:57:14.302 INFO:tasks.workunit.client.1.smithi097.stdout:5/692: link db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/f99 db/d18/fe5 0 2024-09-16T08:57:14.303 INFO:tasks.workunit.client.1.smithi097.stdout:8/616: dread d8/dd/d15/d16/f95 [0,4194304] 0 2024-09-16T08:57:14.303 INFO:tasks.workunit.client.1.smithi097.stdout:8/617: truncate d8/dd/d15/d2c/d4a/d4e/f70 410641 0 2024-09-16T08:57:14.306 INFO:tasks.workunit.client.1.smithi097.stdout:1/730: rename d5/d17/d54/d6b/d71/d94/fd6 to d5/d14/fed 0 2024-09-16T08:57:14.307 INFO:tasks.workunit.client.0.smithi031.stdout:6/313: rmdir d0/dd/d35 39 2024-09-16T08:57:14.307 INFO:tasks.workunit.client.1.smithi097.stdout:0/649: mknod da/d1a/d41/db2/ced 0 2024-09-16T08:57:14.308 INFO:tasks.workunit.client.1.smithi097.stdout:0/650: truncate da/d7e/dc4/fe0 712773 0 2024-09-16T08:57:14.308 INFO:tasks.workunit.client.1.smithi097.stdout:0/651: creat da/d1a/d26/d47/fee x:0 0 0 2024-09-16T08:57:14.308 INFO:tasks.workunit.client.1.smithi097.stdout:0/652: fsync da/df/f16 0 2024-09-16T08:57:14.309 INFO:tasks.workunit.client.0.smithi031.stdout:5/257: rename d2/fb to d2/dc/d25/d21/f4f 0 2024-09-16T08:57:14.311 INFO:tasks.workunit.client.1.smithi097.stdout:3/657: symlink d7/d28/d30/d69/le5 0 2024-09-16T08:57:14.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/212: mkdir d3/d5/da/d34/d4d 0 2024-09-16T08:57:14.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/213: dread - d3/d5/da/db/d21/f41 zero size 2024-09-16T08:57:14.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/214: chown d3/d5/da/db/d21/d3b/f4b 4041 1 2024-09-16T08:57:14.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/215: creat d3/d48/f4e x:0 0 0 2024-09-16T08:57:14.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/216: fdatasync d3/d5/da/db/f1e 0 2024-09-16T08:57:14.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/217: creat d3/d5/da/f4f x:0 0 0 2024-09-16T08:57:14.312 INFO:tasks.workunit.client.0.smithi031.stdout:9/218: chown d3/d5/l16 251151833 1 2024-09-16T08:57:14.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/219: rename d3/d5/da/d34/d3c/d42 to d3/d5/da/d34/d3c/d42/d50 22 2024-09-16T08:57:14.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/220: write d3/d5/da/f2b [5799930,81650] 0 2024-09-16T08:57:14.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/221: dread - d3/d48/f49 zero size 2024-09-16T08:57:14.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/222: chown d3/d5/da/d34/d3c 124 1 2024-09-16T08:57:14.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/223: readlink d3/d5/da/db/d21/l24 0 2024-09-16T08:57:14.313 INFO:tasks.workunit.client.1.smithi097.stdout:7/575: symlink d4/d7/d7b/d43/lc0 0 2024-09-16T08:57:14.313 INFO:tasks.workunit.client.1.smithi097.stdout:2/646: dwrite d3/d1b/f97 [0,4194304] 0 2024-09-16T08:57:14.314 INFO:tasks.workunit.client.0.smithi031.stdout:8/196: dwrite d4/d2f/f41 [0,4194304] 0 2024-09-16T08:57:14.314 INFO:tasks.workunit.client.0.smithi031.stdout:3/275: dwrite d3/f2a [0,4194304] 0 2024-09-16T08:57:14.314 INFO:tasks.workunit.client.0.smithi031.stdout:3/276: stat d3/c17 0 2024-09-16T08:57:14.314 INFO:tasks.workunit.client.1.smithi097.stdout:2/647: write d3/d1b/f83 [727484,42718] 0 2024-09-16T08:57:14.317 INFO:tasks.workunit.client.0.smithi031.stdout:2/176: mknod d6/da/d37/c38 0 2024-09-16T08:57:14.317 INFO:tasks.workunit.client.0.smithi031.stdout:2/177: dread - d6/d20/d23/d2c/f34 zero size 2024-09-16T08:57:14.321 INFO:tasks.workunit.client.0.smithi031.stdout:7/299: mkdir d2/d2d/d68 0 2024-09-16T08:57:14.324 INFO:tasks.workunit.client.0.smithi031.stdout:6/314: rmdir d0/dd/d35/d5b 0 2024-09-16T08:57:14.325 INFO:tasks.workunit.client.1.smithi097.stdout:9/578: symlink d2/d30/d74/d8c/lb8 0 2024-09-16T08:57:14.326 INFO:tasks.workunit.client.0.smithi031.stdout:5/258: stat d2/dc/d25/d21/d2e/c39 0 2024-09-16T08:57:14.327 INFO:tasks.workunit.client.1.smithi097.stdout:4/622: truncate d0/d7/f75 587162 0 2024-09-16T08:57:14.330 INFO:tasks.workunit.client.1.smithi097.stdout:5/693: creat db/d18/d20/fe6 x:0 0 0 2024-09-16T08:57:14.331 INFO:tasks.workunit.client.1.smithi097.stdout:8/618: creat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/fc0 x:0 0 0 2024-09-16T08:57:14.337 INFO:tasks.workunit.client.1.smithi097.stdout:9/579: dread d2/f46 [0,4194304] 0 2024-09-16T08:57:14.337 INFO:tasks.workunit.client.1.smithi097.stdout:9/580: write d2/d5/d3a/d70/f92 [59572,23279] 0 2024-09-16T08:57:14.337 INFO:tasks.workunit.client.1.smithi097.stdout:3/658: getdents d7/d28/d30/d88/db3 0 2024-09-16T08:57:14.339 INFO:tasks.workunit.client.0.smithi031.stdout:3/277: dwrite d3/d14/d37/d33/f3c [0,4194304] 0 2024-09-16T08:57:14.340 INFO:tasks.workunit.client.1.smithi097.stdout:7/576: creat d4/d9/d2b/fc1 x:0 0 0 2024-09-16T08:57:14.340 INFO:tasks.workunit.client.1.smithi097.stdout:7/577: fdatasync d4/d1b/d41/f81 0 2024-09-16T08:57:14.340 INFO:tasks.workunit.client.0.smithi031.stdout:2/178: dwrite d6/d20/f33 [0,4194304] 0 2024-09-16T08:57:14.343 INFO:tasks.workunit.client.0.smithi031.stdout:8/197: getdents d4/d7/d21/d35 0 2024-09-16T08:57:14.344 INFO:tasks.workunit.client.1.smithi097.stdout:5/694: dread db/d18/d1c/d42/d46/d7e/f53 [0,4194304] 0 2024-09-16T08:57:14.350 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.348+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d1019b540 con 0x7f5cfc073dd0 2024-09-16T08:57:14.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.350+0000 7f5d037fe640 1 -- 172.21.15.31:0/2655786152 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f5d1019b540 con 0x7f5cfc073dd0 2024-09-16T08:57:14.352 INFO:tasks.workunit.client.0.smithi031.stdout:7/300: mkdir d2/d8/d69 0 2024-09-16T08:57:14.353 INFO:tasks.workunit.client.1.smithi097.stdout:9/581: dread d2/fe [0,4194304] 0 2024-09-16T08:57:14.354 INFO:tasks.workunit.client.1.smithi097.stdout:2/648: mknod d3/d4/d68/d8b/d7c/cc1 0 2024-09-16T08:57:14.354 INFO:tasks.workunit.client.1.smithi097.stdout:2/649: write d3/f71 [34284,47285] 0 2024-09-16T08:57:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f5cfc073dd0 msgr2=0x7f5cfc076290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f5cfc073dd0 0x7f5cfc076290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c02aaa0 tx=0x7f5d0c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 msgr2=0x7f5d10195b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 0x7f5d10195b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800eb40 tx=0x7f5d0800c2d0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 shutdown_connections 2024-09-16T08:57:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f5cfc073dd0 0x7f5cfc076290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5d10100dc0 0x7f5d10195b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 --2- 172.21.15.31:0/2655786152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d100ffbc0 0x7f5d10195610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.353+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 >> 172.21.15.31:0/2655786152 conn(0x7f5d100fb350 msgr2=0x7f5d100fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.354+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 shutdown_connections 2024-09-16T08:57:14.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.354+0000 7f5d182b2640 1 -- 172.21.15.31:0/2655786152 wait complete. 2024-09-16T08:57:14.356 INFO:tasks.workunit.client.0.smithi031.stdout:6/315: truncate d0/ff 1093076 0 2024-09-16T08:57:14.360 INFO:tasks.workunit.client.1.smithi097.stdout:8/619: link d8/dd/d15/f8f d8/dd/d15/d16/d7c/da2/fc1 0 2024-09-16T08:57:14.360 INFO:tasks.workunit.client.1.smithi097.stdout:8/620: chown d8/dd/c8d 2662221 1 2024-09-16T08:57:14.362 INFO:tasks.workunit.client.0.smithi031.stdout:5/259: symlink d2/dc/d3d/l50 0 2024-09-16T08:57:14.363 INFO:tasks.workunit.client.0.smithi031.stdout:5/260: dread d2/dc/d25/d21/f26 [0,4194304] 0 2024-09-16T08:57:14.363 INFO:tasks.workunit.client.1.smithi097.stdout:7/578: fsync d4/d9/d2b/fc1 0 2024-09-16T08:57:14.369 INFO:tasks.workunit.client.0.smithi031.stdout:9/224: unlink d3/d5/da/db/d21/d3b/f4b 0 2024-09-16T08:57:14.372 INFO:tasks.workunit.client.0.smithi031.stdout:9/225: dread f2 [0,4194304] 0 2024-09-16T08:57:14.376 INFO:tasks.workunit.client.0.smithi031.stdout:6/316: unlink d0/dd/d35/d59/c57 0 2024-09-16T08:57:14.376 INFO:tasks.workunit.client.0.smithi031.stdout:6/317: dread - d0/d17/d19/f26 zero size 2024-09-16T08:57:14.376 INFO:tasks.workunit.client.0.smithi031.stdout:6/318: dread - d0/dd/d2b/f4a zero size 2024-09-16T08:57:14.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/319: dread d0/d17/d19/f32 [0,4194304] 0 2024-09-16T08:57:14.377 INFO:tasks.workunit.client.1.smithi097.stdout:3/659: symlink d7/d15/d52/d81/le6 0 2024-09-16T08:57:14.379 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:57:14.380 INFO:tasks.workunit.client.1.smithi097.stdout:5/695: truncate db/d18/f68 332629 0 2024-09-16T08:57:14.381 INFO:tasks.workunit.client.1.smithi097.stdout:9/582: mkdir d2/d5/d3a/d3c/d73/db9 0 2024-09-16T08:57:14.382 INFO:tasks.workunit.client.1.smithi097.stdout:2/650: symlink d3/d4/d9/d67/db2/lc2 0 2024-09-16T08:57:14.384 INFO:tasks.workunit.client.1.smithi097.stdout:8/621: creat d8/dd/d15/d2c/d4a/d4d/d62/fc2 x:0 0 0 2024-09-16T08:57:14.385 INFO:tasks.workunit.client.1.smithi097.stdout:6/637: rename d1/dc/d1e to d1/dd3 0 2024-09-16T08:57:14.385 INFO:tasks.workunit.client.1.smithi097.stdout:6/638: write d1/dc/d3c/f6d [4763600,75673] 0 2024-09-16T08:57:14.385 INFO:tasks.workunit.client.1.smithi097.stdout:6/639: chown d1/dd3/d9e/dca/d81/dad/d5a/c18 23971 1 2024-09-16T08:57:14.386 INFO:tasks.workunit.client.1.smithi097.stdout:3/660: creat d7/de/d6c/fe7 x:0 0 0 2024-09-16T08:57:14.386 INFO:tasks.workunit.client.1.smithi097.stdout:3/661: write d7/d15/d52/fbf [2594913,61138] 0 2024-09-16T08:57:14.387 INFO:tasks.workunit.client.1.smithi097.stdout:5/696: creat db/d1a/d6d/dc6/fe7 x:0 0 0 2024-09-16T08:57:14.387 INFO:tasks.workunit.client.1.smithi097.stdout:3/662: read d7/de/d3c/d4f/f7c [52031,29386] 0 2024-09-16T08:57:14.387 INFO:tasks.workunit.client.1.smithi097.stdout:3/663: write d7/de/d3c/fe4 [696857,95494] 0 2024-09-16T08:57:14.387 INFO:tasks.workunit.client.1.smithi097.stdout:9/583: mknod d2/d5/d56/cba 0 2024-09-16T08:57:14.388 INFO:tasks.workunit.client.1.smithi097.stdout:2/651: mknod d3/d4/d68/d8b/cc3 0 2024-09-16T08:57:14.388 INFO:tasks.workunit.client.1.smithi097.stdout:2/652: chown d3/d4/d68/d8b/d7c/d7d/f6b 8034 1 2024-09-16T08:57:14.390 INFO:tasks.workunit.client.1.smithi097.stdout:1/731: rename d5/db/f52 to d5/db/dd4/fee 0 2024-09-16T08:57:14.391 INFO:tasks.workunit.client.1.smithi097.stdout:6/640: mknod d1/dd3/d9e/dca/d16/d4f/cd4 0 2024-09-16T08:57:14.392 INFO:tasks.workunit.client.0.smithi031.stdout:9/226: write d3/f2a [1229102,72591] 0 2024-09-16T08:57:14.393 INFO:tasks.workunit.client.0.smithi031.stdout:9/227: chown d3/d5/da/d34/d3c 411530 1 2024-09-16T08:57:14.393 INFO:tasks.workunit.client.0.smithi031.stdout:9/228: chown d3/d5/da 392283884 1 2024-09-16T08:57:14.393 INFO:tasks.workunit.client.0.smithi031.stdout:9/229: fsync d3/d5/da/d34/f46 0 2024-09-16T08:57:14.393 INFO:tasks.workunit.client.0.smithi031.stdout:7/301: dwrite d2/d8/d11/f5c [0,4194304] 0 2024-09-16T08:57:14.393 INFO:tasks.workunit.client.1.smithi097.stdout:3/664: getdents d7 0 2024-09-16T08:57:14.394 INFO:tasks.workunit.client.1.smithi097.stdout:9/584: link d2/l23 d2/d30/d74/d8c/d94/lbb 0 2024-09-16T08:57:14.394 INFO:tasks.workunit.client.0.smithi031.stdout:2/179: dwrite d6/fb [0,4194304] 0 2024-09-16T08:57:14.395 INFO:tasks.workunit.client.0.smithi031.stdout:3/278: dwrite d3/d14/d37/d1a/d1d/d3b/f4f [0,4194304] 0 2024-09-16T08:57:14.395 INFO:tasks.workunit.client.1.smithi097.stdout:2/653: mknod d3/cc4 0 2024-09-16T08:57:14.396 INFO:tasks.workunit.client.0.smithi031.stdout:8/198: dwrite d4/d2f/f41 [0,4194304] 0 2024-09-16T08:57:14.397 INFO:tasks.workunit.client.0.smithi031.stdout:5/261: dwrite d2/dc/f3b [0,4194304] 0 2024-09-16T08:57:14.399 INFO:tasks.workunit.client.0.smithi031.stdout:9/230: rmdir d3/d5/da/db/d21/d3b/d43 0 2024-09-16T08:57:14.399 INFO:tasks.workunit.client.1.smithi097.stdout:5/697: write db/d18/d1c/f39 [4098536,4786] 0 2024-09-16T08:57:14.399 INFO:tasks.workunit.client.1.smithi097.stdout:5/698: fsync db/d18/d1c/d42/d46/d7e/d76/fc0 0 2024-09-16T08:57:14.400 INFO:tasks.workunit.client.1.smithi097.stdout:1/732: rmdir d5/db/dc/d19 39 2024-09-16T08:57:14.401 INFO:tasks.workunit.client.0.smithi031.stdout:7/302: creat d2/f6a x:0 0 0 2024-09-16T08:57:14.403 INFO:tasks.workunit.client.1.smithi097.stdout:9/585: read d2/d5/d3a/d3c/d73/f7a [2415298,128005] 0 2024-09-16T08:57:14.404 INFO:tasks.workunit.client.1.smithi097.stdout:9/586: fdatasync d2/d30/f5a 0 2024-09-16T08:57:14.404 INFO:tasks.workunit.client.0.smithi031.stdout:6/320: dwrite d0/d17/f30 [0,4194304] 0 2024-09-16T08:57:14.406 INFO:tasks.workunit.client.1.smithi097.stdout:8/622: dwrite d8/f31 [0,4194304] 0 2024-09-16T08:57:14.411 INFO:tasks.workunit.client.1.smithi097.stdout:6/641: rename d1/dd3/d9e/dca/d16/d6a/la3 to d1/dc/d3c/ld5 0 2024-09-16T08:57:14.411 INFO:tasks.workunit.client.1.smithi097.stdout:6/642: chown d1/dd3/d9e/dca/d16/d4f/f78 682216095 1 2024-09-16T08:57:14.414 INFO:tasks.workunit.client.1.smithi097.stdout:2/654: mknod d3/d4/d68/d8b/d61/dac/cc5 0 2024-09-16T08:57:14.414 INFO:tasks.workunit.client.1.smithi097.stdout:2/655: stat d3/d4/d8/c75 0 2024-09-16T08:57:14.414 INFO:tasks.workunit.client.1.smithi097.stdout:2/656: chown d3/d4/d8/d51/l52 2 1 2024-09-16T08:57:14.418 INFO:tasks.workunit.client.1.smithi097.stdout:5/699: dwrite db/d18/d1c/d42/d46/d7e/d64/fb3 [0,4194304] 0 2024-09-16T08:57:14.418 INFO:tasks.workunit.client.1.smithi097.stdout:5/700: write db/d18/d1c/d42/d46/d7e/d8f/fc2 [430342,95610] 0 2024-09-16T08:57:14.418 INFO:tasks.workunit.client.1.smithi097.stdout:5/701: chown db/d18/d20/d2b/c2f 413 1 2024-09-16T08:57:14.418 INFO:tasks.workunit.client.1.smithi097.stdout:3/665: dread d7/d15/d52/fbf [0,4194304] 0 2024-09-16T08:57:14.419 INFO:tasks.workunit.client.0.smithi031.stdout:2/180: creat d6/d20/d23/d28/f39 x:0 0 0 2024-09-16T08:57:14.423 INFO:tasks.workunit.client.1.smithi097.stdout:1/733: creat d5/d70/d85/db5/fef x:0 0 0 2024-09-16T08:57:14.423 INFO:tasks.workunit.client.1.smithi097.stdout:1/734: stat d5/d17/d54/d6b/d71/lc2 0 2024-09-16T08:57:14.424 INFO:tasks.workunit.client.0.smithi031.stdout:3/279: truncate d3/fb 2570057 0 2024-09-16T08:57:14.426 INFO:tasks.workunit.client.1.smithi097.stdout:9/587: link d2/d30/d74/d8c/d94/d96/dac/l80 d2/d30/lbc 0 2024-09-16T08:57:14.426 INFO:tasks.workunit.client.1.smithi097.stdout:9/588: dread - d2/d5/d3a/f4c zero size 2024-09-16T08:57:14.426 INFO:tasks.workunit.client.1.smithi097.stdout:9/589: chown d2/d3f/l67 249955 1 2024-09-16T08:57:14.426 INFO:tasks.workunit.client.1.smithi097.stdout:9/590: chown d2/d5/d72/l9d 1290933028 1 2024-09-16T08:57:14.427 INFO:tasks.workunit.client.0.smithi031.stdout:8/199: mkdir d4/d7/d21/d37/d46 0 2024-09-16T08:57:14.429 INFO:tasks.workunit.client.1.smithi097.stdout:8/623: mknod d8/dd/d15/d16/cc3 0 2024-09-16T08:57:14.429 INFO:tasks.workunit.client.1.smithi097.stdout:8/624: fdatasync d8/dd/d15/d16/fa1 0 2024-09-16T08:57:14.430 INFO:tasks.workunit.client.0.smithi031.stdout:5/262: creat d2/d4b/f51 x:0 0 0 2024-09-16T08:57:14.430 INFO:tasks.workunit.client.0.smithi031.stdout:5/263: creat d2/dc/d36/f52 x:0 0 0 2024-09-16T08:57:14.430 INFO:tasks.workunit.client.1.smithi097.stdout:6/643: creat d1/dc/fd6 x:0 0 0 2024-09-16T08:57:14.431 INFO:tasks.workunit.client.1.smithi097.stdout:2/657: dwrite d3/d4/d68/d8b/d7c/d7d/f18 [0,4194304] 0 2024-09-16T08:57:14.432 INFO:tasks.workunit.client.0.smithi031.stdout:9/231: mknod d3/d5/da/c51 0 2024-09-16T08:57:14.433 INFO:tasks.workunit.client.0.smithi031.stdout:7/303: truncate d2/d8/d11/f45 1525197 0 2024-09-16T08:57:14.433 INFO:tasks.workunit.client.0.smithi031.stdout:7/304: stat d2/f56 0 2024-09-16T08:57:14.433 INFO:tasks.workunit.client.0.smithi031.stdout:7/305: fsync d2/d8/f63 0 2024-09-16T08:57:14.436 INFO:tasks.workunit.client.1.smithi097.stdout:5/702: creat db/d18/d1c/d42/d46/d7e/fe8 x:0 0 0 2024-09-16T08:57:14.436 INFO:tasks.workunit.client.1.smithi097.stdout:5/703: chown db/d1a/fe2 24 1 2024-09-16T08:57:14.437 INFO:tasks.workunit.client.0.smithi031.stdout:6/321: mkdir d0/d17/d5e 0 2024-09-16T08:57:14.438 INFO:tasks.workunit.client.1.smithi097.stdout:3/666: mknod d7/de/d6c/dcf/ce8 0 2024-09-16T08:57:14.443 INFO:tasks.workunit.client.0.smithi031.stdout:2/181: mknod d6/d20/d23/d2c/c3a 0 2024-09-16T08:57:14.444 INFO:tasks.workunit.client.0.smithi031.stdout:3/280: creat d3/d14/d37/d1a/d1d/d3b/d3f/f52 x:0 0 0 2024-09-16T08:57:14.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/200: unlink d4/d2d/f33 0 2024-09-16T08:57:14.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/201: readlink d4/l30 0 2024-09-16T08:57:14.449 INFO:tasks.workunit.client.0.smithi031.stdout:5/264: truncate d2/fa 5483997 0 2024-09-16T08:57:14.451 INFO:tasks.workunit.client.0.smithi031.stdout:2/182: write d6/da/fd [1314637,100566] 0 2024-09-16T08:57:14.451 INFO:tasks.workunit.client.0.smithi031.stdout:9/232: unlink d3/d5/da/db/l1a 0 2024-09-16T08:57:14.454 INFO:tasks.workunit.client.0.smithi031.stdout:7/306: creat d2/d2d/d68/f6b x:0 0 0 2024-09-16T08:57:14.454 INFO:tasks.workunit.client.0.smithi031.stdout:7/307: stat d2/d8/d69 0 2024-09-16T08:57:14.456 INFO:tasks.workunit.client.1.smithi097.stdout:9/591: rmdir d2/d5/d3a/d3c/d73/db9 0 2024-09-16T08:57:14.457 INFO:tasks.workunit.client.1.smithi097.stdout:9/592: stat d2/c1f 0 2024-09-16T08:57:14.457 INFO:tasks.workunit.client.1.smithi097.stdout:9/593: chown d2/f50 2165 1 2024-09-16T08:57:14.458 INFO:tasks.workunit.client.0.smithi031.stdout:6/322: symlink d0/d17/d5e/l5f 0 2024-09-16T08:57:14.458 INFO:tasks.workunit.client.0.smithi031.stdout:6/323: fdatasync d0/d17/f55 0 2024-09-16T08:57:14.459 INFO:tasks.workunit.client.1.smithi097.stdout:8/625: chown d8/dd/l38 134287 1 2024-09-16T08:57:14.460 INFO:tasks.workunit.client.0.smithi031.stdout:3/281: mknod d3/d14/c53 0 2024-09-16T08:57:14.460 INFO:tasks.workunit.client.0.smithi031.stdout:3/282: fsync d3/fb 0 2024-09-16T08:57:14.460 INFO:tasks.workunit.client.0.smithi031.stdout:3/283: dread - d3/d14/d37/d33/f4d zero size 2024-09-16T08:57:14.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/644: rename d1/dd3/d9e/dca/d16/fc0 to d1/dd3/d9e/dca/db6/fd7 0 2024-09-16T08:57:14.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/645: chown d1/dd3/db0/lc3 30194 1 2024-09-16T08:57:14.464 INFO:tasks.workunit.client.0.smithi031.stdout:8/202: rename d4/l30 to d4/d7/d21/d35/d40/l47 0 2024-09-16T08:57:14.466 INFO:tasks.workunit.client.1.smithi097.stdout:2/658: rename d3/d4/d68/d8b/d7c/d7d/f18 to d3/d4/d68/d8b/d7c/d7d/fc6 0 2024-09-16T08:57:14.467 INFO:tasks.workunit.client.0.smithi031.stdout:5/265: creat d2/dc/d25/f53 x:0 0 0 2024-09-16T08:57:14.467 INFO:tasks.workunit.client.0.smithi031.stdout:5/266: chown d2/dc/d36/f41 1 1 2024-09-16T08:57:14.468 INFO:tasks.workunit.client.0.smithi031.stdout:3/284: write d3/f2a [4165821,34696] 0 2024-09-16T08:57:14.469 INFO:tasks.workunit.client.0.smithi031.stdout:3/285: write d3/d14/d37/d1a/f26 [5032782,129674] 0 2024-09-16T08:57:14.469 INFO:tasks.workunit.client.0.smithi031.stdout:3/286: write d3/d14/d37/d1a/d1d/d3b/f43 [1364131,123685] 0 2024-09-16T08:57:14.469 INFO:tasks.workunit.client.0.smithi031.stdout:5/267: dread d2/dc/f3b [4194304,4194304] 0 2024-09-16T08:57:14.469 INFO:tasks.workunit.client.1.smithi097.stdout:3/667: link d7/de/l24 d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/le9 0 2024-09-16T08:57:14.472 INFO:tasks.workunit.client.1.smithi097.stdout:9/594: creat d2/d5/fbd x:0 0 0 2024-09-16T08:57:14.473 INFO:tasks.workunit.client.0.smithi031.stdout:2/183: creat d6/d20/d23/f3b x:0 0 0 2024-09-16T08:57:14.474 INFO:tasks.workunit.client.0.smithi031.stdout:7/308: stat d2/l3d 0 2024-09-16T08:57:14.476 INFO:tasks.workunit.client.1.smithi097.stdout:6/646: mkdir d1/dd8 0 2024-09-16T08:57:14.477 INFO:tasks.workunit.client.1.smithi097.stdout:9/595: creat d2/d30/d74/d8c/d94/d96/dac/d79/fbe x:0 0 0 2024-09-16T08:57:14.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/287: write d3/f35 [2945335,1920] 0 2024-09-16T08:57:14.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/288: chown d3/d14/d37/d1a/d24/l45 8 1 2024-09-16T08:57:14.478 INFO:tasks.workunit.client.0.smithi031.stdout:3/289: write d3/f4 [3164403,125749] 0 2024-09-16T08:57:14.478 INFO:tasks.workunit.client.0.smithi031.stdout:9/233: rename d3/d5/da/db/c19 to d3/d5/da/db/d21/c52 0 2024-09-16T08:57:14.481 INFO:tasks.workunit.client.1.smithi097.stdout:8/626: mknod d8/dd/cc4 0 2024-09-16T08:57:14.483 INFO:tasks.workunit.client.0.smithi031.stdout:5/268: symlink d2/dc/d25/d21/d2e/l54 0 2024-09-16T08:57:14.485 INFO:tasks.workunit.client.0.smithi031.stdout:7/309: dread d2/d8/f4a [0,4194304] 0 2024-09-16T08:57:14.485 INFO:tasks.workunit.client.0.smithi031.stdout:7/310: write d2/d2d/f4b [2205,45740] 0 2024-09-16T08:57:14.486 INFO:tasks.workunit.client.1.smithi097.stdout:3/668: dread d7/d28/d30/d88/fa5 [0,4194304] 0 2024-09-16T08:57:14.486 INFO:tasks.workunit.client.1.smithi097.stdout:3/669: readlink d7/d28/d63/l82 0 2024-09-16T08:57:14.486 INFO:tasks.workunit.client.1.smithi097.stdout:3/670: chown d7/de/d9c/dca 433931 1 2024-09-16T08:57:14.487 INFO:tasks.workunit.client.1.smithi097.stdout:6/647: truncate d1/dd3/d9e/dca/d81/dad/f9a 877244 0 2024-09-16T08:57:14.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/324: dread d0/d17/f18 [4194304,4194304] 0 2024-09-16T08:57:14.489 INFO:tasks.workunit.client.0.smithi031.stdout:2/184: symlink d6/d20/d23/d2c/l3c 0 2024-09-16T08:57:14.491 INFO:tasks.workunit.client.0.smithi031.stdout:3/290: symlink d3/d14/d37/l54 0 2024-09-16T08:57:14.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 -- 172.21.15.31:0/3281285194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 msgr2=0x7f90a81074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 --2- 172.21.15.31:0/3281285194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a81074b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009c20 tx=0x7f90a002aa30 comp rx=0 tx=0).stop 2024-09-16T08:57:14.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 -- 172.21.15.31:0/3281285194 shutdown_connections 2024-09-16T08:57:14.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 --2- 172.21.15.31:0/3281285194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a81074b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 --2- 172.21.15.31:0/3281285194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90a80716c0 0x7f90a8071ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 -- 172.21.15.31:0/3281285194 >> 172.21.15.31:0/3281285194 conn(0x7f90a806cfa0 msgr2=0x7f90a806f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:14.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 -- 172.21.15.31:0/3281285194 shutdown_connections 2024-09-16T08:57:14.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.491+0000 7f90af6c8640 1 -- 172.21.15.31:0/3281285194 wait complete. 2024-09-16T08:57:14.494 INFO:tasks.workunit.client.0.smithi031.stdout:8/203: rename d4/d7/d21/d24/c25 to d4/d7/d21/d35/d40/c48 0 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.492+0000 7f90af6c8640 1 Processor -- start 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.492+0000 7f90af6c8640 1 -- start start 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.492+0000 7f90af6c8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90a80716c0 0x7f90a819e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.492+0000 7f90af6c8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.493+0000 7f90af6c8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a819ebe0 con 0x7f90a8072000 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.493+0000 7f90af6c8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a819ed50 con 0x7f90a80716c0 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.493+0000 7f90acc63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.493+0000 7f90acc63640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a819e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34800/0 (socket says 172.21.15.31:34800) 2024-09-16T08:57:14.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.493+0000 7f90acc63640 1 -- 172.21.15.31:0/2488715031 learned_addr learned my addr 172.21.15.31:0/2488715031 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:14.495 INFO:tasks.workunit.client.0.smithi031.stdout:5/269: mknod d2/dc/d3d/c55 0 2024-09-16T08:57:14.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.493+0000 7f90ad464640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90a80716c0 0x7f90a819e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:14.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.494+0000 7f90acc63640 1 -- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90a80716c0 msgr2=0x7f90a819e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.494+0000 7f90acc63640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90a80716c0 0x7f90a819e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.494+0000 7f90acc63640 1 -- 172.21.15.31:0/2488715031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a0009900 con 0x7f90a8072000 2024-09-16T08:57:14.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.494+0000 7f90acc63640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a819e610 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90a002fc30 tx=0x7f90a002fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:14.497 INFO:tasks.workunit.client.0.smithi031.stdout:7/311: creat d2/d8/d69/f6c x:0 0 0 2024-09-16T08:57:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.495+0000 7f909e7fc640 1 -- 172.21.15.31:0/2488715031 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a003d070 con 0x7f90a8072000 2024-09-16T08:57:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.495+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a81085c0 con 0x7f90a8072000 2024-09-16T08:57:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.495+0000 7f909e7fc640 1 -- 172.21.15.31:0/2488715031 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90a0038910 con 0x7f90a8072000 2024-09-16T08:57:14.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.495+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a8108b30 con 0x7f90a8072000 2024-09-16T08:57:14.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.496+0000 7f909e7fc640 1 -- 172.21.15.31:0/2488715031 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a00437a0 con 0x7f90a8072000 2024-09-16T08:57:14.498 INFO:tasks.workunit.client.1.smithi097.stdout:8/627: read d8/fad [1310732,712] 0 2024-09-16T08:57:14.498 INFO:tasks.workunit.client.1.smithi097.stdout:8/628: fsync d8/dd/d42/f89 0 2024-09-16T08:57:14.499 INFO:tasks.workunit.client.0.smithi031.stdout:2/185: rename d6/da/f15 to d6/da/f3d 0 2024-09-16T08:57:14.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.498+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a8110fb0 con 0x7f90a8072000 2024-09-16T08:57:14.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/291: mknod d3/d14/c55 0 2024-09-16T08:57:14.501 INFO:tasks.workunit.client.0.smithi031.stdout:3/292: write d3/d14/d37/d33/f38 [5044970,123766] 0 2024-09-16T08:57:14.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.500+0000 7f909e7fc640 1 -- 172.21.15.31:0/2488715031 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f90a0041070 con 0x7f90a8072000 2024-09-16T08:57:14.501 INFO:tasks.workunit.client.0.smithi031.stdout:6/325: write d0/d17/f30 [253855,72971] 0 2024-09-16T08:57:14.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.500+0000 7f909e7fc640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9094073d40 0x7f9094076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:14.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.500+0000 7f909e7fc640 1 -- 172.21.15.31:0/2488715031 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f90a00b6880 con 0x7f90a8072000 2024-09-16T08:57:14.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.500+0000 7f90ad464640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9094073d40 0x7f9094076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:14.502 INFO:tasks.workunit.client.1.smithi097.stdout:8/629: link d8/dd/d15/d16/l78 d8/dd/d15/d16/d7c/da2/lc5 0 2024-09-16T08:57:14.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.501+0000 7f90ad464640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9094073d40 0x7f9094076200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f90a8068be0 tx=0x7f90a40090e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:14.503 INFO:tasks.workunit.client.0.smithi031.stdout:8/204: truncate d4/f29 432378 0 2024-09-16T08:57:14.503 INFO:tasks.workunit.client.0.smithi031.stdout:8/205: fdatasync d4/d7/f28 0 2024-09-16T08:57:14.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/270: mknod d2/dc/d3d/c56 0 2024-09-16T08:57:14.509 INFO:tasks.workunit.client.0.smithi031.stdout:7/312: mkdir d2/d2d/d6d 0 2024-09-16T08:57:14.509 INFO:tasks.workunit.client.0.smithi031.stdout:7/313: truncate d2/d8/d69/f6c 168153 0 2024-09-16T08:57:14.509 INFO:tasks.workunit.client.0.smithi031.stdout:3/293: mknod d3/d14/d37/c56 0 2024-09-16T08:57:14.510 INFO:tasks.workunit.client.0.smithi031.stdout:8/206: mknod d4/d7/d21/d37/d46/c49 0 2024-09-16T08:57:14.511 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.509+0000 7f909e7fc640 1 -- 172.21.15.31:0/2488715031 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f90a007a030 con 0x7f90a8072000 2024-09-16T08:57:14.511 INFO:tasks.workunit.client.0.smithi031.stdout:7/314: mkdir d2/d8/d11/d5d/d6e 0 2024-09-16T08:57:14.512 INFO:tasks.workunit.client.0.smithi031.stdout:2/186: write d6/da/f27 [1314885,92464] 0 2024-09-16T08:57:14.514 INFO:tasks.workunit.client.0.smithi031.stdout:3/294: rename d3/d14/d37/d1a/d1d/c3a to d3/d14/d37/d1a/d1d/d3b/d3f/d49/c57 0 2024-09-16T08:57:14.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/315: dread d2/f35 [0,4194304] 0 2024-09-16T08:57:14.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/316: creat d2/d2d/f6f x:0 0 0 2024-09-16T08:57:14.515 INFO:tasks.workunit.client.0.smithi031.stdout:7/317: stat d2/d2d/l5f 0 2024-09-16T08:57:14.518 INFO:tasks.workunit.client.0.smithi031.stdout:3/295: dread d3/d14/d37/d1a/d1d/f2d [0,4194304] 0 2024-09-16T08:57:14.518 INFO:tasks.workunit.client.0.smithi031.stdout:3/296: fsync d3/d14/d37/d33/f3c 0 2024-09-16T08:57:14.519 INFO:tasks.workunit.client.1.smithi097.stdout:8/630: dread d8/dd/d15/d2c/d4a/d4d/f5c [0,4194304] 0 2024-09-16T08:57:14.519 INFO:tasks.workunit.client.1.smithi097.stdout:8/631: chown d8/dd/d15/d2c/l9f 912 1 2024-09-16T08:57:14.524 INFO:tasks.workunit.client.0.smithi031.stdout:9/234: dwrite d3/d5/da/db/f18 [0,4194304] 0 2024-09-16T08:57:14.524 INFO:tasks.workunit.client.0.smithi031.stdout:9/235: stat d3/d5 0 2024-09-16T08:57:14.524 INFO:tasks.workunit.client.0.smithi031.stdout:9/236: chown d3/d5/da/db/d21/f2f 169 1 2024-09-16T08:57:14.524 INFO:tasks.workunit.client.0.smithi031.stdout:9/237: creat d3/d5/da/d34/d3c/d42/f53 x:0 0 0 2024-09-16T08:57:14.525 INFO:tasks.workunit.client.1.smithi097.stdout:9/596: dwrite d2/d5/d56/f57 [0,4194304] 0 2024-09-16T08:57:14.525 INFO:tasks.workunit.client.1.smithi097.stdout:8/632: creat d8/dd/d15/d2c/d4a/fc6 x:0 0 0 2024-09-16T08:57:14.525 INFO:tasks.workunit.client.0.smithi031.stdout:6/326: read d0/d17/d19/f1d [2648055,102068] 0 2024-09-16T08:57:14.535 INFO:tasks.workunit.client.0.smithi031.stdout:3/297: write d3/d14/d37/d1a/d1d/d3b/f4f [1232888,21133] 0 2024-09-16T08:57:14.538 INFO:tasks.workunit.client.1.smithi097.stdout:9/597: rename d2/d5/d3a/d3c/c86 to d2/d5/d72/da2/cbf 0 2024-09-16T08:57:14.538 INFO:tasks.workunit.client.0.smithi031.stdout:9/238: symlink d3/d5/da/d34/d4d/l54 0 2024-09-16T08:57:14.540 INFO:tasks.workunit.client.1.smithi097.stdout:8/633: symlink d8/dd/d15/d2c/d4a/d4d/lc7 0 2024-09-16T08:57:14.540 INFO:tasks.workunit.client.0.smithi031.stdout:2/187: rmdir d6/d20/d23 39 2024-09-16T08:57:14.542 INFO:tasks.workunit.client.0.smithi031.stdout:6/327: unlink d0/dd/d2b/f4a 0 2024-09-16T08:57:14.543 INFO:tasks.workunit.client.1.smithi097.stdout:3/671: dwrite d7/d28/fd4 [0,4194304] 0 2024-09-16T08:57:14.543 INFO:tasks.workunit.client.1.smithi097.stdout:3/672: write d7/de/d3c/d4f/f80 [2016451,60335] 0 2024-09-16T08:57:14.543 INFO:tasks.workunit.client.1.smithi097.stdout:6/648: dwrite d1/dd3/d9e/dca/d81/f93 [0,4194304] 0 2024-09-16T08:57:14.543 INFO:tasks.workunit.client.1.smithi097.stdout:3/673: write d7/de/d3c/d3f/d60/dcb/fa7 [448825,72567] 0 2024-09-16T08:57:14.543 INFO:tasks.workunit.client.1.smithi097.stdout:3/674: getdents d7/de/d8d/d90/d92/da8 0 2024-09-16T08:57:14.543 INFO:tasks.workunit.client.1.smithi097.stdout:3/675: creat d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa/fea x:0 0 0 2024-09-16T08:57:14.544 INFO:tasks.workunit.client.0.smithi031.stdout:5/271: dwrite d2/f13 [0,4194304] 0 2024-09-16T08:57:14.544 INFO:tasks.workunit.client.0.smithi031.stdout:5/272: readlink d2/dc/d3d/l3e 0 2024-09-16T08:57:14.544 INFO:tasks.workunit.client.0.smithi031.stdout:7/318: dwrite d2/d2d/f2e [4194304,4194304] 0 2024-09-16T08:57:14.544 INFO:tasks.workunit.client.0.smithi031.stdout:8/207: dwrite d4/d2d/f3a [0,4194304] 0 2024-09-16T08:57:14.545 INFO:tasks.workunit.client.1.smithi097.stdout:6/649: dread d1/f5b [0,4194304] 0 2024-09-16T08:57:14.545 INFO:tasks.workunit.client.0.smithi031.stdout:3/298: creat d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 x:0 0 0 2024-09-16T08:57:14.546 INFO:tasks.workunit.client.1.smithi097.stdout:9/598: rmdir d2/d30/d63 39 2024-09-16T08:57:14.546 INFO:tasks.workunit.client.1.smithi097.stdout:6/650: dread d1/dd3/d9e/dca/d16/d4f/d9f/fba [0,4194304] 0 2024-09-16T08:57:14.546 INFO:tasks.workunit.client.1.smithi097.stdout:6/651: stat d1/dd3/d9e/dca/l46 0 2024-09-16T08:57:14.546 INFO:tasks.workunit.client.1.smithi097.stdout:6/652: write d1/dd3/d9e/dca/d16/d6a/f99 [41887,8140] 0 2024-09-16T08:57:14.548 INFO:tasks.workunit.client.0.smithi031.stdout:9/239: symlink d3/d5/da/db/d21/d3b/l55 0 2024-09-16T08:57:14.548 INFO:tasks.workunit.client.0.smithi031.stdout:9/240: truncate d3/d5/da/db/f1e 3450331 0 2024-09-16T08:57:14.548 INFO:tasks.workunit.client.0.smithi031.stdout:9/241: dread - d3/d48/f4e zero size 2024-09-16T08:57:14.548 INFO:tasks.workunit.client.1.smithi097.stdout:6/653: write d1/dc/d3c/f44 [1187701,78956] 0 2024-09-16T08:57:14.549 INFO:tasks.workunit.client.1.smithi097.stdout:6/654: write d1/dc/d3c/f6f [714413,130815] 0 2024-09-16T08:57:14.549 INFO:tasks.workunit.client.1.smithi097.stdout:8/634: symlink d8/dd/d15/d2c/d4a/d4e/lc8 0 2024-09-16T08:57:14.549 INFO:tasks.workunit.client.1.smithi097.stdout:8/635: write d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f8b [1764832,107910] 0 2024-09-16T08:57:14.550 INFO:tasks.workunit.client.0.smithi031.stdout:2/188: symlink d6/d20/d23/d2c/l3e 0 2024-09-16T08:57:14.554 INFO:tasks.workunit.client.0.smithi031.stdout:6/328: rename d0/d17/f40 to d0/d10/f60 0 2024-09-16T08:57:14.554 INFO:tasks.workunit.client.0.smithi031.stdout:5/273: rename d2 to d2/dc/d25/d57 22 2024-09-16T08:57:14.560 INFO:tasks.workunit.client.1.smithi097.stdout:3/676: mkdir d7/de/d3c/d3f/d60/d6b/db8/deb 0 2024-09-16T08:57:14.564 INFO:tasks.workunit.client.1.smithi097.stdout:9/599: mknod d2/d30/cc0 0 2024-09-16T08:57:14.564 INFO:tasks.workunit.client.1.smithi097.stdout:9/600: readlink d2/d30/d74/d8c/d94/lbb 0 2024-09-16T08:57:14.564 INFO:tasks.workunit.client.1.smithi097.stdout:9/601: write d2/d5/d3a/d3c/f6d [693976,129013] 0 2024-09-16T08:57:14.569 INFO:tasks.workunit.client.1.smithi097.stdout:9/602: write d2/d5/d72/f89 [228723,80059] 0 2024-09-16T08:57:14.569 INFO:tasks.workunit.client.1.smithi097.stdout:6/655: mknod d1/dc/d3c/cd9 0 2024-09-16T08:57:14.570 INFO:tasks.workunit.client.0.smithi031.stdout:7/319: mknod d2/c70 0 2024-09-16T08:57:14.570 INFO:tasks.workunit.client.0.smithi031.stdout:7/320: write d2/d8/fd [734595,71284] 0 2024-09-16T08:57:14.570 INFO:tasks.workunit.client.0.smithi031.stdout:7/321: stat d2/c70 0 2024-09-16T08:57:14.570 INFO:tasks.workunit.client.0.smithi031.stdout:7/322: chown d2/d8/l10 2366 1 2024-09-16T08:57:14.572 INFO:tasks.workunit.client.0.smithi031.stdout:8/208: link d4/d7/d21/d35/d40/c48 d4/d7/c4a 0 2024-09-16T08:57:14.573 INFO:tasks.workunit.client.0.smithi031.stdout:3/299: rmdir d3/d14/d37/d1a/d1d 39 2024-09-16T08:57:14.573 INFO:tasks.workunit.client.0.smithi031.stdout:3/300: chown d3/d14/d37/d1a/d1d/d3b/f43 127518653 1 2024-09-16T08:57:14.577 INFO:tasks.workunit.client.0.smithi031.stdout:2/189: truncate d6/da/fd 1096206 0 2024-09-16T08:57:14.580 INFO:tasks.workunit.client.0.smithi031.stdout:7/323: creat d2/d8/d11/d5d/d6e/f71 x:0 0 0 2024-09-16T08:57:14.580 INFO:tasks.workunit.client.0.smithi031.stdout:6/329: dwrite d0/dd/d2b/f31 [0,4194304] 0 2024-09-16T08:57:14.581 INFO:tasks.workunit.client.0.smithi031.stdout:5/274: dwrite d2/dc/fd [0,4194304] 0 2024-09-16T08:57:14.581 INFO:tasks.workunit.client.0.smithi031.stdout:5/275: chown d2/dc/f12 31162 1 2024-09-16T08:57:14.581 INFO:tasks.workunit.client.0.smithi031.stdout:8/209: getdents d4/d7/d21/d37/d46 0 2024-09-16T08:57:14.584 INFO:tasks.workunit.client.0.smithi031.stdout:9/242: rmdir d3/d5/da/d34/d3c/d42 39 2024-09-16T08:57:14.589 INFO:tasks.workunit.client.0.smithi031.stdout:7/324: creat d2/d8/d11/d66/f72 x:0 0 0 2024-09-16T08:57:14.595 INFO:tasks.workunit.client.1.smithi097.stdout:3/677: mkdir d7/d15/dec 0 2024-09-16T08:57:14.595 INFO:tasks.workunit.client.1.smithi097.stdout:3/678: read - d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/fb6 zero size 2024-09-16T08:57:14.597 INFO:tasks.workunit.client.1.smithi097.stdout:9/603: mkdir d2/d5/d72/dc1 0 2024-09-16T08:57:14.598 INFO:tasks.workunit.client.0.smithi031.stdout:5/276: symlink d2/dc/d3d/l58 0 2024-09-16T08:57:14.600 INFO:tasks.workunit.client.1.smithi097.stdout:6/656: symlink d1/dc/lda 0 2024-09-16T08:57:14.601 INFO:tasks.workunit.client.0.smithi031.stdout:9/243: rename d3/d5/da/db/d21/l29 to d3/d5/da/db/l56 0 2024-09-16T08:57:14.610 INFO:tasks.workunit.client.0.smithi031.stdout:7/325: write d2/f1a [1604073,421] 0 2024-09-16T08:57:14.610 INFO:tasks.workunit.client.0.smithi031.stdout:2/190: dwrite d6/da/f27 [0,4194304] 0 2024-09-16T08:57:14.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/301: dwrite d3/d14/d37/d1a/d1d/d3b/f4f [0,4194304] 0 2024-09-16T08:57:14.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/302: chown f1 47 1 2024-09-16T08:57:14.611 INFO:tasks.workunit.client.1.smithi097.stdout:3/679: dread d7/de/d3c/d3f/f7b [0,4194304] 0 2024-09-16T08:57:14.611 INFO:tasks.workunit.client.1.smithi097.stdout:3/680: write d7/d28/d30/d88/fba [175722,21704] 0 2024-09-16T08:57:14.614 INFO:tasks.workunit.client.1.smithi097.stdout:8/636: rename d8/dd/l66 to d8/dd/d15/d2c/d4a/d4d/d3f/lc9 0 2024-09-16T08:57:14.614 INFO:tasks.workunit.client.1.smithi097.stdout:8/637: fdatasync d8/dd/d15/f29 0 2024-09-16T08:57:14.616 INFO:tasks.workunit.client.1.smithi097.stdout:9/604: link d2/d5/d72/da2/cbf d2/d30/d74/d8c/d94/d96/cc2 0 2024-09-16T08:57:14.617 INFO:tasks.workunit.client.0.smithi031.stdout:6/330: rmdir d0/d17 39 2024-09-16T08:57:14.618 INFO:tasks.workunit.client.1.smithi097.stdout:3/681: symlink d7/de/d3c/d3f/d60/d6b/db8/led 0 2024-09-16T08:57:14.618 INFO:tasks.workunit.client.1.smithi097.stdout:9/605: dread d2/d5/f71 [0,4194304] 0 2024-09-16T08:57:14.618 INFO:tasks.workunit.client.0.smithi031.stdout:5/277: creat d2/f59 x:0 0 0 2024-09-16T08:57:14.620 INFO:tasks.workunit.client.1.smithi097.stdout:6/657: rename d1/f4e to d1/dd3/d9e/dca/d16/d4f/d9f/fdb 0 2024-09-16T08:57:14.621 INFO:tasks.workunit.client.1.smithi097.stdout:8/638: truncate d8/dd/d15/d2c/d4a/d4d/f49 9708126 0 2024-09-16T08:57:14.621 INFO:tasks.workunit.client.1.smithi097.stdout:8/639: chown d8/dd/d71/d9c/ca8 735310 1 2024-09-16T08:57:14.623 INFO:tasks.workunit.client.1.smithi097.stdout:3/682: mkdir d7/d15/dee 0 2024-09-16T08:57:14.623 INFO:tasks.workunit.client.1.smithi097.stdout:3/683: dread - d7/de/d3c/d3f/d60/fe1 zero size 2024-09-16T08:57:14.623 INFO:tasks.workunit.client.1.smithi097.stdout:3/684: chown d7/de/d3c/f73 2065308 1 2024-09-16T08:57:14.624 INFO:tasks.workunit.client.1.smithi097.stdout:8/640: rename d8/dd/d15/d16/fa4 to d8/dd/d71/d9c/fca 0 2024-09-16T08:57:14.625 INFO:tasks.workunit.client.1.smithi097.stdout:8/641: fsync d8/f13 0 2024-09-16T08:57:14.627 INFO:tasks.workunit.client.0.smithi031.stdout:7/326: creat d2/d8/d11/d5d/f73 x:0 0 0 2024-09-16T08:57:14.627 INFO:tasks.workunit.client.0.smithi031.stdout:7/327: creat d2/d8/d11/f74 x:0 0 0 2024-09-16T08:57:14.627 INFO:tasks.workunit.client.0.smithi031.stdout:7/328: chown d2/l34 510211 1 2024-09-16T08:57:14.628 INFO:tasks.workunit.client.0.smithi031.stdout:9/244: getdents d3/d5/da/db/d21 0 2024-09-16T08:57:14.631 INFO:tasks.workunit.client.0.smithi031.stdout:3/303: rename d3/f35 to d3/f59 0 2024-09-16T08:57:14.631 INFO:tasks.workunit.client.0.smithi031.stdout:3/304: dread - d3/d14/d37/d1a/d24/f40 zero size 2024-09-16T08:57:14.632 INFO:tasks.workunit.client.0.smithi031.stdout:3/305: chown d3/d14/d37/d1a/d1d/d3b/d3f/d49 113 1 2024-09-16T08:57:14.632 INFO:tasks.workunit.client.0.smithi031.stdout:3/306: fdatasync d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 0 2024-09-16T08:57:14.632 INFO:tasks.workunit.client.0.smithi031.stdout:3/307: dread - d3/d14/d37/d33/f4d zero size 2024-09-16T08:57:14.632 INFO:tasks.workunit.client.0.smithi031.stdout:3/308: fsync d3/d14/d37/d33/f39 0 2024-09-16T08:57:14.633 INFO:tasks.workunit.client.0.smithi031.stdout:5/278: truncate d2/dc/f3b 2162661 0 2024-09-16T08:57:14.633 INFO:tasks.workunit.client.1.smithi097.stdout:6/658: read d1/dd3/d9e/dca/f11 [306511,71933] 0 2024-09-16T08:57:14.633 INFO:tasks.workunit.client.1.smithi097.stdout:6/659: fsync d1/dd3/d9e/dca/d81/dc4/dc6/fcf 0 2024-09-16T08:57:14.633 INFO:tasks.workunit.client.1.smithi097.stdout:6/660: chown d1/dd3/d9e/dca/d16/d4f/l58 116383147 1 2024-09-16T08:57:14.634 INFO:tasks.workunit.client.1.smithi097.stdout:6/661: mkdir d1/dd3/d9e/dca/d81/dc4/ddc 0 2024-09-16T08:57:14.637 INFO:tasks.workunit.client.0.smithi031.stdout:7/329: dread d2/f56 [0,4194304] 0 2024-09-16T08:57:14.637 INFO:tasks.workunit.client.0.smithi031.stdout:7/330: read d2/d2d/f4b [6482,54585] 0 2024-09-16T08:57:14.638 INFO:tasks.workunit.client.0.smithi031.stdout:9/245: unlink d3/d5/da/db/l47 0 2024-09-16T08:57:14.638 INFO:tasks.workunit.client.0.smithi031.stdout:9/246: chown d3/d5/da/d34/d3c/d42/f44 4 1 2024-09-16T08:57:14.638 INFO:tasks.workunit.client.0.smithi031.stdout:9/247: chown d3/d5/da/d34/d3c/d42/f44 0 1 2024-09-16T08:57:14.639 INFO:tasks.workunit.client.0.smithi031.stdout:6/331: rename d0/dd/d2b/f33 to d0/d17/d19/f61 0 2024-09-16T08:57:14.640 INFO:tasks.workunit.client.0.smithi031.stdout:6/332: chown d0/dd/d35/l49 2279014 1 2024-09-16T08:57:14.640 INFO:tasks.workunit.client.0.smithi031.stdout:9/248: symlink d3/d5/da/db/d21/d3b/l57 0 2024-09-16T08:57:14.641 INFO:tasks.workunit.client.0.smithi031.stdout:6/333: symlink d0/dd/d35/d59/l62 0 2024-09-16T08:57:14.641 INFO:tasks.workunit.client.0.smithi031.stdout:6/334: fsync d0/d17/f2d 0 2024-09-16T08:57:14.643 INFO:tasks.workunit.client.0.smithi031.stdout:6/335: creat d0/dd/d35/d59/d3f/f63 x:0 0 0 2024-09-16T08:57:14.643 INFO:tasks.workunit.client.0.smithi031.stdout:6/336: readlink d0/l16 0 2024-09-16T08:57:14.643 INFO:tasks.workunit.client.0.smithi031.stdout:6/337: creat d0/dd/d35/f64 x:0 0 0 2024-09-16T08:57:14.644 INFO:tasks.workunit.client.0.smithi031.stdout:6/338: mkdir d0/dd/d35/d59/d3f/d65 0 2024-09-16T08:57:14.645 INFO:tasks.workunit.client.0.smithi031.stdout:6/339: link d0/dd/d35/d59/d3f/l53 d0/d17/d19/l66 0 2024-09-16T08:57:14.646 INFO:tasks.workunit.client.1.smithi097.stdout:4/623: sync 2024-09-16T08:57:14.646 INFO:tasks.workunit.client.1.smithi097.stdout:0/653: sync 2024-09-16T08:57:14.647 INFO:tasks.workunit.client.1.smithi097.stdout:7/579: sync 2024-09-16T08:57:14.647 INFO:tasks.workunit.client.1.smithi097.stdout:2/659: sync 2024-09-16T08:57:14.647 INFO:tasks.workunit.client.1.smithi097.stdout:2/660: write d3/d4/d68/d8b/d61/dac/fc0 [156143,113334] 0 2024-09-16T08:57:14.648 INFO:tasks.workunit.client.1.smithi097.stdout:1/735: sync 2024-09-16T08:57:14.648 INFO:tasks.workunit.client.1.smithi097.stdout:5/704: sync 2024-09-16T08:57:14.648 INFO:tasks.workunit.client.1.smithi097.stdout:1/736: fdatasync d5/d17/d54/d6b/f9c 0 2024-09-16T08:57:14.648 INFO:tasks.workunit.client.1.smithi097.stdout:5/705: dread - db/d1a/faa zero size 2024-09-16T08:57:14.648 INFO:tasks.workunit.client.1.smithi097.stdout:5/706: creat db/d18/d1c/d42/d46/d7e/fe9 x:0 0 0 2024-09-16T08:57:14.648 INFO:tasks.workunit.client.0.smithi031.stdout:3/309: dread d3/d14/d37/d33/f38 [0,4194304] 0 2024-09-16T08:57:14.649 INFO:tasks.workunit.client.1.smithi097.stdout:4/624: mkdir d0/d23/dda 0 2024-09-16T08:57:14.649 INFO:tasks.workunit.client.1.smithi097.stdout:1/737: read d5/d17/d54/d6b/f9c [924229,669] 0 2024-09-16T08:57:14.649 INFO:tasks.workunit.client.1.smithi097.stdout:6/662: dread d1/dd3/d9e/dca/f2f [4194304,4194304] 0 2024-09-16T08:57:14.649 INFO:tasks.workunit.client.1.smithi097.stdout:0/654: creat da/d24/d4c/d80/d42/fef x:0 0 0 2024-09-16T08:57:14.649 INFO:tasks.workunit.client.1.smithi097.stdout:6/663: write d1/dd3/d9e/db3/fbd [1028017,71349] 0 2024-09-16T08:57:14.649 INFO:tasks.workunit.client.1.smithi097.stdout:1/738: write d5/db/dc/d19/f3f [1744939,105050] 0 2024-09-16T08:57:14.649 INFO:tasks.workunit.client.1.smithi097.stdout:7/580: creat d4/d12/d1f/fc2 x:0 0 0 2024-09-16T08:57:14.650 INFO:tasks.workunit.client.0.smithi031.stdout:3/310: creat d3/f5a x:0 0 0 2024-09-16T08:57:14.650 INFO:tasks.workunit.client.0.smithi031.stdout:3/311: write d3/f2a [250863,95184] 0 2024-09-16T08:57:14.650 INFO:tasks.workunit.client.0.smithi031.stdout:7/331: dread d2/d8/d11/f45 [0,4194304] 0 2024-09-16T08:57:14.650 INFO:tasks.workunit.client.0.smithi031.stdout:7/332: stat d2/d8 0 2024-09-16T08:57:14.651 INFO:tasks.workunit.client.1.smithi097.stdout:4/625: creat d0/d36/fdb x:0 0 0 2024-09-16T08:57:14.651 INFO:tasks.workunit.client.0.smithi031.stdout:3/312: creat d3/d14/d37/d47/f5b x:0 0 0 2024-09-16T08:57:14.654 INFO:tasks.workunit.client.1.smithi097.stdout:6/664: rename d1/dd3/d9e/dca/d16/d4f/f78 to d1/dd3/d9e/dca/d16/fdd 0 2024-09-16T08:57:14.654 INFO:tasks.workunit.client.1.smithi097.stdout:6/665: chown d1/dd3/d9e/db3 28 1 2024-09-16T08:57:14.654 INFO:tasks.workunit.client.1.smithi097.stdout:1/739: unlink d5/d2c/dcb/l8b 0 2024-09-16T08:57:14.655 INFO:tasks.workunit.client.1.smithi097.stdout:7/581: mknod d4/d7/d77/da8/cc3 0 2024-09-16T08:57:14.656 INFO:tasks.workunit.client.1.smithi097.stdout:6/666: rmdir d1/dd3/d9e/dca/d81/dc4 39 2024-09-16T08:57:14.656 INFO:tasks.workunit.client.1.smithi097.stdout:6/667: chown d1/dd3/d9e/dca/f1c 43 1 2024-09-16T08:57:14.657 INFO:tasks.workunit.client.1.smithi097.stdout:7/582: mknod d4/d7/d7b/d43/cc4 0 2024-09-16T08:57:14.657 INFO:tasks.workunit.client.1.smithi097.stdout:7/583: truncate d4/d1b/d41/f95 888489 0 2024-09-16T08:57:14.657 INFO:tasks.workunit.client.1.smithi097.stdout:7/584: chown d4/d7/d7b/d43 49241 1 2024-09-16T08:57:14.657 INFO:tasks.workunit.client.1.smithi097.stdout:7/585: chown d4/l74 380458941 1 2024-09-16T08:57:14.658 INFO:tasks.workunit.client.1.smithi097.stdout:7/586: creat d4/d7/d7b/d43/daf/fc5 x:0 0 0 2024-09-16T08:57:14.659 INFO:tasks.workunit.client.1.smithi097.stdout:7/587: mkdir d4/d9/dc/dc6 0 2024-09-16T08:57:14.660 INFO:tasks.workunit.client.1.smithi097.stdout:7/588: dread - d4/d7/d7b/d43/fa7 zero size 2024-09-16T08:57:14.660 INFO:tasks.workunit.client.1.smithi097.stdout:7/589: truncate d4/d9/dc/d8e/f92 9509 0 2024-09-16T08:57:14.665 INFO:tasks.workunit.client.1.smithi097.stdout:4/626: dread d0/d7/fb [8388608,4194304] 0 2024-09-16T08:57:14.665 INFO:tasks.workunit.client.0.smithi031.stdout:7/333: dread d2/d8/d11/f24 [0,4194304] 0 2024-09-16T08:57:14.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/334: write d2/d2d/f4b [233501,80976] 0 2024-09-16T08:57:14.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/335: write d2/d8/f1d [235383,22086] 0 2024-09-16T08:57:14.667 INFO:tasks.workunit.client.0.smithi031.stdout:7/336: truncate d2/d8/d11/f40 3884669 0 2024-09-16T08:57:14.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/340: read d0/dd/d35/d59/d3f/f4b [1492127,66524] 0 2024-09-16T08:57:14.739 INFO:tasks.workunit.client.0.smithi031.stdout:6/341: mkdir d0/d17/d67 0 2024-09-16T08:57:14.739 INFO:tasks.workunit.client.0.smithi031.stdout:6/342: stat d0/dd/d35/d59/f47 0 2024-09-16T08:57:14.743 INFO:tasks.workunit.client.0.smithi031.stdout:2/191: dwrite d6/f8 [0,4194304] 0 2024-09-16T08:57:14.750 INFO:tasks.workunit.client.1.smithi097.stdout:9/606: dread d2/d5/d72/f89 [0,4194304] 0 2024-09-16T08:57:14.750 INFO:tasks.workunit.client.1.smithi097.stdout:9/607: fsync d2/d5/d56/f5d 0 2024-09-16T08:57:14.751 INFO:tasks.workunit.client.1.smithi097.stdout:2/661: dread d3/d4/d68/d8b/fba [0,4194304] 0 2024-09-16T08:57:14.752 INFO:tasks.workunit.client.1.smithi097.stdout:2/662: write d3/d4/d68/d8b/d61/fb1 [467748,49725] 0 2024-09-16T08:57:14.754 INFO:tasks.workunit.client.1.smithi097.stdout:2/663: link d3/d4/d68/d7f/fbd d3/d4/d8/d51/fc7 0 2024-09-16T08:57:14.754 INFO:tasks.workunit.client.1.smithi097.stdout:2/664: truncate d3/d4/d8/fad 848753 0 2024-09-16T08:57:14.756 INFO:tasks.workunit.client.1.smithi097.stdout:2/665: write d3/d4/d68/d8b/d7c/d7d/f6d [167091,13525] 0 2024-09-16T08:57:14.757 INFO:tasks.workunit.client.1.smithi097.stdout:3/685: dwrite d7/de/d3c/f4d [0,4194304] 0 2024-09-16T08:57:14.757 INFO:tasks.workunit.client.1.smithi097.stdout:3/686: write d7/d28/d30/f94 [2066495,75114] 0 2024-09-16T08:57:14.759 INFO:tasks.workunit.client.1.smithi097.stdout:2/666: rmdir d3/d4/d68/d8b/d7c/d7d/d53/da3/da4 0 2024-09-16T08:57:14.762 INFO:tasks.workunit.client.1.smithi097.stdout:3/687: symlink d7/de/d8d/lef 0 2024-09-16T08:57:14.762 INFO:tasks.workunit.client.1.smithi097.stdout:2/667: unlink d3/d4/d68/d7f/ca6 0 2024-09-16T08:57:14.764 INFO:tasks.workunit.client.1.smithi097.stdout:3/688: creat d7/de/d3c/dde/ff0 x:0 0 0 2024-09-16T08:57:14.764 INFO:tasks.workunit.client.1.smithi097.stdout:3/689: readlink d7/d15/d52/lc0 0 2024-09-16T08:57:14.764 INFO:tasks.workunit.client.1.smithi097.stdout:2/668: creat d3/d4/fc8 x:0 0 0 2024-09-16T08:57:14.766 INFO:tasks.workunit.client.1.smithi097.stdout:2/669: mknod d3/d4/d68/d8b/d61/cc9 0 2024-09-16T08:57:14.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/192: write d6/d20/f33 [3726144,81432] 0 2024-09-16T08:57:14.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/193: creat d6/d20/f3f x:0 0 0 2024-09-16T08:57:14.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/194: dread - d6/d20/d23/f3b zero size 2024-09-16T08:57:14.776 INFO:tasks.workunit.client.0.smithi031.stdout:2/195: rmdir d6/da/d37 39 2024-09-16T08:57:14.776 INFO:tasks.workunit.client.0.smithi031.stdout:2/196: fdatasync d6/da/f17 0 2024-09-16T08:57:14.776 INFO:tasks.workunit.client.0.smithi031.stdout:2/197: fdatasync d6/d20/f3f 0 2024-09-16T08:57:14.776 INFO:tasks.workunit.client.0.smithi031.stdout:2/198: chown d6/d20/d23/d2c/c30 106 1 2024-09-16T08:57:14.777 INFO:tasks.workunit.client.0.smithi031.stdout:5/279: dwrite d2/f34 [0,4194304] 0 2024-09-16T08:57:14.778 INFO:tasks.workunit.client.1.smithi097.stdout:1/740: dwrite d5/d14/fed [0,4194304] 0 2024-09-16T08:57:14.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/642: dwrite d8/dd/d42/f51 [0,4194304] 0 2024-09-16T08:57:14.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/643: chown d8/dd/d15/d2c/d4a/d4d/l6b 11752 1 2024-09-16T08:57:14.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/644: write d8/dd/d15/d16/d7c/da2/fc1 [1764433,54029] 0 2024-09-16T08:57:14.787 INFO:tasks.workunit.client.1.smithi097.stdout:8/645: unlink d8/dd/d71/d9c/ca8 0 2024-09-16T08:57:14.788 INFO:tasks.workunit.client.1.smithi097.stdout:6/668: dwrite d1/dd3/d9e/dca/d81/f88 [0,4194304] 0 2024-09-16T08:57:14.788 INFO:tasks.workunit.client.0.smithi031.stdout:9/249: dwrite d3/d5/da/db/d21/f3d [0,4194304] 0 2024-09-16T08:57:14.788 INFO:tasks.workunit.client.0.smithi031.stdout:3/313: dwrite d3/f2a [0,4194304] 0 2024-09-16T08:57:14.789 INFO:tasks.workunit.client.1.smithi097.stdout:5/707: dwrite db/d18/d1c/d42/d46/d7e/d76/fc0 [0,4194304] 0 2024-09-16T08:57:14.790 INFO:tasks.workunit.client.0.smithi031.stdout:5/280: dread d2/f5 [0,4194304] 0 2024-09-16T08:57:14.791 INFO:tasks.workunit.client.0.smithi031.stdout:6/343: dwrite d0/d17/d19/f1d [4194304,4194304] 0 2024-09-16T08:57:14.791 INFO:tasks.workunit.client.0.smithi031.stdout:6/344: readlink d0/dd/d35/d59/l1b 0 2024-09-16T08:57:14.792 INFO:tasks.workunit.client.1.smithi097.stdout:6/669: symlink d1/dc/lde 0 2024-09-16T08:57:14.792 INFO:tasks.workunit.client.1.smithi097.stdout:8/646: dread d8/dd/f2f [0,4194304] 0 2024-09-16T08:57:14.792 INFO:tasks.workunit.client.1.smithi097.stdout:8/647: write d8/dd/d15/d2c/f4b [2050354,32318] 0 2024-09-16T08:57:14.792 INFO:tasks.workunit.client.1.smithi097.stdout:9/608: dwrite d2/d30/d74/d8c/d94/d96/dac/f81 [0,4194304] 0 2024-09-16T08:57:14.792 INFO:tasks.workunit.client.1.smithi097.stdout:9/609: chown d2/d5/d56/cba 16 1 2024-09-16T08:57:14.795 INFO:tasks.workunit.client.0.smithi031.stdout:2/199: truncate d6/da/f27 589278 0 2024-09-16T08:57:14.796 INFO:tasks.workunit.client.0.smithi031.stdout:7/337: fsync d2/d8/d11/d5d/f73 0 2024-09-16T08:57:14.799 INFO:tasks.workunit.client.0.smithi031.stdout:3/314: creat d3/d14/d37/d1a/f5c x:0 0 0 2024-09-16T08:57:14.799 INFO:tasks.workunit.client.0.smithi031.stdout:3/315: read - d3/f5a zero size 2024-09-16T08:57:14.799 INFO:tasks.workunit.client.1.smithi097.stdout:9/610: dread d2/d5/d56/f8a [0,4194304] 0 2024-09-16T08:57:14.808 INFO:tasks.workunit.client.0.smithi031.stdout:9/250: dwrite d3/d5/da/d34/f35 [0,4194304] 0 2024-09-16T08:57:14.814 INFO:tasks.workunit.client.1.smithi097.stdout:0/655: rmdir da/d24/d4c/d80/d42 39 2024-09-16T08:57:14.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.817+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90a810e110 con 0x7f9094073d40 2024-09-16T08:57:14.819 INFO:tasks.workunit.client.1.smithi097.stdout:6/670: rmdir d1/dd3/d9e/dca/d16/d4f 39 2024-09-16T08:57:14.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.818+0000 7f909e7fc640 1 -- 172.21.15.31:0/2488715031 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f90a810e110 con 0x7f9094073d40 2024-09-16T08:57:14.822 INFO:tasks.workunit.client.1.smithi097.stdout:9/611: dwrite d2/d5/d56/f57 [0,4194304] 0 2024-09-16T08:57:14.822 INFO:tasks.workunit.client.1.smithi097.stdout:9/612: fdatasync d2/d5/fa 0 2024-09-16T08:57:14.822 INFO:tasks.workunit.client.1.smithi097.stdout:9/613: creat d2/d5/d56/d6b/fc3 x:0 0 0 2024-09-16T08:57:14.822 INFO:tasks.workunit.client.1.smithi097.stdout:9/614: stat d2/d5/d8d/f9f 0 2024-09-16T08:57:14.823 INFO:tasks.workunit.client.1.smithi097.stdout:9/615: creat d2/d5/d56/fc4 x:0 0 0 2024-09-16T08:57:14.823 INFO:tasks.workunit.client.0.smithi031.stdout:3/316: dwrite d3/fa [0,4194304] 0 2024-09-16T08:57:14.824 INFO:tasks.workunit.client.1.smithi097.stdout:8/648: mkdir d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb 0 2024-09-16T08:57:14.824 INFO:tasks.workunit.client.1.smithi097.stdout:8/649: dread - d8/dd/d71/d9c/fca zero size 2024-09-16T08:57:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9094073d40 msgr2=0x7f9094076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9094073d40 0x7f9094076200 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f90a8068be0 tx=0x7f90a40090e0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 msgr2=0x7f90a819e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a819e610 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f90a002fc30 tx=0x7f90a002fc60 comp rx=0 tx=0).stop 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 shutdown_connections 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9094073d40 0x7f9094076200 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90a8072000 0x7f90a819e610 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 --2- 172.21.15.31:0/2488715031 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90a80716c0 0x7f90a819e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.821+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 >> 172.21.15.31:0/2488715031 conn(0x7f90a806cfa0 msgr2=0x7f90a810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.822+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 shutdown_connections 2024-09-16T08:57:14.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.822+0000 7f90af6c8640 1 -- 172.21.15.31:0/2488715031 wait complete. 2024-09-16T08:57:14.826 INFO:tasks.workunit.client.0.smithi031.stdout:5/281: mknod d2/dc/d3d/c5a 0 2024-09-16T08:57:14.826 INFO:tasks.workunit.client.1.smithi097.stdout:4/627: write d0/d7/f75 [1034383,121849] 0 2024-09-16T08:57:14.826 INFO:tasks.workunit.client.1.smithi097.stdout:8/650: dread d8/dd/d15/d2c/d4a/d4e/db2/fbd [0,4194304] 0 2024-09-16T08:57:14.826 INFO:tasks.workunit.client.1.smithi097.stdout:8/651: fsync d8/dd/d15/d16/d7c/da2/fbb 0 2024-09-16T08:57:14.830 INFO:tasks.workunit.client.0.smithi031.stdout:6/345: creat d0/dd/d35/d59/d3f/d65/f68 x:0 0 0 2024-09-16T08:57:14.832 INFO:tasks.workunit.client.1.smithi097.stdout:0/656: creat da/d24/ff0 x:0 0 0 2024-09-16T08:57:14.834 INFO:tasks.workunit.client.0.smithi031.stdout:2/200: mkdir d6/d2e/d40 0 2024-09-16T08:57:14.836 INFO:tasks.workunit.client.1.smithi097.stdout:5/708: symlink db/d18/lea 0 2024-09-16T08:57:14.838 INFO:tasks.workunit.client.0.smithi031.stdout:7/338: mknod d2/d8/d11/d5d/d6e/c75 0 2024-09-16T08:57:14.839 INFO:tasks.workunit.client.1.smithi097.stdout:6/671: mknod d1/cdf 0 2024-09-16T08:57:14.843 INFO:tasks.workunit.client.0.smithi031.stdout:6/346: dwrite d0/dd/d35/f52 [0,4194304] 0 2024-09-16T08:57:14.844 INFO:tasks.workunit.client.0.smithi031.stdout:9/251: truncate d3/f2a 3638673 0 2024-09-16T08:57:14.844 INFO:tasks.workunit.client.0.smithi031.stdout:9/252: creat d3/d5/da/db/d21/d3b/f58 x:0 0 0 2024-09-16T08:57:14.845 INFO:tasks.workunit.client.1.smithi097.stdout:2/670: rmdir d3/d4 39 2024-09-16T08:57:14.848 INFO:tasks.workunit.client.1.smithi097.stdout:9/616: truncate d2/f4 235194 0 2024-09-16T08:57:14.848 INFO:tasks.workunit.client.1.smithi097.stdout:9/617: creat d2/d30/d74/d8c/d94/d96/dac/d79/fc5 x:0 0 0 2024-09-16T08:57:14.848 INFO:tasks.workunit.client.1.smithi097.stdout:9/618: dread - d2/d5/d56/faf zero size 2024-09-16T08:57:14.849 INFO:tasks.workunit.client.0.smithi031.stdout:3/317: unlink d3/d14/d37/d1a/f28 0 2024-09-16T08:57:14.851 INFO:tasks.workunit.client.1.smithi097.stdout:4/628: mkdir d0/d23/ddc 0 2024-09-16T08:57:14.851 INFO:tasks.workunit.client.1.smithi097.stdout:4/629: write d0/d23/d5f/fc9 [632721,10698] 0 2024-09-16T08:57:14.852 INFO:tasks.workunit.client.0.smithi031.stdout:5/282: mknod d2/dc/d1d/c5b 0 2024-09-16T08:57:14.852 INFO:tasks.workunit.client.0.smithi031.stdout:5/283: dread - d2/f2b zero size 2024-09-16T08:57:14.852 INFO:tasks.workunit.client.0.smithi031.stdout:6/347: dread d0/dd/d2b/f31 [0,4194304] 0 2024-09-16T08:57:14.853 INFO:tasks.workunit.client.1.smithi097.stdout:8/652: mknod d8/dd/d15/d2c/d4a/d4e/d97/ccc 0 2024-09-16T08:57:14.853 INFO:tasks.workunit.client.1.smithi097.stdout:8/653: read - d8/dd/d15/d2c/d4a/d4d/d3f/d99/fb0 zero size 2024-09-16T08:57:14.855 INFO:tasks.workunit.client.0.smithi031.stdout:2/201: rename d6/d20/d23 to d6/d41 0 2024-09-16T08:57:14.855 INFO:tasks.workunit.client.0.smithi031.stdout:2/202: chown d6/d41/d2c 27 1 2024-09-16T08:57:14.856 INFO:tasks.workunit.client.0.smithi031.stdout:2/203: dread - d6/d41/f2b zero size 2024-09-16T08:57:14.856 INFO:tasks.workunit.client.1.smithi097.stdout:1/741: truncate d5/d14/fed 169 0 2024-09-16T08:57:14.858 INFO:tasks.workunit.client.1.smithi097.stdout:0/657: rename da/d1a/d26/d47/d97/l90 to da/d7e/dde/lf1 0 2024-09-16T08:57:14.858 INFO:tasks.workunit.client.1.smithi097.stdout:0/658: stat da/d1a/d41/cbd 0 2024-09-16T08:57:14.858 INFO:tasks.workunit.client.1.smithi097.stdout:0/659: stat da/df/d8a/fd3 0 2024-09-16T08:57:14.858 INFO:tasks.workunit.client.1.smithi097.stdout:0/660: write da/d1a/d9c/fb9 [633088,108837] 0 2024-09-16T08:57:14.858 INFO:tasks.workunit.client.1.smithi097.stdout:0/661: stat da/d24/d76/f8b 0 2024-09-16T08:57:14.860 INFO:tasks.workunit.client.0.smithi031.stdout:7/339: link d2/d8/l2b d2/d2d/l76 0 2024-09-16T08:57:14.861 INFO:tasks.workunit.client.1.smithi097.stdout:0/662: dread da/df/d3d/f66 [0,4194304] 0 2024-09-16T08:57:14.861 INFO:tasks.workunit.client.1.smithi097.stdout:0/663: chown da/d24 16 1 2024-09-16T08:57:14.861 INFO:tasks.workunit.client.1.smithi097.stdout:0/664: truncate da/d1a/d26/d47/d97/d84/fc5 263604 0 2024-09-16T08:57:14.861 INFO:tasks.workunit.client.1.smithi097.stdout:0/665: write da/df/fc2 [673492,3373] 0 2024-09-16T08:57:14.861 INFO:tasks.workunit.client.1.smithi097.stdout:0/666: stat da/d1a/d26/d47/db3/d60/d68 0 2024-09-16T08:57:14.861 INFO:tasks.workunit.client.1.smithi097.stdout:0/667: creat da/d1a/d41/db2/ff2 x:0 0 0 2024-09-16T08:57:14.862 INFO:tasks.workunit.client.1.smithi097.stdout:5/709: creat db/d18/d1c/d42/d46/d7e/d76/dc4/feb x:0 0 0 2024-09-16T08:57:14.864 INFO:tasks.workunit.client.1.smithi097.stdout:6/672: truncate d1/dd3/d9e/dca/f2f 1581372 0 2024-09-16T08:57:14.864 INFO:tasks.workunit.client.1.smithi097.stdout:6/673: chown d1/dd3/l33 310 1 2024-09-16T08:57:14.864 INFO:tasks.workunit.client.1.smithi097.stdout:6/674: truncate d1/dc/f7d 4215123 0 2024-09-16T08:57:14.865 INFO:tasks.workunit.client.0.smithi031.stdout:3/318: link d3/d14/d37/d33/l29 d3/d14/d37/d1a/d1d/d3b/d3f/l5d 0 2024-09-16T08:57:14.867 INFO:tasks.workunit.client.1.smithi097.stdout:2/671: symlink d3/d4/d68/d7f/lca 0 2024-09-16T08:57:14.868 INFO:tasks.workunit.client.0.smithi031.stdout:6/348: truncate d0/d10/f3b 2084604 0 2024-09-16T08:57:14.872 INFO:tasks.workunit.client.0.smithi031.stdout:7/340: dread d2/d8/d11/f40 [0,4194304] 0 2024-09-16T08:57:14.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/341: creat d2/d2d/f77 x:0 0 0 2024-09-16T08:57:14.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/342: dread - d2/d8/d11/d66/f72 zero size 2024-09-16T08:57:14.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/343: write d2/d8/fd [4643116,120287] 0 2024-09-16T08:57:14.873 INFO:tasks.workunit.client.0.smithi031.stdout:7/344: chown d2/d8/d11/d5d 181256318 1 2024-09-16T08:57:14.873 INFO:tasks.workunit.client.1.smithi097.stdout:8/654: dwrite d8/dd/d15/d16/d7c/fb7 [0,4194304] 0 2024-09-16T08:57:14.873 INFO:tasks.workunit.client.1.smithi097.stdout:8/655: dread - d8/dd/d15/d2c/d4a/d4d/d62/fc2 zero size 2024-09-16T08:57:14.874 INFO:tasks.workunit.client.1.smithi097.stdout:0/668: dread da/df/f3f [0,4194304] 0 2024-09-16T08:57:14.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/253: rename d3/d5/da/d34/d3c/d42/f44 to d3/d48/f59 0 2024-09-16T08:57:14.876 INFO:tasks.workunit.client.1.smithi097.stdout:9/619: creat d2/d5/d3a/d3c/fc6 x:0 0 0 2024-09-16T08:57:14.877 INFO:tasks.workunit.client.0.smithi031.stdout:2/204: mkdir d6/da/d37/d42 0 2024-09-16T08:57:14.879 INFO:tasks.workunit.client.1.smithi097.stdout:4/630: mknod d0/d7/d52/d26/cdd 0 2024-09-16T08:57:14.879 INFO:tasks.workunit.client.0.smithi031.stdout:3/319: dread d3/d14/d37/d1a/d1d/f34 [0,4194304] 0 2024-09-16T08:57:14.880 INFO:tasks.workunit.client.0.smithi031.stdout:3/320: readlink d3/l19 0 2024-09-16T08:57:14.880 INFO:tasks.workunit.client.0.smithi031.stdout:3/321: stat d3/d14/d37/d1a/f5c 0 2024-09-16T08:57:14.880 INFO:tasks.workunit.client.0.smithi031.stdout:3/322: write d3/d14/d37/d33/f39 [289083,19996] 0 2024-09-16T08:57:14.883 INFO:tasks.workunit.client.0.smithi031.stdout:7/345: mknod d2/d8/d11/d5d/d6e/c78 0 2024-09-16T08:57:14.883 INFO:tasks.workunit.client.0.smithi031.stdout:7/346: write d2/d2d/f6f [171805,124912] 0 2024-09-16T08:57:14.887 INFO:tasks.workunit.client.0.smithi031.stdout:5/284: rename d2/dc/d25/d21/f29 to d2/dc/d25/d21/d2e/f5c 0 2024-09-16T08:57:14.887 INFO:tasks.workunit.client.0.smithi031.stdout:5/285: write d2/dc/f44 [941130,121462] 0 2024-09-16T08:57:14.888 INFO:tasks.workunit.client.1.smithi097.stdout:1/742: truncate d5/d17/d54/d6b/d71/d94/f99 3376473 0 2024-09-16T08:57:14.890 INFO:tasks.workunit.client.0.smithi031.stdout:9/254: link d3/d5/da/db/d21/d3b/l55 d3/d5/l5a 0 2024-09-16T08:57:14.890 INFO:tasks.workunit.client.0.smithi031.stdout:9/255: chown d3/d5/da/f2b 2040129 1 2024-09-16T08:57:14.891 INFO:tasks.workunit.client.0.smithi031.stdout:3/323: link d3/l30 d3/l5e 0 2024-09-16T08:57:14.895 INFO:tasks.workunit.client.1.smithi097.stdout:5/710: unlink db/d1a/f86 0 2024-09-16T08:57:14.895 INFO:tasks.workunit.client.1.smithi097.stdout:5/711: readlink db/d6e/lbe 0 2024-09-16T08:57:14.895 INFO:tasks.workunit.client.1.smithi097.stdout:5/712: fsync db/d1a/f5b 0 2024-09-16T08:57:14.897 INFO:tasks.workunit.client.0.smithi031.stdout:7/347: symlink d2/d8/d69/l79 0 2024-09-16T08:57:14.897 INFO:tasks.workunit.client.0.smithi031.stdout:6/349: rmdir d0/dd/d35/d59/d3f 39 2024-09-16T08:57:14.900 INFO:tasks.workunit.client.1.smithi097.stdout:6/675: symlink d1/dd3/d9e/le0 0 2024-09-16T08:57:14.900 INFO:tasks.workunit.client.0.smithi031.stdout:9/256: write d3/d5/f2d [1147888,128660] 0 2024-09-16T08:57:14.903 INFO:tasks.workunit.client.0.smithi031.stdout:2/205: rename d6/fb to d6/d20/f43 0 2024-09-16T08:57:14.903 INFO:tasks.workunit.client.0.smithi031.stdout:2/206: stat d6/l22 0 2024-09-16T08:57:14.904 INFO:tasks.workunit.client.1.smithi097.stdout:5/713: dread db/d1a/f38 [4194304,4194304] 0 2024-09-16T08:57:14.904 INFO:tasks.workunit.client.1.smithi097.stdout:5/714: dread - db/d18/d1c/d42/d46/d7e/f97 zero size 2024-09-16T08:57:14.904 INFO:tasks.workunit.client.1.smithi097.stdout:5/715: creat db/d18/d20/fec x:0 0 0 2024-09-16T08:57:14.904 INFO:tasks.workunit.client.0.smithi031.stdout:7/348: dread d2/d8/d11/f45 [0,4194304] 0 2024-09-16T08:57:14.907 INFO:tasks.workunit.client.1.smithi097.stdout:2/672: creat d3/d4/d8/d51/fcb x:0 0 0 2024-09-16T08:57:14.907 INFO:tasks.workunit.client.1.smithi097.stdout:2/673: dread - d3/d4/d68/d7f/fbd zero size 2024-09-16T08:57:14.907 INFO:tasks.workunit.client.1.smithi097.stdout:2/674: stat d3/d4/d9/f5a 0 2024-09-16T08:57:14.907 INFO:tasks.workunit.client.1.smithi097.stdout:8/656: creat d8/dd/d15/d2c/d4a/d4d/d3d/fcd x:0 0 0 2024-09-16T08:57:14.908 INFO:tasks.workunit.client.0.smithi031.stdout:5/286: mknod d2/dc/d3d/c5d 0 2024-09-16T08:57:14.909 INFO:tasks.workunit.client.0.smithi031.stdout:5/287: write d2/dc/d25/d21/f4e [778451,120877] 0 2024-09-16T08:57:14.910 INFO:tasks.workunit.client.0.smithi031.stdout:6/350: mknod d0/dd/d35/c69 0 2024-09-16T08:57:14.910 INFO:tasks.workunit.client.0.smithi031.stdout:6/351: truncate d0/dd/d2b/f45 465457 0 2024-09-16T08:57:14.911 INFO:tasks.workunit.client.1.smithi097.stdout:0/669: truncate da/d24/d4b/fa3 1674225 0 2024-09-16T08:57:14.914 INFO:tasks.workunit.client.1.smithi097.stdout:9/620: truncate d2/d3f/f43 3821618 0 2024-09-16T08:57:14.915 INFO:tasks.workunit.client.0.smithi031.stdout:7/349: unlink d2/d8/c21 0 2024-09-16T08:57:14.915 INFO:tasks.workunit.client.0.smithi031.stdout:7/350: fdatasync d2/d8/f57 0 2024-09-16T08:57:14.917 INFO:tasks.workunit.client.1.smithi097.stdout:4/631: truncate d0/d7/df/f8f 628314 0 2024-09-16T08:57:14.920 INFO:tasks.workunit.client.0.smithi031.stdout:3/324: dwrite d3/d14/f2f [0,4194304] 0 2024-09-16T08:57:14.921 INFO:tasks.workunit.client.1.smithi097.stdout:0/670: write da/d24/d76/f9a [2189353,49015] 0 2024-09-16T08:57:14.922 INFO:tasks.workunit.client.1.smithi097.stdout:1/743: mknod d5/d17/d54/d6b/d71/dd3/cf0 0 2024-09-16T08:57:14.923 INFO:tasks.workunit.client.0.smithi031.stdout:5/288: unlink d2/dc/d25/d21/f4f 0 2024-09-16T08:57:14.925 INFO:tasks.workunit.client.0.smithi031.stdout:5/289: write d2/dc/f3a [324564,70676] 0 2024-09-16T08:57:14.927 INFO:tasks.workunit.client.1.smithi097.stdout:6/676: symlink d1/dd3/le1 0 2024-09-16T08:57:14.930 INFO:tasks.workunit.client.0.smithi031.stdout:7/351: mkdir d2/d8/d69/d7a 0 2024-09-16T08:57:14.936 INFO:tasks.workunit.client.1.smithi097.stdout:5/716: rename db/d6e/d79/dba/ce3 to db/d6e/ced 0 2024-09-16T08:57:14.941 INFO:tasks.workunit.client.1.smithi097.stdout:6/677: dread d1/dd3/d9e/dca/f2c [0,4194304] 0 2024-09-16T08:57:14.944 INFO:tasks.workunit.client.1.smithi097.stdout:0/671: creat da/d24/d76/ff3 x:0 0 0 2024-09-16T08:57:14.945 INFO:tasks.workunit.client.0.smithi031.stdout:9/257: dwrite d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:14.949 INFO:tasks.workunit.client.0.smithi031.stdout:2/207: dwrite d6/f9 [4194304,4194304] 0 2024-09-16T08:57:14.950 INFO:tasks.workunit.client.1.smithi097.stdout:9/621: dwrite d2/d3f/f4b [0,4194304] 0 2024-09-16T08:57:14.952 INFO:tasks.workunit.client.1.smithi097.stdout:0/672: rmdir da/d1a/d26/d47/d97/dd0 0 2024-09-16T08:57:14.952 INFO:tasks.workunit.client.1.smithi097.stdout:0/673: dread - da/d1a/d26/d47/db3/d60/d73/d85/f8d zero size 2024-09-16T08:57:14.952 INFO:tasks.workunit.client.1.smithi097.stdout:0/674: chown da/df/f3f 6725504 1 2024-09-16T08:57:14.958 INFO:tasks.workunit.client.1.smithi097.stdout:2/675: dwrite d3/d4/d9/d67/f84 [0,4194304] 0 2024-09-16T08:57:14.959 INFO:tasks.workunit.client.1.smithi097.stdout:6/678: write d1/dd3/d9e/dca/d81/f88 [1657760,103649] 0 2024-09-16T08:57:14.959 INFO:tasks.workunit.client.1.smithi097.stdout:1/744: dwrite d5/db/dc/f41 [0,4194304] 0 2024-09-16T08:57:14.959 INFO:tasks.workunit.client.0.smithi031.stdout:3/325: getdents d3/d14/d37/d1a/d1d/d1e 0 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.958+0000 7f33caffd640 1 -- 172.21.15.31:0/2266358747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40ffca0 msgr2=0x7f33c4100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.958+0000 7f33caffd640 1 --2- 172.21.15.31:0/2266358747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40ffca0 0x7f33c4100120 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009c20 tx=0x7f33b402a980 comp rx=0 tx=0).stop 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.959+0000 7f33caffd640 1 -- 172.21.15.31:0/2266358747 shutdown_connections 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.959+0000 7f33caffd640 1 --2- 172.21.15.31:0/2266358747 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40ffca0 0x7f33c4100120 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.959+0000 7f33caffd640 1 --2- 172.21.15.31:0/2266358747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40feaa0 0x7f33c40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.959+0000 7f33caffd640 1 -- 172.21.15.31:0/2266358747 >> 172.21.15.31:0/2266358747 conn(0x7f33c40fa230 msgr2=0x7f33c40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.959+0000 7f33caffd640 1 -- 172.21.15.31:0/2266358747 shutdown_connections 2024-09-16T08:57:14.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.959+0000 7f33caffd640 1 -- 172.21.15.31:0/2266358747 wait complete. 2024-09-16T08:57:14.962 INFO:tasks.workunit.client.0.smithi031.stdout:6/352: rmdir d0/dd/d35/d59 39 2024-09-16T08:57:14.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.960+0000 7f33caffd640 1 Processor -- start 2024-09-16T08:57:14.962 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.960+0000 7f33caffd640 1 -- start start 2024-09-16T08:57:14.963 INFO:tasks.workunit.client.1.smithi097.stdout:5/717: dwrite db/d18/d1c/d42/d46/d7e/d8f/fd8 [0,4194304] 0 2024-09-16T08:57:14.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.961+0000 7f33caffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40feaa0 0x7f33c4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:14.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.961+0000 7f33caffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40ffca0 0x7f33c4197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:14.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.961+0000 7f33caffd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c4198450 con 0x7f33c40feaa0 2024-09-16T08:57:14.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.961+0000 7f33caffd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c41985c0 con 0x7f33c40ffca0 2024-09-16T08:57:14.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.962+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40ffca0 0x7f33c4197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:14.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.962+0000 7f33c3fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40ffca0 0x7f33c4197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:40638/0 (socket says 172.21.15.31:40638) 2024-09-16T08:57:14.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.962+0000 7f33c3fff640 1 -- 172.21.15.31:0/2522629635 learned_addr learned my addr 172.21.15.31:0/2522629635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:14.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.963+0000 7f33c8d99640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40feaa0 0x7f33c4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:14.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.963+0000 7f33c3fff640 1 -- 172.21.15.31:0/2522629635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40feaa0 msgr2=0x7f33c4197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:14.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.963+0000 7f33c3fff640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40feaa0 0x7f33c4197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:14.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.963+0000 7f33c3fff640 1 -- 172.21.15.31:0/2522629635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4009900 con 0x7f33c40ffca0 2024-09-16T08:57:14.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.964+0000 7f33c3fff640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40ffca0 0x7f33c4197e80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33b402fa50 tx=0x7f33b4030680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:14.967 INFO:tasks.workunit.client.1.smithi097.stdout:9/622: symlink d2/d30/d74/d8c/lc7 0 2024-09-16T08:57:14.967 INFO:tasks.workunit.client.0.smithi031.stdout:1/225: sync 2024-09-16T08:57:14.967 INFO:tasks.workunit.client.0.smithi031.stdout:1/226: truncate d2/d4/f31 427160 0 2024-09-16T08:57:14.967 INFO:tasks.workunit.client.0.smithi031.stdout:1/227: chown d2/d4/d22/l48 89188397 1 2024-09-16T08:57:14.968 INFO:tasks.workunit.client.1.smithi097.stdout:4/632: dwrite d0/d35/f5c [0,4194304] 0 2024-09-16T08:57:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.965+0000 7f33c1ffb640 1 -- 172.21.15.31:0/2522629635 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33b4003970 con 0x7f33c40ffca0 2024-09-16T08:57:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.965+0000 7f33c1ffb640 1 -- 172.21.15.31:0/2522629635 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33b4004040 con 0x7f33c40ffca0 2024-09-16T08:57:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.965+0000 7f33c1ffb640 1 -- 172.21.15.31:0/2522629635 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33b40396e0 con 0x7f33c40ffca0 2024-09-16T08:57:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.965+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c419d000 con 0x7f33c40ffca0 2024-09-16T08:57:14.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.965+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c419d510 con 0x7f33c40ffca0 2024-09-16T08:57:14.968 INFO:tasks.workunit.client.0.smithi031.stdout:3/326: write d3/fa [618016,21196] 0 2024-09-16T08:57:14.970 INFO:tasks.workunit.client.1.smithi097.stdout:6/679: dread d1/dd3/d9e/dca/d81/dad/f62 [0,4194304] 0 2024-09-16T08:57:14.971 INFO:tasks.workunit.client.1.smithi097.stdout:0/675: fdatasync da/d24/d4c/d80/d3b/f5a 0 2024-09-16T08:57:14.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.970+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a4005280 con 0x7f33c40ffca0 2024-09-16T08:57:14.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.971+0000 7f33c1ffb640 1 -- 172.21.15.31:0/2522629635 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f33b400a040 con 0x7f33c40ffca0 2024-09-16T08:57:14.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.972+0000 7f33c1ffb640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f33b0073d80 0x7f33b0076240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:14.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.972+0000 7f33c1ffb640 1 -- 172.21.15.31:0/2522629635 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f33b40b6260 con 0x7f33c40ffca0 2024-09-16T08:57:14.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.973+0000 7f33c8d99640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f33b0073d80 0x7f33b0076240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:14.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.974+0000 7f33c8d99640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f33b0073d80 0x7f33b0076240 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33c40ffb00 tx=0x7f33bc008040 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:14.978 INFO:tasks.workunit.client.0.smithi031.stdout:7/352: symlink d2/d8/d11/d5d/d6e/l7b 0 2024-09-16T08:57:14.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:14.978+0000 7f33c1ffb640 1 -- 172.21.15.31:0/2522629635 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f33b4079800 con 0x7f33c40ffca0 2024-09-16T08:57:14.981 INFO:tasks.workunit.client.1.smithi097.stdout:4/633: dread d0/d7/d52/f50 [4194304,4194304] 0 2024-09-16T08:57:14.981 INFO:tasks.workunit.client.1.smithi097.stdout:4/634: write d0/d7/f5b [633380,53222] 0 2024-09-16T08:57:14.983 INFO:tasks.workunit.client.0.smithi031.stdout:1/228: dread d2/de/f2f [0,4194304] 0 2024-09-16T08:57:14.984 INFO:tasks.workunit.client.0.smithi031.stdout:1/229: dread d2/d4/f31 [0,4194304] 0 2024-09-16T08:57:14.985 INFO:tasks.workunit.client.1.smithi097.stdout:1/745: creat d5/db/dc/d19/ff1 x:0 0 0 2024-09-16T08:57:14.985 INFO:tasks.workunit.client.1.smithi097.stdout:1/746: truncate d5/fd9 1245078 0 2024-09-16T08:57:14.987 INFO:tasks.workunit.client.0.smithi031.stdout:9/258: link d3/d48/f59 d3/f5b 0 2024-09-16T08:57:14.990 INFO:tasks.workunit.client.0.smithi031.stdout:3/327: symlink d3/d14/d37/d1a/d1d/d1e/l5f 0 2024-09-16T08:57:14.992 INFO:tasks.workunit.client.0.smithi031.stdout:2/208: dwrite d6/f16 [0,4194304] 0 2024-09-16T08:57:14.993 INFO:tasks.workunit.client.0.smithi031.stdout:5/290: rmdir d2/dc/d25/d21/d2e 39 2024-09-16T08:57:14.995 INFO:tasks.workunit.client.1.smithi097.stdout:5/718: mknod db/d1a/d6d/dc6/cee 0 2024-09-16T08:57:14.995 INFO:tasks.workunit.client.1.smithi097.stdout:5/719: write db/d1a/f2c [838644,98787] 0 2024-09-16T08:57:14.998 INFO:tasks.workunit.client.0.smithi031.stdout:7/353: mknod d2/d2d/c7c 0 2024-09-16T08:57:14.999 INFO:tasks.workunit.client.0.smithi031.stdout:7/354: chown d2/d8/d11/f40 5 1 2024-09-16T08:57:14.999 INFO:tasks.workunit.client.0.smithi031.stdout:7/355: write d2/d8/d11/d5d/d6e/f71 [923407,42285] 0 2024-09-16T08:57:15.001 INFO:tasks.workunit.client.1.smithi097.stdout:0/676: link da/d1a/d26/d47/db3/d60/d68/f5d da/d24/d4b/ff4 0 2024-09-16T08:57:15.002 INFO:tasks.workunit.client.1.smithi097.stdout:6/680: dwrite d1/dc/d3c/f6d [4194304,4194304] 0 2024-09-16T08:57:15.009 INFO:tasks.workunit.client.0.smithi031.stdout:6/353: dwrite d0/d10/f5a [0,4194304] 0 2024-09-16T08:57:15.009 INFO:tasks.workunit.client.0.smithi031.stdout:1/230: unlink d2/d4/d22/f4e 0 2024-09-16T08:57:15.011 INFO:tasks.workunit.client.1.smithi097.stdout:5/720: write db/d18/d20/f29 [1756509,12465] 0 2024-09-16T08:57:15.011 INFO:tasks.workunit.client.1.smithi097.stdout:2/676: dwrite d3/d4/d9/d78/f94 [0,4194304] 0 2024-09-16T08:57:15.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:14 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:15.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:14 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:15.012 INFO:tasks.workunit.client.1.smithi097.stdout:1/747: rename d5/d2c/f2e to d5/d2c/dcb/ff2 0 2024-09-16T08:57:15.015 INFO:tasks.workunit.client.0.smithi031.stdout:2/209: mkdir d6/da/d37/d44 0 2024-09-16T08:57:15.017 INFO:tasks.workunit.client.1.smithi097.stdout:6/681: mknod d1/dd3/d9e/dca/db6/ce2 0 2024-09-16T08:57:15.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/291: mknod d2/dc/c5e 0 2024-09-16T08:57:15.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/292: write d2/f59 [530514,8949] 0 2024-09-16T08:57:15.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/293: write d2/f5 [212871,122517] 0 2024-09-16T08:57:15.019 INFO:tasks.workunit.client.1.smithi097.stdout:3/690: sync 2024-09-16T08:57:15.019 INFO:tasks.workunit.client.1.smithi097.stdout:7/590: sync 2024-09-16T08:57:15.019 INFO:tasks.workunit.client.1.smithi097.stdout:3/691: truncate d7/de/d8d/d90/d92/fdd 849810 0 2024-09-16T08:57:15.019 INFO:tasks.workunit.client.1.smithi097.stdout:0/677: dwrite da/d1a/d26/d47/db3/d60/d73/d85/fcd [0,4194304] 0 2024-09-16T08:57:15.024 INFO:tasks.workunit.client.1.smithi097.stdout:2/677: dread d3/d4/d9/f5a [0,4194304] 0 2024-09-16T08:57:15.024 INFO:tasks.workunit.client.1.smithi097.stdout:2/678: fsync d3/d4/d77/f96 0 2024-09-16T08:57:15.025 INFO:tasks.workunit.client.1.smithi097.stdout:5/721: link db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 db/d1a/d6d/dc6/fef 0 2024-09-16T08:57:15.029 INFO:tasks.workunit.client.0.smithi031.stdout:5/294: write d2/f13 [2260269,70162] 0 2024-09-16T08:57:15.030 INFO:tasks.workunit.client.0.smithi031.stdout:4/135: sync 2024-09-16T08:57:15.030 INFO:tasks.workunit.client.0.smithi031.stdout:0/243: sync 2024-09-16T08:57:15.030 INFO:tasks.workunit.client.0.smithi031.stdout:8/210: sync 2024-09-16T08:57:15.030 INFO:tasks.workunit.client.1.smithi097.stdout:1/748: mkdir d5/d17/d54/d6b/da8/dc5/dc7/df3 0 2024-09-16T08:57:15.031 INFO:tasks.workunit.client.1.smithi097.stdout:1/749: write d5/d17/fc9 [2722661,91210] 0 2024-09-16T08:57:15.031 INFO:tasks.workunit.client.1.smithi097.stdout:1/750: fdatasync d5/db/dc/fad 0 2024-09-16T08:57:15.033 INFO:tasks.workunit.client.1.smithi097.stdout:6/682: mkdir d1/dd3/d9e/db3/de3 0 2024-09-16T08:57:15.033 INFO:tasks.workunit.client.1.smithi097.stdout:6/683: readlink d1/dc/la8 0 2024-09-16T08:57:15.033 INFO:tasks.workunit.client.1.smithi097.stdout:6/684: fdatasync d1/dd3/d9e/dca/d16/d4f/f92 0 2024-09-16T08:57:15.036 INFO:tasks.workunit.client.1.smithi097.stdout:7/591: mknod d4/d7/d30/cc7 0 2024-09-16T08:57:15.040 INFO:tasks.workunit.client.0.smithi031.stdout:1/231: mknod d2/d4/d36/c4f 0 2024-09-16T08:57:15.041 INFO:tasks.workunit.client.0.smithi031.stdout:0/244: dread d6/f27 [0,4194304] 0 2024-09-16T08:57:15.043 INFO:tasks.workunit.client.1.smithi097.stdout:3/692: mkdir d7/d28/d30/d88/df1 0 2024-09-16T08:57:15.043 INFO:tasks.workunit.client.1.smithi097.stdout:3/693: read - d7/de/d8d/d90/d92/fb1 zero size 2024-09-16T08:57:15.051 INFO:tasks.workunit.client.0.smithi031.stdout:2/210: rename d6/d41/d2c/l3e to d6/da/l45 0 2024-09-16T08:57:15.054 INFO:tasks.workunit.client.1.smithi097.stdout:0/678: mkdir da/d24/df5 0 2024-09-16T08:57:15.057 INFO:tasks.workunit.client.1.smithi097.stdout:5/722: creat db/d18/d1c/d42/d46/ff0 x:0 0 0 2024-09-16T08:57:15.062 INFO:tasks.workunit.client.0.smithi031.stdout:5/295: symlink d2/dc/d25/d21/d2e/l5f 0 2024-09-16T08:57:15.062 INFO:tasks.workunit.client.0.smithi031.stdout:5/296: read - d2/f2b zero size 2024-09-16T08:57:15.063 INFO:tasks.workunit.client.0.smithi031.stdout:3/328: mkdir d3/d14/d37/d1a/d1d/d3b/d60 0 2024-09-16T08:57:15.063 INFO:tasks.workunit.client.0.smithi031.stdout:3/329: chown d3/d14/d37/d1a/d24/l45 14101 1 2024-09-16T08:57:15.063 INFO:tasks.workunit.client.0.smithi031.stdout:3/330: write d3/fa [3145074,119905] 0 2024-09-16T08:57:15.063 INFO:tasks.workunit.client.0.smithi031.stdout:6/354: dwrite d0/d17/d19/f2e [0,4194304] 0 2024-09-16T08:57:15.065 INFO:tasks.workunit.client.1.smithi097.stdout:6/685: symlink d1/dd3/d9e/dca/d16/le4 0 2024-09-16T08:57:15.065 INFO:tasks.workunit.client.0.smithi031.stdout:4/136: rmdir d0/d2/d16 39 2024-09-16T08:57:15.065 INFO:tasks.workunit.client.1.smithi097.stdout:6/686: write d1/dd3/f84 [67617,72234] 0 2024-09-16T08:57:15.070 INFO:tasks.workunit.client.0.smithi031.stdout:6/355: dread d0/d17/f18 [4194304,4194304] 0 2024-09-16T08:57:15.071 INFO:tasks.workunit.client.0.smithi031.stdout:8/211: link d4/d7/c11 d4/d7/d21/d24/c4b 0 2024-09-16T08:57:15.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/245: truncate d6/d12/f16 2289260 0 2024-09-16T08:57:15.077 INFO:tasks.workunit.client.0.smithi031.stdout:0/246: write d6/d12/d47/f4e [4716112,3641] 0 2024-09-16T08:57:15.078 INFO:tasks.workunit.client.1.smithi097.stdout:2/679: dwrite d3/d1b/f83 [0,4194304] 0 2024-09-16T08:57:15.078 INFO:tasks.workunit.client.1.smithi097.stdout:2/680: fdatasync d3/d4/d9/d67/f84 0 2024-09-16T08:57:15.078 INFO:tasks.workunit.client.1.smithi097.stdout:2/681: write d3/d4/d9/d67/fb6 [856053,83128] 0 2024-09-16T08:57:15.079 INFO:tasks.workunit.client.0.smithi031.stdout:0/247: dread d6/d12/f22 [0,4194304] 0 2024-09-16T08:57:15.079 INFO:tasks.workunit.client.0.smithi031.stdout:0/248: chown d6/f42 202862160 1 2024-09-16T08:57:15.080 INFO:tasks.workunit.client.0.smithi031.stdout:5/297: creat d2/dc/d25/d21/f60 x:0 0 0 2024-09-16T08:57:15.084 INFO:tasks.workunit.client.0.smithi031.stdout:0/249: dread d6/f27 [0,4194304] 0 2024-09-16T08:57:15.084 INFO:tasks.workunit.client.0.smithi031.stdout:1/232: dwrite d2/d15/d1f/d43/f4c [0,4194304] 0 2024-09-16T08:57:15.089 INFO:tasks.workunit.client.1.smithi097.stdout:1/751: dwrite d5/d70/f72 [0,4194304] 0 2024-09-16T08:57:15.089 INFO:tasks.workunit.client.1.smithi097.stdout:1/752: fdatasync d5/d2c/d3b/fb7 0 2024-09-16T08:57:15.089 INFO:tasks.workunit.client.0.smithi031.stdout:4/137: creat d0/f2d x:0 0 0 2024-09-16T08:57:15.089 INFO:tasks.workunit.client.0.smithi031.stdout:6/356: symlink d0/dd/d35/d59/l6a 0 2024-09-16T08:57:15.090 INFO:tasks.workunit.client.0.smithi031.stdout:6/357: chown d0/dd/d35/d59/d3f/f63 3 1 2024-09-16T08:57:15.090 INFO:tasks.workunit.client.1.smithi097.stdout:7/592: dwrite d4/d1b/d41/f81 [4194304,4194304] 0 2024-09-16T08:57:15.090 INFO:tasks.workunit.client.0.smithi031.stdout:3/331: mknod d3/d14/d37/d1a/d1d/d3b/d3f/c61 0 2024-09-16T08:57:15.093 INFO:tasks.workunit.client.0.smithi031.stdout:2/211: dwrite d6/fc [0,4194304] 0 2024-09-16T08:57:15.093 INFO:tasks.workunit.client.0.smithi031.stdout:2/212: fsync d6/da/fd 0 2024-09-16T08:57:15.094 INFO:tasks.workunit.client.0.smithi031.stdout:2/213: fsync d6/da/ff 0 2024-09-16T08:57:15.094 INFO:tasks.workunit.client.0.smithi031.stdout:2/214: chown d6/da/d37/d42 916 1 2024-09-16T08:57:15.094 INFO:tasks.workunit.client.0.smithi031.stdout:2/215: dread - d6/d41/d2c/f34 zero size 2024-09-16T08:57:15.095 INFO:tasks.workunit.client.1.smithi097.stdout:6/687: dwrite d1/db7/fce [0,4194304] 0 2024-09-16T08:57:15.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/138: unlink d0/d2/l12 0 2024-09-16T08:57:15.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/139: creat d0/f2e x:0 0 0 2024-09-16T08:57:15.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/140: chown d0/d2/d16/c20 414 1 2024-09-16T08:57:15.097 INFO:tasks.workunit.client.0.smithi031.stdout:4/141: truncate d0/d2/d16/f19 1344470 0 2024-09-16T08:57:15.098 INFO:tasks.workunit.client.0.smithi031.stdout:6/358: mknod d0/d17/d5e/c6b 0 2024-09-16T08:57:15.100 INFO:tasks.workunit.client.1.smithi097.stdout:3/694: creat d7/de/d9c/ff2 x:0 0 0 2024-09-16T08:57:15.100 INFO:tasks.workunit.client.1.smithi097.stdout:0/679: creat da/d1a/d26/d94/ff6 x:0 0 0 2024-09-16T08:57:15.101 INFO:tasks.workunit.client.1.smithi097.stdout:5/723: mkdir db/d18/d20/d2b/df1 0 2024-09-16T08:57:15.101 INFO:tasks.workunit.client.1.smithi097.stdout:5/724: stat db/d18/d1c/d42/d46/f9b 0 2024-09-16T08:57:15.102 INFO:tasks.workunit.client.0.smithi031.stdout:3/332: rename d3/d14/d37/d33/f38 to d3/d14/d37/d1a/d1d/d3b/d3f/f62 0 2024-09-16T08:57:15.102 INFO:tasks.workunit.client.0.smithi031.stdout:3/333: chown d3/f5a 4983 1 2024-09-16T08:57:15.103 INFO:tasks.workunit.client.1.smithi097.stdout:2/682: write d3/d4/d68/d8b/d7c/d7d/f6b [600640,51955] 0 2024-09-16T08:57:15.103 INFO:tasks.workunit.client.1.smithi097.stdout:2/683: fsync d3/d4/d9/f4f 0 2024-09-16T08:57:15.103 INFO:tasks.workunit.client.1.smithi097.stdout:2/684: fdatasync d3/d4/d9/d67/db2/fbc 0 2024-09-16T08:57:15.103 INFO:tasks.workunit.client.1.smithi097.stdout:2/685: write d3/d4/d77/f70 [930081,68431] 0 2024-09-16T08:57:15.108 INFO:tasks.workunit.client.0.smithi031.stdout:4/142: mknod d0/d2/d16/d1e/c2f 0 2024-09-16T08:57:15.108 INFO:tasks.workunit.client.0.smithi031.stdout:4/143: getdents d0/d2/d16/d2b 0 2024-09-16T08:57:15.108 INFO:tasks.workunit.client.0.smithi031.stdout:4/144: stat d0/f1 0 2024-09-16T08:57:15.109 INFO:tasks.workunit.client.1.smithi097.stdout:6/688: dread d1/dd3/d9e/dca/d16/f89 [0,4194304] 0 2024-09-16T08:57:15.109 INFO:tasks.workunit.client.0.smithi031.stdout:6/359: fsync d0/d17/d19/f32 0 2024-09-16T08:57:15.111 INFO:tasks.workunit.client.0.smithi031.stdout:3/334: mkdir d3/d14/d37/d1a/d24/d63 0 2024-09-16T08:57:15.113 INFO:tasks.workunit.client.0.smithi031.stdout:2/216: rmdir d6/d2e 39 2024-09-16T08:57:15.113 INFO:tasks.workunit.client.0.smithi031.stdout:2/217: read d6/d20/f33 [1852261,125955] 0 2024-09-16T08:57:15.114 INFO:tasks.workunit.client.1.smithi097.stdout:1/753: unlink d5/d2c/dcb/l42 0 2024-09-16T08:57:15.114 INFO:tasks.workunit.client.1.smithi097.stdout:5/725: dread db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/f99 [0,4194304] 0 2024-09-16T08:57:15.114 INFO:tasks.workunit.client.1.smithi097.stdout:5/726: truncate db/d18/d1c/d42/d46/d7e/f51 4913989 0 2024-09-16T08:57:15.115 INFO:tasks.workunit.client.0.smithi031.stdout:6/360: mkdir d0/dd/d35/d6c 0 2024-09-16T08:57:15.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/335: write d3/f4 [4219204,50257] 0 2024-09-16T08:57:15.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/336: read - d3/d14/d37/d1a/d24/f40 zero size 2024-09-16T08:57:15.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/337: chown d3/d14/c53 11863 1 2024-09-16T08:57:15.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/338: readlink d3/l15 0 2024-09-16T08:57:15.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/339: stat d3/d14/c50 0 2024-09-16T08:57:15.116 INFO:tasks.workunit.client.0.smithi031.stdout:3/340: stat d3/d14/d37/d47/f5b 0 2024-09-16T08:57:15.120 INFO:tasks.workunit.client.0.smithi031.stdout:1/233: dwrite d2/d4/f1c [0,4194304] 0 2024-09-16T08:57:15.120 INFO:tasks.workunit.client.0.smithi031.stdout:2/218: unlink d6/d20/f33 0 2024-09-16T08:57:15.120 INFO:tasks.workunit.client.0.smithi031.stdout:2/219: readlink d6/d41/d2c/l3c 0 2024-09-16T08:57:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:14 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:14 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:15.123 INFO:tasks.workunit.client.1.smithi097.stdout:7/593: dwrite d4/d7/d7b/d43/fa7 [0,4194304] 0 2024-09-16T08:57:15.124 INFO:tasks.workunit.client.0.smithi031.stdout:2/220: write d6/da/fe [1568535,38531] 0 2024-09-16T08:57:15.124 INFO:tasks.workunit.client.0.smithi031.stdout:2/221: read - d6/d20/f26 zero size 2024-09-16T08:57:15.125 INFO:tasks.workunit.client.0.smithi031.stdout:6/361: mknod d0/dd/d35/d59/c6d 0 2024-09-16T08:57:15.126 INFO:tasks.workunit.client.0.smithi031.stdout:5/298: dwrite d2/dc/f3a [0,4194304] 0 2024-09-16T08:57:15.127 INFO:tasks.workunit.client.0.smithi031.stdout:6/362: dread d0/dd/d35/f52 [4194304,4194304] 0 2024-09-16T08:57:15.128 INFO:tasks.workunit.client.0.smithi031.stdout:3/341: symlink d3/l64 0 2024-09-16T08:57:15.128 INFO:tasks.workunit.client.0.smithi031.stdout:3/342: chown d3/fa 250799 1 2024-09-16T08:57:15.128 INFO:tasks.workunit.client.0.smithi031.stdout:3/343: creat d3/d14/d37/d1a/f65 x:0 0 0 2024-09-16T08:57:15.128 INFO:tasks.workunit.client.0.smithi031.stdout:3/344: creat d3/d14/d37/d47/f66 x:0 0 0 2024-09-16T08:57:15.130 INFO:tasks.workunit.client.0.smithi031.stdout:3/345: dread d3/d14/f2e [0,4194304] 0 2024-09-16T08:57:15.130 INFO:tasks.workunit.client.0.smithi031.stdout:3/346: readlink d3/d14/d37/d1a/d24/l32 0 2024-09-16T08:57:15.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/363: read d0/ff [690731,68081] 0 2024-09-16T08:57:15.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/364: fsync d0/d17/f30 0 2024-09-16T08:57:15.136 INFO:tasks.workunit.client.0.smithi031.stdout:1/234: link d2/d4/f31 d2/d4/d1b/f50 0 2024-09-16T08:57:15.139 INFO:tasks.workunit.client.0.smithi031.stdout:5/299: unlink d2/dc/f44 0 2024-09-16T08:57:15.143 INFO:tasks.workunit.client.1.smithi097.stdout:6/689: dwrite d1/dd3/d9e/dca/d16/d86/fb2 [0,4194304] 0 2024-09-16T08:57:15.144 INFO:tasks.workunit.client.0.smithi031.stdout:3/347: mkdir d3/d14/d37/d1a/d1d/d1e/d67 0 2024-09-16T08:57:15.145 INFO:tasks.workunit.client.0.smithi031.stdout:2/222: dwrite d6/d41/f2b [0,4194304] 0 2024-09-16T08:57:15.146 INFO:tasks.workunit.client.1.smithi097.stdout:3/695: link d7/de/d3c/f79 d7/de/d3c/dcc/ff3 0 2024-09-16T08:57:15.146 INFO:tasks.workunit.client.1.smithi097.stdout:3/696: write d7/d28/d63/d70/f84 [186673,6584] 0 2024-09-16T08:57:15.146 INFO:tasks.workunit.client.1.smithi097.stdout:3/697: chown d7/de/d3c/d3f/d60/dcb/d35/le2 1169000 1 2024-09-16T08:57:15.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/365: creat d0/dd/d35/d59/d56/f6e x:0 0 0 2024-09-16T08:57:15.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/366: fdatasync d0/d17/f55 0 2024-09-16T08:57:15.148 INFO:tasks.workunit.client.0.smithi031.stdout:6/367: write d0/dd/d35/d59/d3f/d65/f68 [655262,58658] 0 2024-09-16T08:57:15.149 INFO:tasks.workunit.client.1.smithi097.stdout:0/680: mknod da/d1a/d26/d94/cf7 0 2024-09-16T08:57:15.150 INFO:tasks.workunit.client.0.smithi031.stdout:5/300: creat d2/dc/f61 x:0 0 0 2024-09-16T08:57:15.150 INFO:tasks.workunit.client.0.smithi031.stdout:5/301: fdatasync d2/dc/f43 0 2024-09-16T08:57:15.151 INFO:tasks.workunit.client.1.smithi097.stdout:2/686: truncate d3/d4/d8/f2b 1205843 0 2024-09-16T08:57:15.153 INFO:tasks.workunit.client.0.smithi031.stdout:1/235: dread d2/d4/f1c [0,4194304] 0 2024-09-16T08:57:15.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/145: dwrite d0/f24 [0,4194304] 0 2024-09-16T08:57:15.155 INFO:tasks.workunit.client.0.smithi031.stdout:3/348: mkdir d3/d14/d37/d1a/d68 0 2024-09-16T08:57:15.156 INFO:tasks.workunit.client.1.smithi097.stdout:5/727: rename db/l11 to db/d18/d1c/d42/d46/d7e/d76/dca/lf2 0 2024-09-16T08:57:15.158 INFO:tasks.workunit.client.0.smithi031.stdout:4/146: dread d0/d2/d16/d1e/f23 [0,4194304] 0 2024-09-16T08:57:15.161 INFO:tasks.workunit.client.0.smithi031.stdout:2/223: mknod d6/d41/c46 0 2024-09-16T08:57:15.161 INFO:tasks.workunit.client.0.smithi031.stdout:4/147: write d0/d2/d16/f19 [518991,86699] 0 2024-09-16T08:57:15.161 INFO:tasks.workunit.client.0.smithi031.stdout:4/148: truncate d0/d2/fe 920641 0 2024-09-16T08:57:15.163 INFO:tasks.workunit.client.0.smithi031.stdout:5/302: creat d2/dc/d3d/d4c/f62 x:0 0 0 2024-09-16T08:57:15.163 INFO:tasks.workunit.client.0.smithi031.stdout:5/303: chown d2/dc/f3a 2508 1 2024-09-16T08:57:15.163 INFO:tasks.workunit.client.0.smithi031.stdout:5/304: stat d2/dc/d3d/l3e 0 2024-09-16T08:57:15.164 INFO:tasks.workunit.client.1.smithi097.stdout:7/594: symlink d4/d9/dc/d8e/lc8 0 2024-09-16T08:57:15.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/236: rename d2/de/c39 to d2/d15/d1f/c51 0 2024-09-16T08:57:15.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/237: write d2/d4/d1b/f2b [254877,47426] 0 2024-09-16T08:57:15.170 INFO:tasks.workunit.client.0.smithi031.stdout:3/349: dread d3/f2a [0,4194304] 0 2024-09-16T08:57:15.170 INFO:tasks.workunit.client.0.smithi031.stdout:3/350: stat d3/le 0 2024-09-16T08:57:15.174 INFO:tasks.workunit.client.0.smithi031.stdout:2/224: unlink d6/d20/f43 0 2024-09-16T08:57:15.174 INFO:tasks.workunit.client.0.smithi031.stdout:2/225: creat d6/d20/f47 x:0 0 0 2024-09-16T08:57:15.174 INFO:tasks.workunit.client.0.smithi031.stdout:2/226: write d6/d20/f26 [398135,117444] 0 2024-09-16T08:57:15.181 INFO:tasks.workunit.client.0.smithi031.stdout:4/149: creat d0/d2/d16/f30 x:0 0 0 2024-09-16T08:57:15.181 INFO:tasks.workunit.client.0.smithi031.stdout:4/150: dread - d0/d2/d16/f30 zero size 2024-09-16T08:57:15.182 INFO:tasks.workunit.client.0.smithi031.stdout:4/151: dread d0/d2/f10 [0,4194304] 0 2024-09-16T08:57:15.184 INFO:tasks.workunit.client.1.smithi097.stdout:6/690: mknod d1/dd3/d9e/db3/de3/ce5 0 2024-09-16T08:57:15.186 INFO:tasks.workunit.client.1.smithi097.stdout:6/691: read d1/f5b [343625,75598] 0 2024-09-16T08:57:15.186 INFO:tasks.workunit.client.1.smithi097.stdout:6/692: chown d1/dd3/d9e/dca/d16/l7f 20446279 1 2024-09-16T08:57:15.187 INFO:tasks.workunit.client.1.smithi097.stdout:6/693: chown d1/dd3/d9e/dca/d81/dad/d5a/l51 83876 1 2024-09-16T08:57:15.188 INFO:tasks.workunit.client.0.smithi031.stdout:2/227: symlink d6/d20/l48 0 2024-09-16T08:57:15.188 INFO:tasks.workunit.client.0.smithi031.stdout:2/228: truncate d6/d20/f32 471003 0 2024-09-16T08:57:15.191 INFO:tasks.workunit.client.1.smithi097.stdout:0/681: truncate f6 3339019 0 2024-09-16T08:57:15.191 INFO:tasks.workunit.client.1.smithi097.stdout:0/682: truncate da/d1a/d26/d47/db3/fd4 215856 0 2024-09-16T08:57:15.191 INFO:tasks.workunit.client.1.smithi097.stdout:0/683: stat da/d24/d4c/d80/d3b/f5a 0 2024-09-16T08:57:15.192 INFO:tasks.workunit.client.0.smithi031.stdout:1/238: mkdir d2/d4/d52 0 2024-09-16T08:57:15.194 INFO:tasks.workunit.client.1.smithi097.stdout:2/687: mkdir d3/d4/d9/d78/dcc 0 2024-09-16T08:57:15.194 INFO:tasks.workunit.client.0.smithi031.stdout:6/368: dwrite d0/d17/f1e [4194304,4194304] 0 2024-09-16T08:57:15.195 INFO:tasks.workunit.client.0.smithi031.stdout:4/152: symlink d0/d2/d16/d1e/l31 0 2024-09-16T08:57:15.198 INFO:tasks.workunit.client.0.smithi031.stdout:1/239: truncate d2/d4/f30 2355183 0 2024-09-16T08:57:15.198 INFO:tasks.workunit.client.0.smithi031.stdout:1/240: stat d2/d4/l3a 0 2024-09-16T08:57:15.199 INFO:tasks.workunit.client.0.smithi031.stdout:1/241: read d2/d4/d1b/f50 [81970,109412] 0 2024-09-16T08:57:15.199 INFO:tasks.workunit.client.1.smithi097.stdout:7/595: symlink d4/d12/d1f/d22/lc9 0 2024-09-16T08:57:15.201 INFO:tasks.workunit.client.1.smithi097.stdout:6/694: dread d1/db7/fce [0,4194304] 0 2024-09-16T08:57:15.201 INFO:tasks.workunit.client.1.smithi097.stdout:6/695: stat d1/dd3/d9e/db3 0 2024-09-16T08:57:15.201 INFO:tasks.workunit.client.1.smithi097.stdout:6/696: chown d1/dd3/d9e/dca/d81/dad/d5a/l69 0 1 2024-09-16T08:57:15.205 INFO:tasks.workunit.client.1.smithi097.stdout:0/684: creat da/d1a/d26/d94/dc9/dcb/ff8 x:0 0 0 2024-09-16T08:57:15.206 INFO:tasks.workunit.client.0.smithi031.stdout:6/369: mknod d0/dd/d2b/c6f 0 2024-09-16T08:57:15.207 INFO:tasks.workunit.client.1.smithi097.stdout:7/596: mknod d4/d12/d1f/d22/cca 0 2024-09-16T08:57:15.207 INFO:tasks.workunit.client.1.smithi097.stdout:7/597: creat d4/d9/dc/fcb x:0 0 0 2024-09-16T08:57:15.208 INFO:tasks.workunit.client.0.smithi031.stdout:4/153: creat d0/d2/d16/d1e/f32 x:0 0 0 2024-09-16T08:57:15.208 INFO:tasks.workunit.client.1.smithi097.stdout:2/688: write d3/f9b [2868883,31848] 0 2024-09-16T08:57:15.209 INFO:tasks.workunit.client.1.smithi097.stdout:6/697: creat d1/dd8/fe6 x:0 0 0 2024-09-16T08:57:15.210 INFO:tasks.workunit.client.0.smithi031.stdout:6/370: rename d0/dd/d35/d59/d56/f6e to d0/dd/d35/d59/d3f/f70 0 2024-09-16T08:57:15.211 INFO:tasks.workunit.client.1.smithi097.stdout:7/598: symlink d4/d7/d7b/lcc 0 2024-09-16T08:57:15.211 INFO:tasks.workunit.client.1.smithi097.stdout:7/599: readlink d4/d12/l2a 0 2024-09-16T08:57:15.212 INFO:tasks.workunit.client.0.smithi031.stdout:4/154: creat d0/d2/f33 x:0 0 0 2024-09-16T08:57:15.213 INFO:tasks.workunit.client.0.smithi031.stdout:4/155: chown d0/d2/l29 532813 1 2024-09-16T08:57:15.215 INFO:tasks.workunit.client.1.smithi097.stdout:2/689: dread d3/d4/d68/d8b/d7c/d7d/f8d [0,4194304] 0 2024-09-16T08:57:15.216 INFO:tasks.workunit.client.0.smithi031.stdout:6/371: rename d0/dd/d2b/c6f to d0/dd/d35/d59/d56/c71 0 2024-09-16T08:57:15.216 INFO:tasks.workunit.client.0.smithi031.stdout:5/305: dwrite d2/dc/d25/d21/d2e/f48 [0,4194304] 0 2024-09-16T08:57:15.217 INFO:tasks.workunit.client.0.smithi031.stdout:5/306: creat d2/f63 x:0 0 0 2024-09-16T08:57:15.217 INFO:tasks.workunit.client.0.smithi031.stdout:5/307: chown d2/dc/d36 2100262593 1 2024-09-16T08:57:15.217 INFO:tasks.workunit.client.1.smithi097.stdout:5/728: dwrite db/d18/d1c/d42/d46/d7e/d76/fc0 [0,4194304] 0 2024-09-16T08:57:15.218 INFO:tasks.workunit.client.1.smithi097.stdout:5/729: dread - db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/fb7 zero size 2024-09-16T08:57:15.218 INFO:tasks.workunit.client.1.smithi097.stdout:5/730: chown db/f40 24 1 2024-09-16T08:57:15.218 INFO:tasks.workunit.client.1.smithi097.stdout:0/685: dread da/d1a/d26/d47/db3/d60/d68/d5e/f72 [0,4194304] 0 2024-09-16T08:57:15.218 INFO:tasks.workunit.client.1.smithi097.stdout:0/686: fsync da/f1b 0 2024-09-16T08:57:15.219 INFO:tasks.workunit.client.1.smithi097.stdout:3/698: dwrite d7/d15/f17 [0,4194304] 0 2024-09-16T08:57:15.219 INFO:tasks.workunit.client.1.smithi097.stdout:7/600: symlink d4/d7/d77/lcd 0 2024-09-16T08:57:15.221 INFO:tasks.workunit.client.1.smithi097.stdout:7/601: dread d4/d1b/f69 [0,4194304] 0 2024-09-16T08:57:15.223 INFO:tasks.workunit.client.1.smithi097.stdout:2/690: rename d3/d4/d68/d8b/d7c/d7d/d53/d47/f4b to d3/d4/d68/d8b/d7c/d7d/fcd 0 2024-09-16T08:57:15.224 INFO:tasks.workunit.client.0.smithi031.stdout:3/351: dwrite d3/d14/d37/d1a/d1d/d3b/f43 [0,4194304] 0 2024-09-16T08:57:15.225 INFO:tasks.workunit.client.0.smithi031.stdout:6/372: unlink d0/d17/d19/l66 0 2024-09-16T08:57:15.234 INFO:tasks.workunit.client.1.smithi097.stdout:3/699: read d7/d28/d30/d69/f6a [4046560,89525] 0 2024-09-16T08:57:15.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/352: dread d3/f2a [0,4194304] 0 2024-09-16T08:57:15.235 INFO:tasks.workunit.client.0.smithi031.stdout:6/373: write d0/dd/f3e [1061195,4217] 0 2024-09-16T08:57:15.240 INFO:tasks.workunit.client.0.smithi031.stdout:0/250: dwrite d6/d12/f16 [0,4194304] 0 2024-09-16T08:57:15.241 INFO:tasks.workunit.client.1.smithi097.stdout:6/698: dwrite d1/dd3/d9e/dca/d16/d4f/f55 [0,4194304] 0 2024-09-16T08:57:15.244 INFO:tasks.workunit.client.1.smithi097.stdout:5/731: rename db/d18/d1c/d42/fa2 to db/d1a/d6d/dce/ff3 0 2024-09-16T08:57:15.245 INFO:tasks.workunit.client.1.smithi097.stdout:5/732: dread db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 [0,4194304] 0 2024-09-16T08:57:15.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/156: dwrite d0/d2/f33 [0,4194304] 0 2024-09-16T08:57:15.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/157: creat d0/d2/d16/d1e/f34 x:0 0 0 2024-09-16T08:57:15.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/158: creat d0/d2/d16/f35 x:0 0 0 2024-09-16T08:57:15.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/159: write d0/d2/d16/f30 [61686,123870] 0 2024-09-16T08:57:15.247 INFO:tasks.workunit.client.0.smithi031.stdout:5/308: symlink d2/dc/d3d/d4c/l64 0 2024-09-16T08:57:15.247 INFO:tasks.workunit.client.0.smithi031.stdout:5/309: write d2/dc/d25/d21/f26 [667114,25943] 0 2024-09-16T08:57:15.247 INFO:tasks.workunit.client.0.smithi031.stdout:5/310: chown d2/dc/fd 58 1 2024-09-16T08:57:15.252 INFO:tasks.workunit.client.1.smithi097.stdout:5/733: mknod db/d6e/d81/cf4 0 2024-09-16T08:57:15.255 INFO:tasks.workunit.client.0.smithi031.stdout:0/251: dread d6/d12/d13/f30 [0,4194304] 0 2024-09-16T08:57:15.256 INFO:tasks.workunit.client.0.smithi031.stdout:3/353: rmdir d3/d14/d37/d1a/d68 0 2024-09-16T08:57:15.258 INFO:tasks.workunit.client.0.smithi031.stdout:6/374: creat d0/f72 x:0 0 0 2024-09-16T08:57:15.258 INFO:tasks.workunit.client.0.smithi031.stdout:6/375: chown d0/d17/d19/f2e 1954 1 2024-09-16T08:57:15.263 INFO:tasks.workunit.client.0.smithi031.stdout:0/252: link d6/d12/d47/f4e d6/f57 0 2024-09-16T08:57:15.266 INFO:tasks.workunit.client.1.smithi097.stdout:7/602: dwrite d4/d7/d19/fa0 [0,4194304] 0 2024-09-16T08:57:15.267 INFO:tasks.workunit.client.1.smithi097.stdout:0/687: dwrite da/d1a/d26/d47/db3/d60/d68/fba [0,4194304] 0 2024-09-16T08:57:15.268 INFO:tasks.workunit.client.1.smithi097.stdout:5/734: dread db/d1a/f2c [0,4194304] 0 2024-09-16T08:57:15.272 INFO:tasks.workunit.client.1.smithi097.stdout:6/699: dwrite d1/dc/d3c/f6d [4194304,4194304] 0 2024-09-16T08:57:15.272 INFO:tasks.workunit.client.1.smithi097.stdout:3/700: dwrite d7/de/d3c/d3f/f5e [0,4194304] 0 2024-09-16T08:57:15.274 INFO:tasks.workunit.client.1.smithi097.stdout:7/603: mknod d4/d7/d7b/d43/daf/cce 0 2024-09-16T08:57:15.277 INFO:tasks.workunit.client.1.smithi097.stdout:6/700: symlink d1/dd3/d9e/dca/le7 0 2024-09-16T08:57:15.285 INFO:tasks.workunit.client.1.smithi097.stdout:3/701: rename d7/de/d3c/dbe/fc8 to d7/d28/d30/d88/df1/ff4 0 2024-09-16T08:57:15.285 INFO:tasks.workunit.client.1.smithi097.stdout:3/702: fdatasync d7/de/d6c/fe7 0 2024-09-16T08:57:15.285 INFO:tasks.workunit.client.1.smithi097.stdout:3/703: getdents d7/de/d8d/d90/d92/da8 0 2024-09-16T08:57:15.285 INFO:tasks.workunit.client.1.smithi097.stdout:3/704: stat d7/c98 0 2024-09-16T08:57:15.285 INFO:tasks.workunit.client.1.smithi097.stdout:3/705: write d7/de/d3c/d4f/f80 [296029,49763] 0 2024-09-16T08:57:15.286 INFO:tasks.workunit.client.1.smithi097.stdout:2/691: unlink d3/d4/d9/f4f 0 2024-09-16T08:57:15.286 INFO:tasks.workunit.client.1.smithi097.stdout:2/692: fdatasync d3/d4/d68/d8b/fa0 0 2024-09-16T08:57:15.288 INFO:tasks.workunit.client.1.smithi097.stdout:6/701: truncate d1/dd3/d9e/dca/d16/f89 4139425 0 2024-09-16T08:57:15.290 INFO:tasks.workunit.client.1.smithi097.stdout:2/693: truncate d3/d4/d68/d8b/d7c/d7d/f8d 889809 0 2024-09-16T08:57:15.290 INFO:tasks.workunit.client.1.smithi097.stdout:7/604: write d4/d9/d1c/f2c [1124042,95503] 0 2024-09-16T08:57:15.290 INFO:tasks.workunit.client.1.smithi097.stdout:2/694: write d3/d4/fc8 [74051,33948] 0 2024-09-16T08:57:15.290 INFO:tasks.workunit.client.1.smithi097.stdout:7/605: write d4/d9/dc/f72 [3751861,79201] 0 2024-09-16T08:57:15.291 INFO:tasks.workunit.client.1.smithi097.stdout:2/695: write d3/f32 [4085405,105804] 0 2024-09-16T08:57:15.292 INFO:tasks.workunit.client.1.smithi097.stdout:6/702: truncate d1/dc/d3c/f6f 608458 0 2024-09-16T08:57:15.297 INFO:tasks.workunit.client.1.smithi097.stdout:7/606: mknod d4/d9/d1c/d5f/ccf 0 2024-09-16T08:57:15.297 INFO:tasks.workunit.client.1.smithi097.stdout:7/607: write d4/d9/dc/f90 [1028839,44584] 0 2024-09-16T08:57:15.297 INFO:tasks.workunit.client.1.smithi097.stdout:7/608: fsync d4/d9/dc/f6b 0 2024-09-16T08:57:15.297 INFO:tasks.workunit.client.1.smithi097.stdout:7/609: fsync d4/d12/d1f/f21 0 2024-09-16T08:57:15.297 INFO:tasks.workunit.client.1.smithi097.stdout:7/610: read d4/d7/d7b/f46 [673354,81552] 0 2024-09-16T08:57:15.298 INFO:tasks.workunit.client.1.smithi097.stdout:2/696: symlink d3/d4/d9/d67/db2/lce 0 2024-09-16T08:57:15.298 INFO:tasks.workunit.client.1.smithi097.stdout:2/697: read d3/d4/d68/d8b/d7c/d7d/f8c [110389,97007] 0 2024-09-16T08:57:15.298 INFO:tasks.workunit.client.1.smithi097.stdout:2/698: fdatasync d3/d4/d68/d8b/d7c/d7d/d53/f58 0 2024-09-16T08:57:15.299 INFO:tasks.workunit.client.1.smithi097.stdout:7/611: symlink d4/d1b/d41/ld0 0 2024-09-16T08:57:15.301 INFO:tasks.workunit.client.1.smithi097.stdout:2/699: symlink d3/d1b/da7/lcf 0 2024-09-16T08:57:15.304 INFO:tasks.workunit.client.1.smithi097.stdout:2/700: rename d3/d4/f7e to d3/d4/d9/d67/fd0 0 2024-09-16T08:57:15.307 INFO:tasks.workunit.client.1.smithi097.stdout:2/701: rename d3/d4/d68/d8b/d7c/d7d/d53/da3/fa9 to d3/d4/d8/d51/fd1 0 2024-09-16T08:57:15.308 INFO:tasks.workunit.client.1.smithi097.stdout:2/702: mknod d3/d4/d68/d8b/d7c/d7d/d53/cd2 0 2024-09-16T08:57:15.310 INFO:tasks.workunit.client.1.smithi097.stdout:2/703: mknod d3/d4/d9/cd3 0 2024-09-16T08:57:15.313 INFO:tasks.workunit.client.0.smithi031.stdout:3/354: dwrite d3/d14/d37/d1a/d1d/f2d [0,4194304] 0 2024-09-16T08:57:15.313 INFO:tasks.workunit.client.0.smithi031.stdout:3/355: chown d3/d14/d37/c56 6 1 2024-09-16T08:57:15.314 INFO:tasks.workunit.client.0.smithi031.stdout:3/356: fsync d3/d14/d37/d47/f5b 0 2024-09-16T08:57:15.315 INFO:tasks.workunit.client.0.smithi031.stdout:3/357: unlink d3/c44 0 2024-09-16T08:57:15.317 INFO:tasks.workunit.client.0.smithi031.stdout:3/358: link d3/d14/d37/d33/f3c d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 0 2024-09-16T08:57:15.317 INFO:tasks.workunit.client.0.smithi031.stdout:3/359: fdatasync d3/d14/d37/d1a/d1d/f34 0 2024-09-16T08:57:15.318 INFO:tasks.workunit.client.0.smithi031.stdout:3/360: write d3/d14/d37/d33/f39 [61183,74068] 0 2024-09-16T08:57:15.331 INFO:tasks.workunit.client.0.smithi031.stdout:0/253: dwrite d6/f3b [0,4194304] 0 2024-09-16T08:57:15.331 INFO:tasks.workunit.client.0.smithi031.stdout:6/376: dwrite d0/d10/f58 [0,4194304] 0 2024-09-16T08:57:15.331 INFO:tasks.workunit.client.0.smithi031.stdout:5/311: dwrite d2/dc/d25/d21/f26 [0,4194304] 0 2024-09-16T08:57:15.332 INFO:tasks.workunit.client.1.smithi097.stdout:5/735: dwrite db/d18/d1c/d42/d46/d7e/d76/fc0 [0,4194304] 0 2024-09-16T08:57:15.332 INFO:tasks.workunit.client.1.smithi097.stdout:5/736: getdents db/d18/d1c/d42/d46/d7e/d64/d8d 0 2024-09-16T08:57:15.333 INFO:tasks.workunit.client.1.smithi097.stdout:2/704: dwrite d3/d4/d8/d51/f59 [0,4194304] 0 2024-09-16T08:57:15.333 INFO:tasks.workunit.client.1.smithi097.stdout:2/705: fdatasync d3/d4/d9/d78/f94 0 2024-09-16T08:57:15.333 INFO:tasks.workunit.client.1.smithi097.stdout:2/706: chown d3/d4/d68/d8b/d61 436329 1 2024-09-16T08:57:15.333 INFO:tasks.workunit.client.1.smithi097.stdout:3/706: dwrite d7/d28/d30/d69/fdb [0,4194304] 0 2024-09-16T08:57:15.333 INFO:tasks.workunit.client.1.smithi097.stdout:6/703: dwrite d1/dd3/d9e/dca/d16/d4f/d9f/fdb [0,4194304] 0 2024-09-16T08:57:15.334 INFO:tasks.workunit.client.1.smithi097.stdout:7/612: dwrite d4/d1b/d41/f95 [0,4194304] 0 2024-09-16T08:57:15.349 INFO:tasks.workunit.client.0.smithi031.stdout:3/361: dwrite f1 [4194304,4194304] 0 2024-09-16T08:57:15.349 INFO:tasks.workunit.client.0.smithi031.stdout:3/362: dread - d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 zero size 2024-09-16T08:57:15.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.347+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f33a4000bf0 con 0x7f33b0073d80 2024-09-16T08:57:15.350 INFO:tasks.workunit.client.0.smithi031.stdout:5/312: dwrite d2/f1b [0,4194304] 0 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.357+0000 7f33c1ffb640 1 -- 172.21.15.31:0/2522629635 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f33a4000bf0 con 0x7f33b0073d80 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (6m) 2m ago 8m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (8m) 2m ago 8m 7528k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 66d87845da99 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (7m) 2s ago 7m 7671k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 edd14d48bff9 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (8m) 2m ago 8m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (7m) 2s ago 7m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:57:15.359 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (6m) 2m ago 8m 83.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (2m) 2m ago 2m 11.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (2m) 2m ago 2m 18.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (2m) 2s ago 2m 22.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (2m) 2s ago 2m 15.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:9283,8765,8443 running (10m) 2m ago 10m 486M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 1c78052ec7c9 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (32s) 2s ago 7m 496M - 19.3.0-4917-g8293d73f 425e316ba8eb 6aa6df83241b 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (10m) 2m ago 10m 47.7M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (7m) 2s ago 7m 40.5M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (8m) 2m ago 8m 15.3M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (7m) 2s ago 7m 16.3M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 2m ago 6m 47.3M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 2m ago 5m 54.9M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:57:15.360 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 2m ago 5m 46.6M 3948M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:57:15.361 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (4m) 2s ago 4m 183M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:57:15.361 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (4m) 2s ago 4m 206M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:57:15.361 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (4m) 2s ago 4m 205M 4970M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:57:15.361 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 2m ago 8m 60.0M - 2.43.0 a07b618ecd1d b0b74488871e 2024-09-16T08:57:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f33b0073d80 msgr2=0x7f33b0076240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f33b0073d80 0x7f33b0076240 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f33c40ffb00 tx=0x7f33bc008040 comp rx=0 tx=0).stop 2024-09-16T08:57:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40ffca0 msgr2=0x7f33c4197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40ffca0 0x7f33c4197e80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33b402fa50 tx=0x7f33b4030680 comp rx=0 tx=0).stop 2024-09-16T08:57:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 shutdown_connections 2024-09-16T08:57:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f33b0073d80 0x7f33b0076240 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33c40ffca0 0x7f33c4197e80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 --2- 172.21.15.31:0/2522629635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33c40feaa0 0x7f33c4197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 >> 172.21.15.31:0/2522629635 conn(0x7f33c40fa230 msgr2=0x7f33c41033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 shutdown_connections 2024-09-16T08:57:15.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.360+0000 7f33caffd640 1 -- 172.21.15.31:0/2522629635 wait complete. 2024-09-16T08:57:15.389 INFO:tasks.workunit.client.1.smithi097.stdout:5/737: creat db/d18/d1c/d42/d46/d7e/d8f/ff5 x:0 0 0 2024-09-16T08:57:15.391 INFO:tasks.workunit.client.1.smithi097.stdout:5/738: dread db/d18/d1c/d42/d46/d7e/d8f/fc2 [0,4194304] 0 2024-09-16T08:57:15.391 INFO:tasks.workunit.client.1.smithi097.stdout:5/739: fsync db/d18/d1c/d42/d46/d7e/f53 0 2024-09-16T08:57:15.399 INFO:tasks.workunit.client.0.smithi031.stdout:4/160: truncate d0/d2/d16/f19 877453 0 2024-09-16T08:57:15.399 INFO:tasks.workunit.client.0.smithi031.stdout:4/161: dread - d0/d2/f1a zero size 2024-09-16T08:57:15.399 INFO:tasks.workunit.client.1.smithi097.stdout:4/635: sync 2024-09-16T08:57:15.399 INFO:tasks.workunit.client.1.smithi097.stdout:9/623: sync 2024-09-16T08:57:15.400 INFO:tasks.workunit.client.1.smithi097.stdout:8/657: sync 2024-09-16T08:57:15.400 INFO:tasks.workunit.client.1.smithi097.stdout:3/707: stat d7/de/d3c/d3f/d60/dcb/d35/d3d/c71 0 2024-09-16T08:57:15.401 INFO:tasks.workunit.client.1.smithi097.stdout:4/636: dread d0/d7/df/d83/fa5 [0,4194304] 0 2024-09-16T08:57:15.403 INFO:tasks.workunit.client.1.smithi097.stdout:8/658: dread d8/dd/d15/f8f [0,4194304] 0 2024-09-16T08:57:15.407 INFO:tasks.workunit.client.1.smithi097.stdout:6/704: creat d1/dd3/d9e/dca/fe8 x:0 0 0 2024-09-16T08:57:15.407 INFO:tasks.workunit.client.1.smithi097.stdout:6/705: write d1/dd3/fc2 [342438,33499] 0 2024-09-16T08:57:15.407 INFO:tasks.workunit.client.1.smithi097.stdout:3/708: write d7/de/d3c/f45 [2488031,127787] 0 2024-09-16T08:57:15.408 INFO:tasks.workunit.client.1.smithi097.stdout:7/613: mknod d4/d7/d7b/cd1 0 2024-09-16T08:57:15.408 INFO:tasks.workunit.client.1.smithi097.stdout:7/614: fsync d4/d7/d7b/d43/f44 0 2024-09-16T08:57:15.412 INFO:tasks.workunit.client.1.smithi097.stdout:5/740: rename db/f2e to db/d18/d1c/d42/d46/d7e/d64/ff6 0 2024-09-16T08:57:15.412 INFO:tasks.workunit.client.1.smithi097.stdout:5/741: readlink db/d18/d1c/l94 0 2024-09-16T08:57:15.416 INFO:tasks.workunit.client.1.smithi097.stdout:4/637: rmdir d0/d35/da4 39 2024-09-16T08:57:15.416 INFO:tasks.workunit.client.1.smithi097.stdout:4/638: fdatasync d0/d7/df/d83/fa5 0 2024-09-16T08:57:15.419 INFO:tasks.workunit.client.1.smithi097.stdout:6/706: truncate d1/dd3/d9e/dca/d81/dad/d5a/f83 429484 0 2024-09-16T08:57:15.419 INFO:tasks.workunit.client.1.smithi097.stdout:6/707: chown d1/dd3/d9e/dca/l14 1550 1 2024-09-16T08:57:15.421 INFO:tasks.workunit.client.1.smithi097.stdout:8/659: dwrite d8/dd/d15/d16/d7c/da2/fbb [0,4194304] 0 2024-09-16T08:57:15.421 INFO:tasks.workunit.client.1.smithi097.stdout:8/660: dread - d8/dd/d15/d2c/d4a/d4d/d3f/f65 zero size 2024-09-16T08:57:15.421 INFO:tasks.workunit.client.1.smithi097.stdout:8/661: chown d8/dd/d15/d2c/f4b 191768211 1 2024-09-16T08:57:15.427 INFO:tasks.workunit.client.0.smithi031.stdout:3/363: mkdir d3/d14/d37/d1a/d1d/d6a 0 2024-09-16T08:57:15.429 INFO:tasks.workunit.client.1.smithi097.stdout:4/639: mknod d0/d36/cde 0 2024-09-16T08:57:15.429 INFO:tasks.workunit.client.1.smithi097.stdout:4/640: dread - d0/d23/d5f/d7d/f85 zero size 2024-09-16T08:57:15.429 INFO:tasks.workunit.client.1.smithi097.stdout:4/641: readlink d0/d23/l8b 0 2024-09-16T08:57:15.430 INFO:tasks.workunit.client.1.smithi097.stdout:4/642: truncate d0/d7/df/f8f 1189634 0 2024-09-16T08:57:15.432 INFO:tasks.workunit.client.1.smithi097.stdout:6/708: truncate d1/dd3/d9e/dca/d81/dad/f62 2981752 0 2024-09-16T08:57:15.432 INFO:tasks.workunit.client.1.smithi097.stdout:1/754: sync 2024-09-16T08:57:15.433 INFO:tasks.workunit.client.1.smithi097.stdout:1/755: dread - d5/db/dc/fda zero size 2024-09-16T08:57:15.433 INFO:tasks.workunit.client.1.smithi097.stdout:1/756: fdatasync d5/d2c/f4b 0 2024-09-16T08:57:15.434 INFO:tasks.workunit.client.0.smithi031.stdout:5/313: rename d2/dc/d25/d21/d2e/f48 to d2/dc/d25/d21/f65 0 2024-09-16T08:57:15.435 INFO:tasks.workunit.client.0.smithi031.stdout:3/364: dread d3/d14/d37/d33/f3c [0,4194304] 0 2024-09-16T08:57:15.435 INFO:tasks.workunit.client.0.smithi031.stdout:3/365: write d3/d14/d37/d1a/d1d/d3b/d3f/f52 [609227,54207] 0 2024-09-16T08:57:15.437 INFO:tasks.workunit.client.1.smithi097.stdout:6/709: read d1/dd3/d9e/dca/d81/dad/d5a/f47 [1993618,129220] 0 2024-09-16T08:57:15.440 INFO:tasks.workunit.client.1.smithi097.stdout:3/709: getdents d7/de/d3c/d3f/d60/dcb 0 2024-09-16T08:57:15.442 INFO:tasks.workunit.client.0.smithi031.stdout:5/314: link d2/dc/c47 d2/dc/d3d/d4c/c66 0 2024-09-16T08:57:15.444 INFO:tasks.workunit.client.0.smithi031.stdout:3/366: stat d3/d14/d37/d1a/d1d/d3b/f4f 0 2024-09-16T08:57:15.445 INFO:tasks.workunit.client.1.smithi097.stdout:2/707: getdents d3/d4/d8/d51 0 2024-09-16T08:57:15.448 INFO:tasks.workunit.client.0.smithi031.stdout:5/315: unlink d2/dc/f12 0 2024-09-16T08:57:15.448 INFO:tasks.workunit.client.0.smithi031.stdout:5/316: fdatasync d2/dc/d1d/f3f 0 2024-09-16T08:57:15.448 INFO:tasks.workunit.client.0.smithi031.stdout:5/317: chown d2/dc/d3d/c55 53145513 1 2024-09-16T08:57:15.448 INFO:tasks.workunit.client.0.smithi031.stdout:5/318: readlink d2/dc/d25/d21/d2e/l54 0 2024-09-16T08:57:15.450 INFO:tasks.workunit.client.0.smithi031.stdout:3/367: creat d3/d14/d37/d1a/d1d/d1e/d67/f6b x:0 0 0 2024-09-16T08:57:15.454 INFO:tasks.workunit.client.1.smithi097.stdout:4/643: symlink d0/d36/da7/ldf 0 2024-09-16T08:57:15.455 INFO:tasks.workunit.client.0.smithi031.stdout:5/319: write d2/dc/f3b [599339,58222] 0 2024-09-16T08:57:15.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/710: mknod d1/dd3/db0/ce9 0 2024-09-16T08:57:15.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/711: creat d1/dd3/d9e/dca/fea x:0 0 0 2024-09-16T08:57:15.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/712: write d1/dd3/d9e/dca/f11 [3087708,101981] 0 2024-09-16T08:57:15.458 INFO:tasks.workunit.client.1.smithi097.stdout:8/662: symlink d8/dd/lce 0 2024-09-16T08:57:15.458 INFO:tasks.workunit.client.1.smithi097.stdout:8/663: write d8/dd/f26 [832996,74768] 0 2024-09-16T08:57:15.458 INFO:tasks.workunit.client.1.smithi097.stdout:8/664: fsync d8/f10 0 2024-09-16T08:57:15.459 INFO:tasks.workunit.client.0.smithi031.stdout:5/320: symlink d2/dc/l67 0 2024-09-16T08:57:15.459 INFO:tasks.workunit.client.0.smithi031.stdout:5/321: truncate d2/dc/d36/f52 51872 0 2024-09-16T08:57:15.460 INFO:tasks.workunit.client.0.smithi031.stdout:5/322: write d2/dc/f43 [224665,79657] 0 2024-09-16T08:57:15.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/368: write d3/d14/d37/d1a/d1d/d3b/f43 [4032813,68778] 0 2024-09-16T08:57:15.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/369: write d3/fb [562581,44532] 0 2024-09-16T08:57:15.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/370: chown d3/d14/d37/f10 3 1 2024-09-16T08:57:15.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/371: chown d3/d14/d37/d1a/d1d/d1e/l5f 301056 1 2024-09-16T08:57:15.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/372: readlink d3/d14/d37/d33/l29 0 2024-09-16T08:57:15.462 INFO:tasks.workunit.client.1.smithi097.stdout:2/708: mkdir d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4 0 2024-09-16T08:57:15.462 INFO:tasks.workunit.client.1.smithi097.stdout:2/709: fsync d3/d4/d68/d8b/d7c/d7d/fb5 0 2024-09-16T08:57:15.462 INFO:tasks.workunit.client.1.smithi097.stdout:9/624: rename d2/d30 to d2/d5/db0/dc8 0 2024-09-16T08:57:15.463 INFO:tasks.workunit.client.1.smithi097.stdout:4/644: creat d0/d7/fe0 x:0 0 0 2024-09-16T08:57:15.465 INFO:tasks.workunit.client.0.smithi031.stdout:3/373: symlink d3/d14/d37/d1a/d1d/d3b/d3f/l6c 0 2024-09-16T08:57:15.465 INFO:tasks.workunit.client.0.smithi031.stdout:3/374: creat d3/d14/d37/d33/f6d x:0 0 0 2024-09-16T08:57:15.466 INFO:tasks.workunit.client.1.smithi097.stdout:3/710: dwrite d7/de/d3c/d3f/d60/fe1 [0,4194304] 0 2024-09-16T08:57:15.469 INFO:tasks.workunit.client.1.smithi097.stdout:6/713: symlink d1/dd3/d9e/leb 0 2024-09-16T08:57:15.470 INFO:tasks.workunit.client.1.smithi097.stdout:8/665: dread d8/dd/f7e [0,4194304] 0 2024-09-16T08:57:15.471 INFO:tasks.workunit.client.0.smithi031.stdout:3/375: rename d3/d14/d37/d1a/d24/l45 to d3/d14/d37/d1a/d1d/d6a/l6e 0 2024-09-16T08:57:15.473 INFO:tasks.workunit.client.1.smithi097.stdout:2/710: unlink d3/cc4 0 2024-09-16T08:57:15.474 INFO:tasks.workunit.client.0.smithi031.stdout:4/162: rmdir d0/d2 39 2024-09-16T08:57:15.474 INFO:tasks.workunit.client.0.smithi031.stdout:4/163: getdents d0/d2/d16/d2b 0 2024-09-16T08:57:15.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/323: dwrite d2/f63 [0,4194304] 0 2024-09-16T08:57:15.476 INFO:tasks.workunit.client.0.smithi031.stdout:3/376: fdatasync d3/d14/d37/d1a/d1d/d3b/f43 0 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 -- 172.21.15.31:0/2618317818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 msgr2=0x7f2734100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 --2- 172.21.15.31:0/2618317818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 0x7f2734100170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2720009880 tx=0x7f272002a4b0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 -- 172.21.15.31:0/2618317818 shutdown_connections 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 --2- 172.21.15.31:0/2618317818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 0x7f2734100170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 --2- 172.21.15.31:0/2618317818 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 0x7f27340feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 -- 172.21.15.31:0/2618317818 >> 172.21.15.31:0/2618317818 conn(0x7f27340fa280 msgr2=0x7f27340fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 -- 172.21.15.31:0/2618317818 shutdown_connections 2024-09-16T08:57:15.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.476+0000 7f273aed2640 1 -- 172.21.15.31:0/2618317818 wait complete. 2024-09-16T08:57:15.479 INFO:tasks.workunit.client.1.smithi097.stdout:9/625: creat d2/d5/d72/da2/fc9 x:0 0 0 2024-09-16T08:57:15.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.477+0000 7f273aed2640 1 Processor -- start 2024-09-16T08:57:15.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.477+0000 7f273aed2640 1 -- start start 2024-09-16T08:57:15.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.477+0000 7f273aed2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 0x7f27340687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.477+0000 7f273aed2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 0x7f2734198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.477+0000 7f273aed2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2734198820 con 0x7f27340feaf0 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.477+0000 7f273aed2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2734198990 con 0x7f27340ffcf0 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.478+0000 7f2738c6e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 0x7f27340687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.478+0000 7f2738c6e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 0x7f27340687d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34830/0 (socket says 172.21.15.31:34830) 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.478+0000 7f2738c6e640 1 -- 172.21.15.31:0/407268374 learned_addr learned my addr 172.21.15.31:0/407268374 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.478+0000 7f2733fff640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 0x7f2734198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.478+0000 7f2738c6e640 1 -- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 msgr2=0x7f2734198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.478+0000 7f2738c6e640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 0x7f2734198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.478+0000 7f2738c6e640 1 -- 172.21.15.31:0/407268374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720009560 con 0x7f27340feaf0 2024-09-16T08:57:15.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.479+0000 7f2738c6e640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 0x7f27340687d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f272800b3d0 tx=0x7f272800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:15.482 INFO:tasks.workunit.client.1.smithi097.stdout:4/645: creat d0/d35/da4/dd1/fe1 x:0 0 0 2024-09-16T08:57:15.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.479+0000 7f2731ffb640 1 -- 172.21.15.31:0/407268374 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2728011800 con 0x7f27340feaf0 2024-09-16T08:57:15.482 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.479+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2734198c70 con 0x7f27340feaf0 2024-09-16T08:57:15.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.479+0000 7f2731ffb640 1 -- 172.21.15.31:0/407268374 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2728011e20 con 0x7f27340feaf0 2024-09-16T08:57:15.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.479+0000 7f2731ffb640 1 -- 172.21.15.31:0/407268374 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f272800f510 con 0x7f27340feaf0 2024-09-16T08:57:15.483 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.479+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2734199210 con 0x7f27340feaf0 2024-09-16T08:57:15.483 INFO:tasks.workunit.client.0.smithi031.stdout:4/164: read d0/fb [3533990,73656] 0 2024-09-16T08:57:15.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.481+0000 7f2731ffb640 1 -- 172.21.15.31:0/407268374 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f272800f780 con 0x7f27340feaf0 2024-09-16T08:57:15.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.483+0000 7f2731ffb640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2724073dd0 0x7f2724076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:15.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.483+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27340feef0 con 0x7f27340feaf0 2024-09-16T08:57:15.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.483+0000 7f2731ffb640 1 -- 172.21.15.31:0/407268374 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2728095cf0 con 0x7f27340feaf0 2024-09-16T08:57:15.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.484+0000 7f2733fff640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2724073dd0 0x7f2724076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:15.486 INFO:tasks.workunit.client.0.smithi031.stdout:5/324: mkdir d2/dc/d25/d68 0 2024-09-16T08:57:15.486 INFO:tasks.workunit.client.0.smithi031.stdout:5/325: chown d2/dc/d25/d68 44 1 2024-09-16T08:57:15.486 INFO:tasks.workunit.client.0.smithi031.stdout:5/326: fdatasync d2/f1c 0 2024-09-16T08:57:15.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.484+0000 7f2733fff640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2724073dd0 0x7f2724076290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2720006010 tx=0x7f27200314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:15.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/377: mkdir d3/d14/d6f 0 2024-09-16T08:57:15.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/378: fdatasync d3/d14/d37/d1a/d1d/d1e/d67/f6b 0 2024-09-16T08:57:15.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/379: write d3/d14/d37/d1a/d1d/d1e/d67/f6b [993528,91830] 0 2024-09-16T08:57:15.490 INFO:tasks.workunit.client.0.smithi031.stdout:4/165: creat d0/d2/f36 x:0 0 0 2024-09-16T08:57:15.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.491+0000 7f2731ffb640 1 -- 172.21.15.31:0/407268374 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f2728059320 con 0x7f27340feaf0 2024-09-16T08:57:15.495 INFO:tasks.workunit.client.1.smithi097.stdout:3/711: unlink d7/de/d6c/dcf/ce8 0 2024-09-16T08:57:15.496 INFO:tasks.workunit.client.0.smithi031.stdout:3/380: rename d3/d14/c50 to d3/d14/d37/c70 0 2024-09-16T08:57:15.496 INFO:tasks.workunit.client.0.smithi031.stdout:3/381: truncate d3/d14/d37/d47/f5b 372493 0 2024-09-16T08:57:15.497 INFO:tasks.workunit.client.1.smithi097.stdout:6/714: symlink d1/lec 0 2024-09-16T08:57:15.498 INFO:tasks.workunit.client.0.smithi031.stdout:4/166: symlink d0/d2/d16/l37 0 2024-09-16T08:57:15.500 INFO:tasks.workunit.client.1.smithi097.stdout:2/711: symlink d3/d4/d68/d8b/d7c/d7d/d53/ld5 0 2024-09-16T08:57:15.500 INFO:tasks.workunit.client.1.smithi097.stdout:2/712: chown d3/d4/d68/d8b/d7c/d7d/f8c 80 1 2024-09-16T08:57:15.502 INFO:tasks.workunit.client.1.smithi097.stdout:4/646: dwrite d0/d39/d71/f77 [0,4194304] 0 2024-09-16T08:57:15.502 INFO:tasks.workunit.client.1.smithi097.stdout:4/647: write d0/d35/da4/dd1/fe1 [400034,4820] 0 2024-09-16T08:57:15.502 INFO:tasks.workunit.client.1.smithi097.stdout:4/648: truncate d0/d23/d5f/d7d/f85 1009459 0 2024-09-16T08:57:15.502 INFO:tasks.workunit.client.1.smithi097.stdout:4/649: readlink d0/d7/l4d 0 2024-09-16T08:57:15.503 INFO:tasks.workunit.client.1.smithi097.stdout:2/713: dread d3/d4/d77/f96 [0,4194304] 0 2024-09-16T08:57:15.503 INFO:tasks.workunit.client.1.smithi097.stdout:2/714: chown d3/d1b/c82 88695505 1 2024-09-16T08:57:15.503 INFO:tasks.workunit.client.1.smithi097.stdout:2/715: readlink d3/d4/d9/d67/l9f 0 2024-09-16T08:57:15.503 INFO:tasks.workunit.client.1.smithi097.stdout:7/615: rename d4/d7/c1a to d4/d7/d19/cd2 0 2024-09-16T08:57:15.503 INFO:tasks.workunit.client.1.smithi097.stdout:7/616: write d4/d7/d7b/f8f [4822518,4062] 0 2024-09-16T08:57:15.503 INFO:tasks.workunit.client.1.smithi097.stdout:7/617: read - d4/d9/d1c/d5f/fbc zero size 2024-09-16T08:57:15.504 INFO:tasks.workunit.client.1.smithi097.stdout:2/716: write d3/d4/d77/f70 [781893,76915] 0 2024-09-16T08:57:15.504 INFO:tasks.workunit.client.1.smithi097.stdout:9/626: mkdir d2/d5/db0/dc8/d63/da8/dca 0 2024-09-16T08:57:15.505 INFO:tasks.workunit.client.0.smithi031.stdout:4/167: unlink d0/d2/c11 0 2024-09-16T08:57:15.506 INFO:tasks.workunit.client.1.smithi097.stdout:3/712: symlink d7/de/d9c/daf/lf5 0 2024-09-16T08:57:15.508 INFO:tasks.workunit.client.1.smithi097.stdout:4/650: creat d0/d35/da4/dc5/fe2 x:0 0 0 2024-09-16T08:57:15.510 INFO:tasks.workunit.client.1.smithi097.stdout:4/651: read d0/d7/df/d83/fb5 [137342,14477] 0 2024-09-16T08:57:15.511 INFO:tasks.workunit.client.0.smithi031.stdout:4/168: dread d0/d2/fe [0,4194304] 0 2024-09-16T08:57:15.511 INFO:tasks.workunit.client.0.smithi031.stdout:4/169: write d0/d2/d16/d1e/f34 [505046,67289] 0 2024-09-16T08:57:15.511 INFO:tasks.workunit.client.0.smithi031.stdout:3/382: dwrite d3/d14/f2e [0,4194304] 0 2024-09-16T08:57:15.512 INFO:tasks.workunit.client.1.smithi097.stdout:1/757: rename d5/d17/d54/d6b/l75 to d5/d17/lf4 0 2024-09-16T08:57:15.512 INFO:tasks.workunit.client.1.smithi097.stdout:1/758: truncate d5/d17/f24 3482455 0 2024-09-16T08:57:15.512 INFO:tasks.workunit.client.1.smithi097.stdout:1/759: write d5/db/dc/fda [341634,128816] 0 2024-09-16T08:57:15.512 INFO:tasks.workunit.client.0.smithi031.stdout:5/327: rmdir d2/dc/d25/d21 39 2024-09-16T08:57:15.513 INFO:tasks.workunit.client.1.smithi097.stdout:2/717: dread d3/d4/d8/f16 [0,4194304] 0 2024-09-16T08:57:15.514 INFO:tasks.workunit.client.1.smithi097.stdout:7/618: symlink d4/d7/d7b/d43/ld3 0 2024-09-16T08:57:15.519 INFO:tasks.workunit.client.1.smithi097.stdout:9/627: unlink d2/d3f/f55 0 2024-09-16T08:57:15.521 INFO:tasks.workunit.client.0.smithi031.stdout:5/328: write d2/dc/d25/d21/f26 [2556080,39553] 0 2024-09-16T08:57:15.522 INFO:tasks.workunit.client.1.smithi097.stdout:3/713: mkdir d7/d28/d30/df6 0 2024-09-16T08:57:15.527 INFO:tasks.workunit.client.1.smithi097.stdout:4/652: truncate d0/d39/f3c 3017427 0 2024-09-16T08:57:15.530 INFO:tasks.workunit.client.1.smithi097.stdout:1/760: mkdir d5/d17/df5 0 2024-09-16T08:57:15.533 INFO:tasks.workunit.client.1.smithi097.stdout:2/718: creat d3/d4/d68/d8b/d61/dac/fd6 x:0 0 0 2024-09-16T08:57:15.537 INFO:tasks.workunit.client.1.smithi097.stdout:6/715: rmdir d1/dd3/d9e/dca 39 2024-09-16T08:57:15.537 INFO:tasks.workunit.client.1.smithi097.stdout:6/716: chown d1/dd3/l3f 173 1 2024-09-16T08:57:15.537 INFO:tasks.workunit.client.1.smithi097.stdout:6/717: readlink d1/dd3/d9e/dca/d16/le4 0 2024-09-16T08:57:15.542 INFO:tasks.workunit.client.1.smithi097.stdout:3/714: rename d7/d28/d30/df6 to d7/d28/d30/d88/df7 0 2024-09-16T08:57:15.544 INFO:tasks.workunit.client.0.smithi031.stdout:5/329: dwrite d2/dc/fd [4194304,4194304] 0 2024-09-16T08:57:15.545 INFO:tasks.workunit.client.1.smithi097.stdout:4/653: symlink d0/d36/d4c/le3 0 2024-09-16T08:57:15.545 INFO:tasks.workunit.client.0.smithi031.stdout:3/383: rmdir d3/d14/d37/d1a 39 2024-09-16T08:57:15.546 INFO:tasks.workunit.client.1.smithi097.stdout:2/719: dread d3/d1b/f97 [0,4194304] 0 2024-09-16T08:57:15.546 INFO:tasks.workunit.client.1.smithi097.stdout:9/628: dwrite f0 [0,4194304] 0 2024-09-16T08:57:15.546 INFO:tasks.workunit.client.1.smithi097.stdout:9/629: chown d2/d5/d56/f57 30 1 2024-09-16T08:57:15.546 INFO:tasks.workunit.client.1.smithi097.stdout:2/720: read - d3/d4/d68/d8b/d7c/d7d/f81 zero size 2024-09-16T08:57:15.546 INFO:tasks.workunit.client.1.smithi097.stdout:9/630: write d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/f91 [312923,117751] 0 2024-09-16T08:57:15.553 INFO:tasks.workunit.client.1.smithi097.stdout:6/718: link d1/dd3/d9e/dca/d16/d4f/d9f/fba d1/dd3/d9e/db3/fed 0 2024-09-16T08:57:15.555 INFO:tasks.workunit.client.0.smithi031.stdout:5/330: unlink d2/dc/d1d/l23 0 2024-09-16T08:57:15.557 INFO:tasks.workunit.client.1.smithi097.stdout:1/761: rename d5/d2c/f9b to d5/db/dc/ff6 0 2024-09-16T08:57:15.559 INFO:tasks.workunit.client.1.smithi097.stdout:3/715: dwrite d7/d28/d63/fa6 [0,4194304] 0 2024-09-16T08:57:15.561 INFO:tasks.workunit.client.1.smithi097.stdout:9/631: creat d2/d5/d3a/d70/da6/fcb x:0 0 0 2024-09-16T08:57:15.565 INFO:tasks.workunit.client.1.smithi097.stdout:6/719: symlink d1/dd3/d9e/dca/d81/dc4/ddc/lee 0 2024-09-16T08:57:15.567 INFO:tasks.workunit.client.1.smithi097.stdout:1/762: read d5/d2c/f4b [1997925,20196] 0 2024-09-16T08:57:15.567 INFO:tasks.workunit.client.1.smithi097.stdout:1/763: write d5/db/dc/fad [161163,25492] 0 2024-09-16T08:57:15.568 INFO:tasks.workunit.client.1.smithi097.stdout:1/764: write d5/d2c/f5e [1318020,28427] 0 2024-09-16T08:57:15.568 INFO:tasks.workunit.client.1.smithi097.stdout:2/721: rename d3/d4/d8/d51/l52 to d3/d4/d9/d78/dcc/ld7 0 2024-09-16T08:57:15.569 INFO:tasks.workunit.client.0.smithi031.stdout:4/170: rmdir d0 39 2024-09-16T08:57:15.574 INFO:tasks.workunit.client.1.smithi097.stdout:0/688: sync 2024-09-16T08:57:15.576 INFO:tasks.workunit.client.0.smithi031.stdout:3/384: symlink d3/d14/d37/l71 0 2024-09-16T08:57:15.576 INFO:tasks.workunit.client.0.smithi031.stdout:3/385: fsync d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 0 2024-09-16T08:57:15.578 INFO:tasks.workunit.client.1.smithi097.stdout:6/720: rename d1/dd3/d9e/leb to d1/dd3/db0/lef 0 2024-09-16T08:57:15.578 INFO:tasks.workunit.client.1.smithi097.stdout:3/716: dwrite d7/de/d3c/d3f/d60/d7a/fa1 [0,4194304] 0 2024-09-16T08:57:15.579 INFO:tasks.workunit.client.0.smithi031.stdout:5/331: mknod d2/dc/d25/c69 0 2024-09-16T08:57:15.579 INFO:tasks.workunit.client.0.smithi031.stdout:5/332: write d2/d4b/f51 [172465,117483] 0 2024-09-16T08:57:15.582 INFO:tasks.workunit.client.0.smithi031.stdout:4/171: creat d0/f38 x:0 0 0 2024-09-16T08:57:15.584 INFO:tasks.workunit.client.1.smithi097.stdout:9/632: getdents d2/d5 0 2024-09-16T08:57:15.589 INFO:tasks.workunit.client.0.smithi031.stdout:3/386: unlink d3/l30 0 2024-09-16T08:57:15.589 INFO:tasks.workunit.client.0.smithi031.stdout:3/387: fsync d3/d14/d37/d47/f66 0 2024-09-16T08:57:15.589 INFO:tasks.workunit.client.0.smithi031.stdout:4/172: dread d0/d2/d16/d1e/f23 [0,4194304] 0 2024-09-16T08:57:15.591 INFO:tasks.workunit.client.1.smithi097.stdout:0/689: mknod da/d24/d4c/d80/cf9 0 2024-09-16T08:57:15.591 INFO:tasks.workunit.client.1.smithi097.stdout:0/690: write da/d1a/d26/d47/d97/d84/fc5 [1287720,6192] 0 2024-09-16T08:57:15.591 INFO:tasks.workunit.client.1.smithi097.stdout:2/722: dwrite d3/d4/d8/f8e [0,4194304] 0 2024-09-16T08:57:15.591 INFO:tasks.workunit.client.1.smithi097.stdout:6/721: dread d1/dd3/d9e/dca/d81/f88 [0,4194304] 0 2024-09-16T08:57:15.591 INFO:tasks.workunit.client.0.smithi031.stdout:3/388: creat d3/d14/d6f/f72 x:0 0 0 2024-09-16T08:57:15.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/173: link d0/d2/d16/f22 d0/f39 0 2024-09-16T08:57:15.597 INFO:tasks.workunit.client.1.smithi097.stdout:9/633: symlink d2/d5/db0/dc8/d74/d8c/d94/d96/dac/lcc 0 2024-09-16T08:57:15.598 INFO:tasks.workunit.client.0.smithi031.stdout:4/174: mkdir d0/d2/d3a 0 2024-09-16T08:57:15.598 INFO:tasks.workunit.client.0.smithi031.stdout:4/175: chown d0/f2e 1771625 1 2024-09-16T08:57:15.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/691: mkdir da/d1a/d26/d47/db3/d60/d73/d85/dfa 0 2024-09-16T08:57:15.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/692: chown da/d24/d4b/f52 124 1 2024-09-16T08:57:15.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/693: readlink da/l9b 0 2024-09-16T08:57:15.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/694: getdents da/df/dd8 0 2024-09-16T08:57:15.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/695: fsync da/d1a/f2b 0 2024-09-16T08:57:15.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/696: chown da/d1a/d26/d47/d97/d84/fc5 52600 1 2024-09-16T08:57:15.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/697: stat da/d1a/d26/d94/cf7 0 2024-09-16T08:57:15.601 INFO:tasks.workunit.client.0.smithi031.stdout:4/176: dread d0/d2/d16/d1e/f23 [0,4194304] 0 2024-09-16T08:57:15.602 INFO:tasks.workunit.client.1.smithi097.stdout:2/723: truncate d3/d4/d68/d8b/d7c/d7d/fcd 793588 0 2024-09-16T08:57:15.602 INFO:tasks.workunit.client.0.smithi031.stdout:5/333: dwrite d2/dc/d36/f52 [0,4194304] 0 2024-09-16T08:57:15.603 INFO:tasks.workunit.client.0.smithi031.stdout:5/334: read d2/dc/d36/f38 [333788,4309] 0 2024-09-16T08:57:15.603 INFO:tasks.workunit.client.0.smithi031.stdout:4/177: creat d0/d2/d16/d1e/f3b x:0 0 0 2024-09-16T08:57:15.605 INFO:tasks.workunit.client.1.smithi097.stdout:3/717: dwrite d7/de/d6c/f37 [0,4194304] 0 2024-09-16T08:57:15.608 INFO:tasks.workunit.client.1.smithi097.stdout:9/634: mkdir d2/d5/d56/d6b/dcd 0 2024-09-16T08:57:15.608 INFO:tasks.workunit.client.1.smithi097.stdout:9/635: fdatasync d2/d5/fbd 0 2024-09-16T08:57:15.609 INFO:tasks.workunit.client.1.smithi097.stdout:0/698: dread da/d1a/d26/f6b [0,4194304] 0 2024-09-16T08:57:15.609 INFO:tasks.workunit.client.1.smithi097.stdout:0/699: write da/d1a/d26/d94/dd5/fd7 [688616,99485] 0 2024-09-16T08:57:15.612 INFO:tasks.workunit.client.1.smithi097.stdout:6/722: getdents d1/dc/d3c 0 2024-09-16T08:57:15.612 INFO:tasks.workunit.client.1.smithi097.stdout:6/723: chown d1/dd3/d9e/dca/d16/l7f 13000 1 2024-09-16T08:57:15.613 INFO:tasks.workunit.client.1.smithi097.stdout:9/636: dread d2/d5/d56/f88 [0,4194304] 0 2024-09-16T08:57:15.615 INFO:tasks.workunit.client.0.smithi031.stdout:5/335: mkdir d2/dc/d1d/d6a 0 2024-09-16T08:57:15.617 INFO:tasks.workunit.client.0.smithi031.stdout:4/178: mkdir d0/d3c 0 2024-09-16T08:57:15.617 INFO:tasks.workunit.client.1.smithi097.stdout:3/718: dread d7/d28/d30/d69/fdb [0,4194304] 0 2024-09-16T08:57:15.619 INFO:tasks.workunit.client.1.smithi097.stdout:0/700: mkdir da/d24/d4b/dfb 0 2024-09-16T08:57:15.621 INFO:tasks.workunit.client.0.smithi031.stdout:4/179: link d0/d2/d16/f30 d0/d2/f3d 0 2024-09-16T08:57:15.624 INFO:tasks.workunit.client.1.smithi097.stdout:5/742: sync 2024-09-16T08:57:15.624 INFO:tasks.workunit.client.1.smithi097.stdout:5/743: write db/d18/d1c/d42/d46/ff0 [651807,18010] 0 2024-09-16T08:57:15.624 INFO:tasks.workunit.client.1.smithi097.stdout:5/744: read db/d18/d1c/f39 [1129263,70435] 0 2024-09-16T08:57:15.627 INFO:tasks.workunit.client.1.smithi097.stdout:2/724: dwrite d3/d4/d68/d8b/d7c/d7d/f6d [0,4194304] 0 2024-09-16T08:57:15.627 INFO:tasks.workunit.client.1.smithi097.stdout:2/725: dread - d3/d4/d68/d8b/d7c/d7d/fb5 zero size 2024-09-16T08:57:15.628 INFO:tasks.workunit.client.1.smithi097.stdout:2/726: readlink d3/l74 0 2024-09-16T08:57:15.628 INFO:tasks.workunit.client.1.smithi097.stdout:6/724: rmdir d1/dd3/d9e/dca/d81/dc4 39 2024-09-16T08:57:15.628 INFO:tasks.workunit.client.1.smithi097.stdout:6/725: fsync d1/dc/d3c/f44 0 2024-09-16T08:57:15.630 INFO:tasks.workunit.client.1.smithi097.stdout:2/727: dread d3/d4/fc8 [0,4194304] 0 2024-09-16T08:57:15.630 INFO:tasks.workunit.client.1.smithi097.stdout:2/728: chown d3/d4/d68/d7f/fbd 62578249 1 2024-09-16T08:57:15.630 INFO:tasks.workunit.client.1.smithi097.stdout:2/729: readlink d3/d4/d9/l5b 0 2024-09-16T08:57:15.630 INFO:tasks.workunit.client.1.smithi097.stdout:2/730: write d3/d4/d68/d8b/d7c/d7d/f1f [4625349,119635] 0 2024-09-16T08:57:15.631 INFO:tasks.workunit.client.1.smithi097.stdout:9/637: rmdir d2/d5/d3a 39 2024-09-16T08:57:15.631 INFO:tasks.workunit.client.1.smithi097.stdout:9/638: dread - d2/d5/db0/dc8/d74/d8c/dad/fb2 zero size 2024-09-16T08:57:15.635 INFO:tasks.workunit.client.1.smithi097.stdout:2/731: write d3/d1b/f97 [681624,3132] 0 2024-09-16T08:57:15.635 INFO:tasks.workunit.client.1.smithi097.stdout:2/732: creat d3/d4/d68/d8b/d61/dac/fd8 x:0 0 0 2024-09-16T08:57:15.636 INFO:tasks.workunit.client.1.smithi097.stdout:3/719: rename d7/d28/d30/d69/fdb to d7/d28/d63/dd8/ff8 0 2024-09-16T08:57:15.636 INFO:tasks.workunit.client.1.smithi097.stdout:5/745: symlink db/d18/d1c/d42/d46/d7e/d76/dca/lf7 0 2024-09-16T08:57:15.640 INFO:tasks.workunit.client.0.smithi031.stdout:5/336: dwrite d2/dc/f3b [0,4194304] 0 2024-09-16T08:57:15.642 INFO:tasks.workunit.client.1.smithi097.stdout:9/639: dread d2/d3f/f43 [0,4194304] 0 2024-09-16T08:57:15.643 INFO:tasks.workunit.client.1.smithi097.stdout:0/701: dwrite da/d24/d76/f8b [0,4194304] 0 2024-09-16T08:57:15.647 INFO:tasks.workunit.client.1.smithi097.stdout:6/726: getdents d1/dd3/d9e/dca/d81/dad/d5a 0 2024-09-16T08:57:15.649 INFO:tasks.workunit.client.1.smithi097.stdout:5/746: dread db/d18/d1c/d42/d46/fa6 [0,4194304] 0 2024-09-16T08:57:15.653 INFO:tasks.workunit.client.1.smithi097.stdout:6/727: read d1/dd3/f68 [850156,53508] 0 2024-09-16T08:57:15.655 INFO:tasks.workunit.client.1.smithi097.stdout:9/640: mknod d2/d5/d56/d6b/cce 0 2024-09-16T08:57:15.655 INFO:tasks.workunit.client.1.smithi097.stdout:9/641: fdatasync d2/d5/d8d/f93 0 2024-09-16T08:57:15.658 INFO:tasks.workunit.client.1.smithi097.stdout:5/747: symlink db/d18/d1c/d42/d46/lf8 0 2024-09-16T08:57:15.660 INFO:tasks.workunit.client.1.smithi097.stdout:2/733: dwrite d3/d4/d68/d8b/f4d [0,4194304] 0 2024-09-16T08:57:15.660 INFO:tasks.workunit.client.1.smithi097.stdout:6/728: truncate d1/dd3/d9e/dca/d81/dad/f61 2807632 0 2024-09-16T08:57:15.661 INFO:tasks.workunit.client.1.smithi097.stdout:3/720: getdents d7/de/d8d/d90/d92 0 2024-09-16T08:57:15.663 INFO:tasks.workunit.client.0.smithi031.stdout:5/337: dwrite d2/dc/d25/d21/d2e/f33 [0,4194304] 0 2024-09-16T08:57:15.667 INFO:tasks.workunit.client.0.smithi031.stdout:5/338: creat d2/dc/d25/d68/f6b x:0 0 0 2024-09-16T08:57:15.668 INFO:tasks.workunit.client.1.smithi097.stdout:9/642: dwrite d2/d5/d56/f5d [0,4194304] 0 2024-09-16T08:57:15.669 INFO:tasks.workunit.client.0.smithi031.stdout:4/180: rmdir d0 39 2024-09-16T08:57:15.670 INFO:tasks.workunit.client.0.smithi031.stdout:5/339: mknod d2/c6c 0 2024-09-16T08:57:15.672 INFO:tasks.workunit.client.0.smithi031.stdout:4/181: symlink d0/d2/d16/d1e/l3e 0 2024-09-16T08:57:15.672 INFO:tasks.workunit.client.0.smithi031.stdout:4/182: chown d0/d2/c6 3 1 2024-09-16T08:57:15.674 INFO:tasks.workunit.client.0.smithi031.stdout:5/340: rename d2/f1b to d2/dc/d25/d21/f6d 0 2024-09-16T08:57:15.674 INFO:tasks.workunit.client.0.smithi031.stdout:5/341: creat d2/dc/d25/d68/f6e x:0 0 0 2024-09-16T08:57:15.674 INFO:tasks.workunit.client.1.smithi097.stdout:8/666: sync 2024-09-16T08:57:15.676 INFO:tasks.workunit.client.1.smithi097.stdout:5/748: symlink db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/lf9 0 2024-09-16T08:57:15.677 INFO:tasks.workunit.client.0.smithi031.stdout:4/183: dread d0/d2/d16/d1e/f34 [0,4194304] 0 2024-09-16T08:57:15.680 INFO:tasks.workunit.client.0.smithi031.stdout:5/342: rename d2/dc/d3d/l50 to d2/dc/l6f 0 2024-09-16T08:57:15.680 INFO:tasks.workunit.client.0.smithi031.stdout:5/343: readlink d2/dc/d25/l40 0 2024-09-16T08:57:15.686 INFO:tasks.workunit.client.0.smithi031.stdout:5/344: symlink d2/l70 0 2024-09-16T08:57:15.688 INFO:tasks.workunit.client.0.smithi031.stdout:4/184: dread d0/d2/f33 [0,4194304] 0 2024-09-16T08:57:15.689 INFO:tasks.workunit.client.1.smithi097.stdout:6/729: link d1/dd3/d9e/dca/f2f d1/dd3/d9e/dca/d81/dc4/dc6/ff0 0 2024-09-16T08:57:15.691 INFO:tasks.workunit.client.0.smithi031.stdout:5/345: link d2/dc/f16 d2/dc/d25/d21/d2e/f71 0 2024-09-16T08:57:15.691 INFO:tasks.workunit.client.1.smithi097.stdout:8/667: dwrite d8/dd/d15/d2c/d48/d56/f6d [0,4194304] 0 2024-09-16T08:57:15.692 INFO:tasks.workunit.client.0.smithi031.stdout:4/185: getdents d0/d2/d16 0 2024-09-16T08:57:15.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/186: link d0/d2/d16/f30 d0/d2/d16/d1e/f3f 0 2024-09-16T08:57:15.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/187: dread - d0/d2/f36 zero size 2024-09-16T08:57:15.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/188: stat d0/fa 0 2024-09-16T08:57:15.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/189: write d0/f2d [939512,129017] 0 2024-09-16T08:57:15.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/190: chown d0/d2/d16/d1e/l3e 1 1 2024-09-16T08:57:15.700 INFO:tasks.workunit.client.0.smithi031.stdout:5/346: dread d2/f13 [0,4194304] 0 2024-09-16T08:57:15.700 INFO:tasks.workunit.client.0.smithi031.stdout:5/347: readlink d2/dc/l6f 0 2024-09-16T08:57:15.701 INFO:tasks.workunit.client.1.smithi097.stdout:0/702: rename da/d1a/d26 to da/d7e/dc4/dfc 0 2024-09-16T08:57:15.705 INFO:tasks.workunit.client.1.smithi097.stdout:8/668: link d8/dd/d15/d2c/d4a/d4d/c8c d8/dd/d15/d2c/d4a/d4e/ccf 0 2024-09-16T08:57:15.707 INFO:tasks.workunit.client.1.smithi097.stdout:3/721: rename d7/de/d3c/d3f/d60/dcb/d35/d3d/f87 to d7/de/d8d/d90/ff9 0 2024-09-16T08:57:15.708 INFO:tasks.workunit.client.1.smithi097.stdout:0/703: truncate da/d24/d4c/d6d/f6e 240426 0 2024-09-16T08:57:15.710 INFO:tasks.workunit.client.1.smithi097.stdout:5/749: getdents db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f 0 2024-09-16T08:57:15.710 INFO:tasks.workunit.client.1.smithi097.stdout:5/750: fsync db/d18/d1c/d42/d46/d7e/d8f/fc2 0 2024-09-16T08:57:15.712 INFO:tasks.workunit.client.1.smithi097.stdout:9/643: rename d2/d5/d8d/l9b to d2/d5/db0/dc8/lcf 0 2024-09-16T08:57:15.713 INFO:tasks.workunit.client.1.smithi097.stdout:2/734: truncate d3/d4/d68/d8b/f4d 1318970 0 2024-09-16T08:57:15.714 INFO:tasks.workunit.client.0.smithi031.stdout:5/348: write d2/dc/f1f [3111797,20528] 0 2024-09-16T08:57:15.715 INFO:tasks.workunit.client.1.smithi097.stdout:5/751: creat db/d18/d1c/d42/d46/d7e/d64/d7f/ffa x:0 0 0 2024-09-16T08:57:15.717 INFO:tasks.workunit.client.1.smithi097.stdout:8/669: getdents d8/dd 0 2024-09-16T08:57:15.717 INFO:tasks.workunit.client.1.smithi097.stdout:8/670: fdatasync d8/dd/d15/d2c/d4a/d4d/d62/f76 0 2024-09-16T08:57:15.717 INFO:tasks.workunit.client.1.smithi097.stdout:8/671: getdents d8/dd/d15/d2c/d72 0 2024-09-16T08:57:15.717 INFO:tasks.workunit.client.1.smithi097.stdout:8/672: getdents d8/dd 0 2024-09-16T08:57:15.717 INFO:tasks.workunit.client.1.smithi097.stdout:8/673: chown d8/dd/d15/d2c/d4a/d4e/d97/lb6 1 1 2024-09-16T08:57:15.719 INFO:tasks.workunit.client.0.smithi031.stdout:4/191: dwrite d0/fb [0,4194304] 0 2024-09-16T08:57:15.720 INFO:tasks.workunit.client.1.smithi097.stdout:6/730: dwrite d1/dd3/d9e/dca/d16/d6a/f99 [0,4194304] 0 2024-09-16T08:57:15.720 INFO:tasks.workunit.client.1.smithi097.stdout:0/704: rename da/d24/d4c/lca to da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/lfd 0 2024-09-16T08:57:15.720 INFO:tasks.workunit.client.1.smithi097.stdout:0/705: dread - da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/fc6 zero size 2024-09-16T08:57:15.720 INFO:tasks.workunit.client.1.smithi097.stdout:0/706: write da/d24/ff0 [804285,60294] 0 2024-09-16T08:57:15.721 INFO:tasks.workunit.client.1.smithi097.stdout:9/644: creat d2/d5/d3a/fd0 x:0 0 0 2024-09-16T08:57:15.724 INFO:tasks.workunit.client.1.smithi097.stdout:2/735: mkdir d3/d4/d68/d8b/d61/dac/dd9 0 2024-09-16T08:57:15.724 INFO:tasks.workunit.client.1.smithi097.stdout:2/736: chown d3/d4/fc8 1 1 2024-09-16T08:57:15.724 INFO:tasks.workunit.client.0.smithi031.stdout:4/192: symlink d0/d2/d16/d2b/l40 0 2024-09-16T08:57:15.726 INFO:tasks.workunit.client.1.smithi097.stdout:5/752: creat db/ffb x:0 0 0 2024-09-16T08:57:15.726 INFO:tasks.workunit.client.1.smithi097.stdout:7/619: sync 2024-09-16T08:57:15.727 INFO:tasks.workunit.client.1.smithi097.stdout:8/674: rmdir d8/dd/d15/d2c 39 2024-09-16T08:57:15.727 INFO:tasks.workunit.client.0.smithi031.stdout:4/193: write d0/d2/f10 [352488,100130] 0 2024-09-16T08:57:15.728 INFO:tasks.workunit.client.0.smithi031.stdout:4/194: fsync d0/d2/d16/d1e/f32 0 2024-09-16T08:57:15.728 INFO:tasks.workunit.client.0.smithi031.stdout:4/195: fdatasync d0/d2/f33 0 2024-09-16T08:57:15.728 INFO:tasks.workunit.client.0.smithi031.stdout:4/196: readlink d0/d2/d16/d1e/l31 0 2024-09-16T08:57:15.730 INFO:tasks.workunit.client.1.smithi097.stdout:6/731: creat d1/dd3/d9e/dca/d16/ff1 x:0 0 0 2024-09-16T08:57:15.730 INFO:tasks.workunit.client.1.smithi097.stdout:6/732: write d1/dd3/d9e/dca/d16/d4f/f94 [318355,781] 0 2024-09-16T08:57:15.731 INFO:tasks.workunit.client.0.smithi031.stdout:4/197: dread d0/d2/f3d [0,4194304] 0 2024-09-16T08:57:15.733 INFO:tasks.workunit.client.1.smithi097.stdout:4/654: sync 2024-09-16T08:57:15.733 INFO:tasks.workunit.client.1.smithi097.stdout:1/765: sync 2024-09-16T08:57:15.734 INFO:tasks.workunit.client.1.smithi097.stdout:9/645: unlink d2/d3f/c59 0 2024-09-16T08:57:15.735 INFO:tasks.workunit.client.1.smithi097.stdout:0/707: truncate da/d24/d4c/d80/d3b/d86/faa 53416 0 2024-09-16T08:57:15.735 INFO:tasks.workunit.client.1.smithi097.stdout:5/753: symlink db/d22/d58/lfc 0 2024-09-16T08:57:15.737 INFO:tasks.workunit.client.1.smithi097.stdout:7/620: stat d4/d9/d51/d88/l8a 0 2024-09-16T08:57:15.737 INFO:tasks.workunit.client.1.smithi097.stdout:1/766: dread d5/d2c/f5e [0,4194304] 0 2024-09-16T08:57:15.739 INFO:tasks.workunit.client.1.smithi097.stdout:4/655: symlink d0/le4 0 2024-09-16T08:57:15.739 INFO:tasks.workunit.client.1.smithi097.stdout:5/754: mknod db/d6e/d95/cfd 0 2024-09-16T08:57:15.739 INFO:tasks.workunit.client.1.smithi097.stdout:5/755: write db/d18/d1c/d42/d46/d7e/d76/dc4/feb [279322,81037] 0 2024-09-16T08:57:15.741 INFO:tasks.workunit.client.1.smithi097.stdout:4/656: mkdir d0/d23/ddc/de5 0 2024-09-16T08:57:15.743 INFO:tasks.workunit.client.1.smithi097.stdout:3/722: dwrite d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa/fea [0,4194304] 0 2024-09-16T08:57:15.746 INFO:tasks.workunit.client.1.smithi097.stdout:4/657: rmdir d0/d36 39 2024-09-16T08:57:15.749 INFO:tasks.workunit.client.1.smithi097.stdout:9/646: write d2/d5/f11 [696828,98302] 0 2024-09-16T08:57:15.756 INFO:tasks.workunit.client.1.smithi097.stdout:4/658: creat d0/d23/ddc/de5/fe6 x:0 0 0 2024-09-16T08:57:15.757 INFO:tasks.workunit.client.1.smithi097.stdout:5/756: fsync db/ffb 0 2024-09-16T08:57:15.758 INFO:tasks.workunit.client.1.smithi097.stdout:9/647: rename d2/d5/db0/dc8/d74/d8c/d94/lbb to d2/d5/db0/dc8/d74/d8c/d94/ld1 0 2024-09-16T08:57:15.758 INFO:tasks.workunit.client.1.smithi097.stdout:9/648: truncate d2/d5/d3a/d3c/fc6 661500 0 2024-09-16T08:57:15.759 INFO:tasks.workunit.client.1.smithi097.stdout:3/723: rmdir d7/de/d3c/dde 39 2024-09-16T08:57:15.759 INFO:tasks.workunit.client.1.smithi097.stdout:0/708: dread da/d7e/dc4/dfc/d94/dd5/fd7 [0,4194304] 0 2024-09-16T08:57:15.760 INFO:tasks.workunit.client.1.smithi097.stdout:4/659: mkdir d0/d35/da4/dc5/de7 0 2024-09-16T08:57:15.760 INFO:tasks.workunit.client.1.smithi097.stdout:5/757: truncate db/d18/d1c/d42/d46/d7e/d64/d7f/f85 231737 0 2024-09-16T08:57:15.763 INFO:tasks.workunit.client.1.smithi097.stdout:3/724: mknod d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/cfa 0 2024-09-16T08:57:15.763 INFO:tasks.workunit.client.1.smithi097.stdout:0/709: getdents da/d7e/dc4/dfc/d47/db3/dbf 0 2024-09-16T08:57:15.765 INFO:tasks.workunit.client.1.smithi097.stdout:4/660: dread d0/d7/f5b [0,4194304] 0 2024-09-16T08:57:15.766 INFO:tasks.workunit.client.1.smithi097.stdout:2/737: dwrite d3/d4/d68/d8b/d7c/d7d/f6b [0,4194304] 0 2024-09-16T08:57:15.766 INFO:tasks.workunit.client.0.smithi031.stdout:5/349: dwrite d2/dc/d25/d21/f45 [0,4194304] 0 2024-09-16T08:57:15.767 INFO:tasks.workunit.client.1.smithi097.stdout:5/758: mknod db/d18/d1c/d42/d46/d7e/d64/d8d/cfe 0 2024-09-16T08:57:15.767 INFO:tasks.workunit.client.1.smithi097.stdout:0/710: mknod da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dad/cfe 0 2024-09-16T08:57:15.767 INFO:tasks.workunit.client.1.smithi097.stdout:0/711: fdatasync da/d24/d4c/d80/d3b/f5a 0 2024-09-16T08:57:15.767 INFO:tasks.workunit.client.1.smithi097.stdout:2/738: dread d3/d4/d8/f25 [0,4194304] 0 2024-09-16T08:57:15.767 INFO:tasks.workunit.client.1.smithi097.stdout:2/739: truncate d3/d4/d68/d8b/d61/dac/fd8 761881 0 2024-09-16T08:57:15.768 INFO:tasks.workunit.client.1.smithi097.stdout:0/712: dread da/d7e/dc4/dfc/d47/db3/d60/f65 [0,4194304] 0 2024-09-16T08:57:15.768 INFO:tasks.workunit.client.1.smithi097.stdout:0/713: stat da/d7e/dc4/dfc/d47/db3/d60 0 2024-09-16T08:57:15.770 INFO:tasks.workunit.client.0.smithi031.stdout:4/198: dwrite d0/d2/d16/d1e/f23 [0,4194304] 0 2024-09-16T08:57:15.770 INFO:tasks.workunit.client.0.smithi031.stdout:4/199: chown d0/d2/f1a 7315 1 2024-09-16T08:57:15.772 INFO:tasks.workunit.client.1.smithi097.stdout:8/675: dwrite d8/fa [0,4194304] 0 2024-09-16T08:57:15.774 INFO:tasks.workunit.client.1.smithi097.stdout:4/661: dread d0/d7/df/d15/d1f/f81 [0,4194304] 0 2024-09-16T08:57:15.778 INFO:tasks.workunit.client.1.smithi097.stdout:4/662: dread d0/d36/d4c/f80 [0,4194304] 0 2024-09-16T08:57:15.780 INFO:tasks.workunit.client.1.smithi097.stdout:5/759: symlink db/d18/d1c/d42/lff 0 2024-09-16T08:57:15.780 INFO:tasks.workunit.client.1.smithi097.stdout:0/714: mknod da/df/cff 0 2024-09-16T08:57:15.780 INFO:tasks.workunit.client.1.smithi097.stdout:5/760: truncate db/d18/d1c/d42/d46/ff0 1690426 0 2024-09-16T08:57:15.780 INFO:tasks.workunit.client.1.smithi097.stdout:4/663: symlink d0/d7/df/le8 0 2024-09-16T08:57:15.781 INFO:tasks.workunit.client.0.smithi031.stdout:4/200: dread d0/d2/d16/d1e/f23 [0,4194304] 0 2024-09-16T08:57:15.782 INFO:tasks.workunit.client.1.smithi097.stdout:7/621: dwrite d4/d1b/d41/f81 [4194304,4194304] 0 2024-09-16T08:57:15.782 INFO:tasks.workunit.client.1.smithi097.stdout:7/622: write d4/d12/d1f/f21 [1766540,13253] 0 2024-09-16T08:57:15.782 INFO:tasks.workunit.client.1.smithi097.stdout:7/623: readlink d4/d9/d2b/l85 0 2024-09-16T08:57:15.783 INFO:tasks.workunit.client.0.smithi031.stdout:4/201: creat d0/d2/f41 x:0 0 0 2024-09-16T08:57:15.784 INFO:tasks.workunit.client.1.smithi097.stdout:1/767: write d5/db/f34 [2002010,93543] 0 2024-09-16T08:57:15.784 INFO:tasks.workunit.client.1.smithi097.stdout:7/624: truncate d4/d9/dc/f72 3566005 0 2024-09-16T08:57:15.784 INFO:tasks.workunit.client.1.smithi097.stdout:7/625: write d4/d12/d1f/fc2 [439750,52498] 0 2024-09-16T08:57:15.786 INFO:tasks.workunit.client.0.smithi031.stdout:4/202: truncate d0/fa 1494935 0 2024-09-16T08:57:15.786 INFO:tasks.workunit.client.1.smithi097.stdout:5/761: dread db/d22/f4b [0,4194304] 0 2024-09-16T08:57:15.787 INFO:tasks.workunit.client.1.smithi097.stdout:5/762: stat db/d18/d1c/d42/d46/d7e/d64/d7f/f85 0 2024-09-16T08:57:15.787 INFO:tasks.workunit.client.1.smithi097.stdout:5/763: write db/d18/d1c/d42/f4c [3355002,104527] 0 2024-09-16T08:57:15.787 INFO:tasks.workunit.client.1.smithi097.stdout:5/764: truncate db/d18/d1c/fc1 1811766 0 2024-09-16T08:57:15.787 INFO:tasks.workunit.client.1.smithi097.stdout:1/768: creat d5/db/ff7 x:0 0 0 2024-09-16T08:57:15.788 INFO:tasks.workunit.client.1.smithi097.stdout:7/626: creat d4/d9/d51/db7/fd4 x:0 0 0 2024-09-16T08:57:15.788 INFO:tasks.workunit.client.1.smithi097.stdout:7/627: truncate d4/d9/d1c/f28 1704905 0 2024-09-16T08:57:15.788 INFO:tasks.workunit.client.1.smithi097.stdout:7/628: fsync d4/d7/fbd 0 2024-09-16T08:57:15.790 INFO:tasks.workunit.client.1.smithi097.stdout:7/629: mknod d4/d9/d51/cd5 0 2024-09-16T08:57:15.790 INFO:tasks.workunit.client.1.smithi097.stdout:7/630: chown d4/d1b/fb5 126221 1 2024-09-16T08:57:15.791 INFO:tasks.workunit.client.1.smithi097.stdout:5/765: mknod db/d18/c100 0 2024-09-16T08:57:15.792 INFO:tasks.workunit.client.1.smithi097.stdout:5/766: chown db/d18/d1c/d42/d46/d7e/d64/c88 116194612 1 2024-09-16T08:57:15.792 INFO:tasks.workunit.client.1.smithi097.stdout:1/769: dread d5/d14/d95/fb8 [0,4194304] 0 2024-09-16T08:57:15.797 INFO:tasks.workunit.client.0.smithi031.stdout:4/203: dread d0/d2/d16/d1e/f23 [0,4194304] 0 2024-09-16T08:57:15.797 INFO:tasks.workunit.client.1.smithi097.stdout:5/767: rmdir db/d18/d20/d2b/df1 0 2024-09-16T08:57:15.798 INFO:tasks.workunit.client.1.smithi097.stdout:1/770: unlink d5/d17/f5b 0 2024-09-16T08:57:15.799 INFO:tasks.workunit.client.1.smithi097.stdout:9/649: dwrite d2/d3f/f41 [0,4194304] 0 2024-09-16T08:57:15.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/740: dwrite d3/d4/d77/f70 [0,4194304] 0 2024-09-16T08:57:15.799 INFO:tasks.workunit.client.1.smithi097.stdout:2/741: readlink d3/d4/d68/d8b/d7c/lb0 0 2024-09-16T08:57:15.800 INFO:tasks.workunit.client.0.smithi031.stdout:4/204: mknod d0/c42 0 2024-09-16T08:57:15.800 INFO:tasks.workunit.client.0.smithi031.stdout:4/205: chown d0/c2a 248415 1 2024-09-16T08:57:15.802 INFO:tasks.workunit.client.1.smithi097.stdout:5/768: mknod db/d18/d1c/d42/d46/d7e/d76/c101 0 2024-09-16T08:57:15.802 INFO:tasks.workunit.client.1.smithi097.stdout:5/769: chown db/d6e/d79/dd5/le0 4652470 1 2024-09-16T08:57:15.804 INFO:tasks.workunit.client.1.smithi097.stdout:1/771: stat d5/d70/c8a 0 2024-09-16T08:57:15.804 INFO:tasks.workunit.client.1.smithi097.stdout:1/772: chown d5/d17/d54/d6b/da8/dc5/dc7 2155982 1 2024-09-16T08:57:15.810 INFO:tasks.workunit.client.1.smithi097.stdout:9/650: link d2/d5/fb d2/d5/d6e/fd2 0 2024-09-16T08:57:15.810 INFO:tasks.workunit.client.0.smithi031.stdout:5/350: dwrite d2/dc/d25/d68/f6e [0,4194304] 0 2024-09-16T08:57:15.815 INFO:tasks.workunit.client.0.smithi031.stdout:4/206: write d0/f1 [1652027,115387] 0 2024-09-16T08:57:15.817 INFO:tasks.workunit.client.1.smithi097.stdout:0/715: dwrite da/d7e/dc4/dfc/d47/faf [0,4194304] 0 2024-09-16T08:57:15.817 INFO:tasks.workunit.client.1.smithi097.stdout:0/716: chown da/d1a/d41/db2/ff2 831 1 2024-09-16T08:57:15.818 INFO:tasks.workunit.client.1.smithi097.stdout:2/742: unlink d3/d4/d68/d8b/f4d 0 2024-09-16T08:57:15.819 INFO:tasks.workunit.client.0.smithi031.stdout:4/207: dread d0/d2/f33 [0,4194304] 0 2024-09-16T08:57:15.819 INFO:tasks.workunit.client.0.smithi031.stdout:4/208: read d0/f1 [3336982,53639] 0 2024-09-16T08:57:15.820 INFO:tasks.workunit.client.1.smithi097.stdout:1/773: getdents d5/db/dd4 0 2024-09-16T08:57:15.823 INFO:tasks.workunit.client.1.smithi097.stdout:5/770: dwrite db/d18/d20/d2b/f87 [0,4194304] 0 2024-09-16T08:57:15.823 INFO:tasks.workunit.client.1.smithi097.stdout:0/717: dread da/d24/d4c/d80/d3b/f6a [0,4194304] 0 2024-09-16T08:57:15.824 INFO:tasks.workunit.client.1.smithi097.stdout:0/718: fsync da/d7e/dc4/dfc/d47/db3/d60/d73/fa4 0 2024-09-16T08:57:15.827 INFO:tasks.workunit.client.1.smithi097.stdout:4/664: dwrite d0/d35/da4/dc5/fe2 [0,4194304] 0 2024-09-16T08:57:15.827 INFO:tasks.workunit.client.1.smithi097.stdout:4/665: chown d0/d36/da7/ldf 60863 1 2024-09-16T08:57:15.828 INFO:tasks.workunit.client.0.smithi031.stdout:5/351: creat d2/dc/d25/d21/d2e/f72 x:0 0 0 2024-09-16T08:57:15.829 INFO:tasks.workunit.client.0.smithi031.stdout:9/259: sync 2024-09-16T08:57:15.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/209: creat d0/f43 x:0 0 0 2024-09-16T08:57:15.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/210: chown d0/d2/f33 83232555 1 2024-09-16T08:57:15.833 INFO:tasks.workunit.client.1.smithi097.stdout:9/651: dwrite d2/f46 [0,4194304] 0 2024-09-16T08:57:15.837 INFO:tasks.workunit.client.0.smithi031.stdout:7/356: sync 2024-09-16T08:57:15.837 INFO:tasks.workunit.client.0.smithi031.stdout:8/212: sync 2024-09-16T08:57:15.837 INFO:tasks.workunit.client.0.smithi031.stdout:8/213: fsync d4/d7/d21/d24/f31 0 2024-09-16T08:57:15.837 INFO:tasks.workunit.client.0.smithi031.stdout:2/229: sync 2024-09-16T08:57:15.839 INFO:tasks.workunit.client.0.smithi031.stdout:4/211: write d0/fa [1033232,121321] 0 2024-09-16T08:57:15.839 INFO:tasks.workunit.client.0.smithi031.stdout:4/212: write d0/d2/d16/f19 [1036712,113952] 0 2024-09-16T08:57:15.842 INFO:tasks.workunit.client.0.smithi031.stdout:5/352: creat d2/dc/d25/d21/f73 x:0 0 0 2024-09-16T08:57:15.843 INFO:tasks.workunit.client.0.smithi031.stdout:9/260: unlink d3/d5/da/l32 0 2024-09-16T08:57:15.844 INFO:tasks.workunit.client.1.smithi097.stdout:2/743: creat d3/d4/d68/d8b/fda x:0 0 0 2024-09-16T08:57:15.845 INFO:tasks.workunit.client.1.smithi097.stdout:2/744: write d3/d4/fc8 [8155,10885] 0 2024-09-16T08:57:15.847 INFO:tasks.workunit.client.1.smithi097.stdout:1/774: rename d5/db/dd4/fee to d5/d14/d95/ff8 0 2024-09-16T08:57:15.847 INFO:tasks.workunit.client.1.smithi097.stdout:1/775: chown d5/d70/le3 1845182634 1 2024-09-16T08:57:15.848 INFO:tasks.workunit.client.0.smithi031.stdout:7/357: symlink d2/d2d/d6d/l7d 0 2024-09-16T08:57:15.849 INFO:tasks.workunit.client.0.smithi031.stdout:4/213: dread d0/d2/d16/f22 [0,4194304] 0 2024-09-16T08:57:15.851 INFO:tasks.workunit.client.0.smithi031.stdout:8/214: creat d4/d7/d21/d35/d40/f4c x:0 0 0 2024-09-16T08:57:15.853 INFO:tasks.workunit.client.0.smithi031.stdout:2/230: rename d6/d20/c29 to d6/d20/c49 0 2024-09-16T08:57:15.854 INFO:tasks.workunit.client.1.smithi097.stdout:5/771: rmdir db/d6e/d81 39 2024-09-16T08:57:15.855 INFO:tasks.workunit.client.0.smithi031.stdout:9/261: dread d3/d5/f2d [0,4194304] 0 2024-09-16T08:57:15.856 INFO:tasks.workunit.client.1.smithi097.stdout:0/719: creat da/d24/d4c/d80/d42/f100 x:0 0 0 2024-09-16T08:57:15.857 INFO:tasks.workunit.client.0.smithi031.stdout:9/262: write d3/d5/da/f3e [329646,13364] 0 2024-09-16T08:57:15.857 INFO:tasks.workunit.client.0.smithi031.stdout:9/263: write d3/d5/da/f2c [257812,78766] 0 2024-09-16T08:57:15.857 INFO:tasks.workunit.client.0.smithi031.stdout:9/264: creat d3/d5/da/d34/f5c x:0 0 0 2024-09-16T08:57:15.859 INFO:tasks.workunit.client.0.smithi031.stdout:5/353: creat d2/dc/d3d/f74 x:0 0 0 2024-09-16T08:57:15.859 INFO:tasks.workunit.client.0.smithi031.stdout:5/354: creat d2/dc/f75 x:0 0 0 2024-09-16T08:57:15.859 INFO:tasks.workunit.client.0.smithi031.stdout:7/358: rmdir d2/d2d/d6d 39 2024-09-16T08:57:15.861 INFO:tasks.workunit.client.0.smithi031.stdout:1/242: sync 2024-09-16T08:57:15.861 INFO:tasks.workunit.client.0.smithi031.stdout:0/254: sync 2024-09-16T08:57:15.861 INFO:tasks.workunit.client.0.smithi031.stdout:0/255: dread - d6/d12/d13/d3c/f45 zero size 2024-09-16T08:57:15.861 INFO:tasks.workunit.client.0.smithi031.stdout:6/377: sync 2024-09-16T08:57:15.861 INFO:tasks.workunit.client.0.smithi031.stdout:3/389: sync 2024-09-16T08:57:15.861 INFO:tasks.workunit.client.0.smithi031.stdout:4/214: rmdir d0/d2/d16/d2b 39 2024-09-16T08:57:15.863 INFO:tasks.workunit.client.0.smithi031.stdout:8/215: symlink d4/l4d 0 2024-09-16T08:57:15.864 INFO:tasks.workunit.client.1.smithi097.stdout:4/666: symlink d0/d7/le9 0 2024-09-16T08:57:15.865 INFO:tasks.workunit.client.0.smithi031.stdout:2/231: symlink d6/da/l4a 0 2024-09-16T08:57:15.867 INFO:tasks.workunit.client.0.smithi031.stdout:9/265: dread d3/fc [0,4194304] 0 2024-09-16T08:57:15.867 INFO:tasks.workunit.client.0.smithi031.stdout:9/266: dread - d3/f3a zero size 2024-09-16T08:57:15.871 INFO:tasks.workunit.client.1.smithi097.stdout:1/776: dwrite d5/db/dc/d19/f4d [0,4194304] 0 2024-09-16T08:57:15.871 INFO:tasks.workunit.client.1.smithi097.stdout:1/777: write d5/f77 [4385465,126791] 0 2024-09-16T08:57:15.872 INFO:tasks.workunit.client.0.smithi031.stdout:1/243: dread d2/d4/f1c [0,4194304] 0 2024-09-16T08:57:15.872 INFO:tasks.workunit.client.0.smithi031.stdout:1/244: write d2/d4/d1b/f3b [481084,87312] 0 2024-09-16T08:57:15.873 INFO:tasks.workunit.client.1.smithi097.stdout:9/652: creat d2/d5/db0/dc8/d63/da8/dca/fd3 x:0 0 0 2024-09-16T08:57:15.873 INFO:tasks.workunit.client.1.smithi097.stdout:9/653: chown d2/d5/db0/dc8/f5a 6 1 2024-09-16T08:57:15.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/267: dread d3/d5/f2d [0,4194304] 0 2024-09-16T08:57:15.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/268: dread - d3/f3a zero size 2024-09-16T08:57:15.875 INFO:tasks.workunit.client.0.smithi031.stdout:5/355: unlink d2/dc/d3d/l58 0 2024-09-16T08:57:15.876 INFO:tasks.workunit.client.0.smithi031.stdout:9/269: dread d3/d5/da/d34/f46 [0,4194304] 0 2024-09-16T08:57:15.878 INFO:tasks.workunit.client.0.smithi031.stdout:7/359: unlink d2/c15 0 2024-09-16T08:57:15.880 INFO:tasks.workunit.client.1.smithi097.stdout:2/745: mknod d3/d4/d68/d8b/d61/dac/cdb 0 2024-09-16T08:57:15.880 INFO:tasks.workunit.client.1.smithi097.stdout:2/746: readlink d3/d4/d9/l5b 0 2024-09-16T08:57:15.880 INFO:tasks.workunit.client.1.smithi097.stdout:2/747: fdatasync d3/d4/fc8 0 2024-09-16T08:57:15.880 INFO:tasks.workunit.client.1.smithi097.stdout:2/748: chown d3/d4/d68/d8b/d7c/lb9 192 1 2024-09-16T08:57:15.882 INFO:tasks.workunit.client.0.smithi031.stdout:0/256: symlink d6/d12/d13/d36/l58 0 2024-09-16T08:57:15.882 INFO:tasks.workunit.client.1.smithi097.stdout:5/772: mkdir db/d18/d1c/d42/d46/d7e/d8f/d102 0 2024-09-16T08:57:15.883 INFO:tasks.workunit.client.1.smithi097.stdout:2/749: dread d3/d4/d68/d8b/d7c/d7d/d53/f6c [0,4194304] 0 2024-09-16T08:57:15.885 INFO:tasks.workunit.client.0.smithi031.stdout:6/378: rename d0/dd/d35/d59/f24 to d0/dd/d2b/f73 0 2024-09-16T08:57:15.887 INFO:tasks.workunit.client.1.smithi097.stdout:0/720: rename da/df/d3d to da/d7e/dc4/dfc/d94/d101 0 2024-09-16T08:57:15.887 INFO:tasks.workunit.client.1.smithi097.stdout:0/721: readlink da/d24/d4c/d6d/ldb 0 2024-09-16T08:57:15.887 INFO:tasks.workunit.client.1.smithi097.stdout:0/722: chown da/d7e/dc4/dfc/d47/d97/d84/fc5 69 1 2024-09-16T08:57:15.890 INFO:tasks.workunit.client.0.smithi031.stdout:4/215: link d0/d2/cf d0/d2/c44 0 2024-09-16T08:57:15.891 INFO:tasks.workunit.client.0.smithi031.stdout:8/216: symlink d4/d7/d36/l4e 0 2024-09-16T08:57:15.894 INFO:tasks.workunit.client.0.smithi031.stdout:6/379: dread d0/d10/f3b [0,4194304] 0 2024-09-16T08:57:15.895 INFO:tasks.workunit.client.1.smithi097.stdout:1/778: getdents d5/ddd 0 2024-09-16T08:57:15.895 INFO:tasks.workunit.client.1.smithi097.stdout:1/779: truncate d5/d17/fc9 3737300 0 2024-09-16T08:57:15.895 INFO:tasks.workunit.client.1.smithi097.stdout:1/780: write d5/db/dc/d19/f88 [1216469,31889] 0 2024-09-16T08:57:15.897 INFO:tasks.workunit.client.1.smithi097.stdout:9/654: creat d2/d3f/fd4 x:0 0 0 2024-09-16T08:57:15.897 INFO:tasks.workunit.client.1.smithi097.stdout:9/655: truncate d2/d5/fb 8679171 0 2024-09-16T08:57:15.901 INFO:tasks.workunit.client.0.smithi031.stdout:5/356: rmdir d2/dc/d1d/d6a 0 2024-09-16T08:57:15.903 INFO:tasks.workunit.client.1.smithi097.stdout:4/667: rename d0/d7/df/d15/db7 to d0/d36/d4c/dea 0 2024-09-16T08:57:15.903 INFO:tasks.workunit.client.1.smithi097.stdout:4/668: write d0/d36/da7/faa [466967,1549] 0 2024-09-16T08:57:15.905 INFO:tasks.workunit.client.0.smithi031.stdout:9/270: creat d3/d5/f5d x:0 0 0 2024-09-16T08:57:15.907 INFO:tasks.workunit.client.0.smithi031.stdout:1/245: dwrite d2/d4/d22/f45 [0,4194304] 0 2024-09-16T08:57:15.907 INFO:tasks.workunit.client.0.smithi031.stdout:7/360: dwrite d2/d8/d11/f74 [0,4194304] 0 2024-09-16T08:57:15.908 INFO:tasks.workunit.client.0.smithi031.stdout:0/257: link d6/d12/d13/f21 d6/d12/d13/d36/f59 0 2024-09-16T08:57:15.908 INFO:tasks.workunit.client.0.smithi031.stdout:0/258: write d6/f7 [4611569,73874] 0 2024-09-16T08:57:15.909 INFO:tasks.workunit.client.1.smithi097.stdout:5/773: dwrite db/d18/d1c/d42/d46/d7e/d64/d7f/d89/fc3 [0,4194304] 0 2024-09-16T08:57:15.909 INFO:tasks.workunit.client.1.smithi097.stdout:9/656: mkdir d2/d5/dd5 0 2024-09-16T08:57:15.911 INFO:tasks.workunit.client.0.smithi031.stdout:3/390: rename d3/d14/d37/d1a/d1d/f2d to d3/d14/d6f/f73 0 2024-09-16T08:57:15.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/216: symlink d0/l45 0 2024-09-16T08:57:15.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/217: stat d0/d2/f1a 0 2024-09-16T08:57:15.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/218: readlink d0/d2/d16/d1e/l3e 0 2024-09-16T08:57:15.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/219: write d0/d2/f27 [322611,128975] 0 2024-09-16T08:57:15.914 INFO:tasks.workunit.client.1.smithi097.stdout:4/669: creat d0/d7/d52/d26/d3d/feb x:0 0 0 2024-09-16T08:57:15.915 INFO:tasks.workunit.client.1.smithi097.stdout:4/670: write d0/d39/fa3 [4901525,58771] 0 2024-09-16T08:57:15.915 INFO:tasks.workunit.client.0.smithi031.stdout:8/217: unlink d4/f18 0 2024-09-16T08:57:15.918 INFO:tasks.workunit.client.0.smithi031.stdout:9/271: mkdir d3/d5/da/d34/d3c/d5e 0 2024-09-16T08:57:15.918 INFO:tasks.workunit.client.0.smithi031.stdout:9/272: fsync d3/d5/da/d34/f46 0 2024-09-16T08:57:15.918 INFO:tasks.workunit.client.0.smithi031.stdout:9/273: chown d3/d5/da/db/fe 11097464 1 2024-09-16T08:57:15.919 INFO:tasks.workunit.client.1.smithi097.stdout:6/733: sync 2024-09-16T08:57:15.919 INFO:tasks.workunit.client.1.smithi097.stdout:6/734: truncate d1/dd8/fe6 159187 0 2024-09-16T08:57:15.920 INFO:tasks.workunit.client.0.smithi031.stdout:7/361: write d2/d8/d11/f74 [3331292,42104] 0 2024-09-16T08:57:15.922 INFO:tasks.workunit.client.1.smithi097.stdout:9/657: symlink d2/d3f/ld6 0 2024-09-16T08:57:15.922 INFO:tasks.workunit.client.1.smithi097.stdout:9/658: dread - d2/d3f/fb4 zero size 2024-09-16T08:57:15.922 INFO:tasks.workunit.client.1.smithi097.stdout:9/659: dread - d2/d5/d3a/d70/da6/fcb zero size 2024-09-16T08:57:15.922 INFO:tasks.workunit.client.1.smithi097.stdout:9/660: chown d2/fe 124089 1 2024-09-16T08:57:15.922 INFO:tasks.workunit.client.0.smithi031.stdout:1/246: getdents d2/d4/d36 0 2024-09-16T08:57:15.923 INFO:tasks.workunit.client.0.smithi031.stdout:1/247: truncate d2/de/f2a 8949171 0 2024-09-16T08:57:15.923 INFO:tasks.workunit.client.1.smithi097.stdout:4/671: rmdir d0/d7/d52/d26/d8c 39 2024-09-16T08:57:15.925 INFO:tasks.workunit.client.1.smithi097.stdout:1/781: rmdir d5/db/dd4 0 2024-09-16T08:57:15.927 INFO:tasks.workunit.client.1.smithi097.stdout:4/672: creat d0/d35/da4/dc5/de7/fec x:0 0 0 2024-09-16T08:57:15.931 INFO:tasks.workunit.client.1.smithi097.stdout:1/782: mkdir d5/dce/df9 0 2024-09-16T08:57:15.931 INFO:tasks.workunit.client.1.smithi097.stdout:1/783: truncate d5/db/dc/fda 1181147 0 2024-09-16T08:57:15.931 INFO:tasks.workunit.client.1.smithi097.stdout:1/784: fdatasync d5/db/dc/fe0 0 2024-09-16T08:57:15.932 INFO:tasks.workunit.client.1.smithi097.stdout:9/661: write d2/d5/f29 [2246185,81152] 0 2024-09-16T08:57:15.932 INFO:tasks.workunit.client.1.smithi097.stdout:5/774: dwrite db/f84 [0,4194304] 0 2024-09-16T08:57:15.932 INFO:tasks.workunit.client.0.smithi031.stdout:5/357: dwrite d2/dc/f43 [0,4194304] 0 2024-09-16T08:57:15.932 INFO:tasks.workunit.client.0.smithi031.stdout:5/358: dread - d2/dc/d3d/d4c/f62 zero size 2024-09-16T08:57:15.934 INFO:tasks.workunit.client.0.smithi031.stdout:6/380: rename d0/d17 to d0/dd/d74 0 2024-09-16T08:57:15.936 INFO:tasks.workunit.client.1.smithi097.stdout:4/673: symlink d0/d7/led 0 2024-09-16T08:57:15.940 INFO:tasks.workunit.client.1.smithi097.stdout:1/785: creat d5/db/dc/ffa x:0 0 0 2024-09-16T08:57:15.941 INFO:tasks.workunit.client.0.smithi031.stdout:4/220: unlink d0/d2/f36 0 2024-09-16T08:57:15.944 INFO:tasks.workunit.client.1.smithi097.stdout:6/735: dwrite d1/dc/f38 [0,4194304] 0 2024-09-16T08:57:15.944 INFO:tasks.workunit.client.0.smithi031.stdout:8/218: mkdir d4/d7/d21/d35/d40/d4f 0 2024-09-16T08:57:15.947 INFO:tasks.workunit.client.1.smithi097.stdout:9/662: rmdir d2/d5/db0/dc8/d74/d8c/d94/d96 39 2024-09-16T08:57:15.948 INFO:tasks.workunit.client.0.smithi031.stdout:9/274: mknod d3/d5/da/d34/d4d/c5f 0 2024-09-16T08:57:15.948 INFO:tasks.workunit.client.0.smithi031.stdout:3/391: dread d3/fb [0,4194304] 0 2024-09-16T08:57:15.951 INFO:tasks.workunit.client.0.smithi031.stdout:7/362: mkdir d2/d8/d11/d5d/d7e 0 2024-09-16T08:57:15.955 INFO:tasks.workunit.client.0.smithi031.stdout:1/248: creat d2/f53 x:0 0 0 2024-09-16T08:57:15.956 INFO:tasks.workunit.client.1.smithi097.stdout:4/674: read d0/d7/d52/d26/d3d/f44 [1871863,86387] 0 2024-09-16T08:57:15.959 INFO:tasks.workunit.client.1.smithi097.stdout:6/736: symlink d1/dd3/d9e/dca/d81/lf2 0 2024-09-16T08:57:15.961 INFO:tasks.workunit.client.1.smithi097.stdout:4/675: dread d0/d23/f3a [4194304,4194304] 0 2024-09-16T08:57:15.962 INFO:tasks.workunit.client.1.smithi097.stdout:6/737: mknod d1/dd3/d9e/dca/d81/cf3 0 2024-09-16T08:57:15.963 INFO:tasks.workunit.client.0.smithi031.stdout:6/381: mknod d0/dd/d35/c75 0 2024-09-16T08:57:15.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/219: read d4/f6 [9400632,39035] 0 2024-09-16T08:57:15.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/220: fdatasync d4/f43 0 2024-09-16T08:57:15.966 INFO:tasks.workunit.client.1.smithi097.stdout:4/676: rmdir d0/d7/dce 0 2024-09-16T08:57:15.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/221: rename d0/d2/d16/d1e/l31 to d0/d2/d3a/l46 0 2024-09-16T08:57:15.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/222: chown d0/d2/l26 496 1 2024-09-16T08:57:15.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/223: chown d0/d2/d16/c20 11894029 1 2024-09-16T08:57:15.971 INFO:tasks.workunit.client.0.smithi031.stdout:9/275: symlink d3/d5/da/l60 0 2024-09-16T08:57:15.971 INFO:tasks.workunit.client.0.smithi031.stdout:4/224: dread d0/fa [0,4194304] 0 2024-09-16T08:57:15.974 INFO:tasks.workunit.client.0.smithi031.stdout:7/363: creat d2/d8/d69/d7a/f7f x:0 0 0 2024-09-16T08:57:15.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/249: creat d2/d4/f54 x:0 0 0 2024-09-16T08:57:15.976 INFO:tasks.workunit.client.0.smithi031.stdout:0/259: dwrite d6/f7 [0,4194304] 0 2024-09-16T08:57:15.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.976+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f273419a610 con 0x7f27340feaf0 2024-09-16T08:57:15.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.977+0000 7f2731ffb640 1 -- 172.21.15.31:0/407268374 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f272805c440 con 0x7f27340feaf0 2024-09-16T08:57:15.979 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:57:15.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:57:15.979 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:57:15.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:15.979 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:57:15.979 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 1, 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 13, 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:57:15.980 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:57:15.981 INFO:tasks.workunit.client.0.smithi031.stdout:6/382: symlink d0/dd/l76 0 2024-09-16T08:57:15.983 INFO:tasks.workunit.client.1.smithi097.stdout:5/775: dwrite db/d18/d1c/d42/d46/d7e/f67 [0,4194304] 0 2024-09-16T08:57:15.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2724073dd0 msgr2=0x7f2724076290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:15.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2724073dd0 0x7f2724076290 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2720006010 tx=0x7f27200314e0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 msgr2=0x7f27340687d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:15.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 0x7f27340687d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f272800b3d0 tx=0x7f272800b890 comp rx=0 tx=0).stop 2024-09-16T08:57:15.983 INFO:tasks.workunit.client.0.smithi031.stdout:3/392: dwrite f1 [4194304,4194304] 0 2024-09-16T08:57:15.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 shutdown_connections 2024-09-16T08:57:15.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2724073dd0 0x7f2724076290 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27340ffcf0 0x7f2734198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 --2- 172.21.15.31:0/407268374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f27340feaf0 0x7f27340687d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:15.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 >> 172.21.15.31:0/407268374 conn(0x7f27340fa280 msgr2=0x7f27341033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:15.984 INFO:tasks.workunit.client.0.smithi031.stdout:8/221: rmdir d4/d34/d44 0 2024-09-16T08:57:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 shutdown_connections 2024-09-16T08:57:15.985 INFO:tasks.workunit.client.0.smithi031.stdout:8/222: truncate d4/d7/d21/f23 1459904 0 2024-09-16T08:57:15.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:15.981+0000 7f273aed2640 1 -- 172.21.15.31:0/407268374 wait complete. 2024-09-16T08:57:15.986 INFO:tasks.workunit.client.0.smithi031.stdout:8/223: dread - d4/d7/f1c zero size 2024-09-16T08:57:15.986 INFO:tasks.workunit.client.0.smithi031.stdout:9/276: mknod d3/d5/da/d34/d4d/c61 0 2024-09-16T08:57:15.986 INFO:tasks.workunit.client.0.smithi031.stdout:4/225: truncate d0/fa 37152 0 2024-09-16T08:57:15.986 INFO:tasks.workunit.client.1.smithi097.stdout:1/786: dwrite d5/d2c/f4b [0,4194304] 0 2024-09-16T08:57:15.987 INFO:tasks.workunit.client.1.smithi097.stdout:9/663: dwrite d2/d5/db0/dc8/d74/d8c/dad/fb2 [0,4194304] 0 2024-09-16T08:57:15.987 INFO:tasks.workunit.client.1.smithi097.stdout:5/776: read db/d18/d1c/fc1 [531501,107933] 0 2024-09-16T08:57:15.987 INFO:tasks.workunit.client.0.smithi031.stdout:1/250: creat d2/d15/d32/f55 x:0 0 0 2024-09-16T08:57:15.989 INFO:tasks.workunit.client.0.smithi031.stdout:0/260: mknod d6/d12/d13/d36/c5a 0 2024-09-16T08:57:15.989 INFO:tasks.workunit.client.0.smithi031.stdout:0/261: read d6/fa [2982829,27351] 0 2024-09-16T08:57:15.989 INFO:tasks.workunit.client.0.smithi031.stdout:0/262: chown d6/f7 0 1 2024-09-16T08:57:15.989 INFO:tasks.workunit.client.0.smithi031.stdout:0/263: chown d6/d12/d13/d36/c4c 1172 1 2024-09-16T08:57:15.989 INFO:tasks.workunit.client.1.smithi097.stdout:4/677: dwrite d0/fd [0,4194304] 0 2024-09-16T08:57:15.990 INFO:tasks.workunit.client.0.smithi031.stdout:6/383: creat d0/dd/d74/d5e/f77 x:0 0 0 2024-09-16T08:57:15.990 INFO:tasks.workunit.client.1.smithi097.stdout:4/678: dread d0/d7/d52/d26/d3d/f76 [0,4194304] 0 2024-09-16T08:57:15.994 INFO:tasks.workunit.client.0.smithi031.stdout:8/224: mkdir d4/d7/d21/d35/d50 0 2024-09-16T08:57:15.994 INFO:tasks.workunit.client.0.smithi031.stdout:8/225: fdatasync d4/d2f/f41 0 2024-09-16T08:57:15.994 INFO:tasks.workunit.client.0.smithi031.stdout:8/226: write d4/d2d/f3a [5038478,71520] 0 2024-09-16T08:57:15.998 INFO:tasks.workunit.client.0.smithi031.stdout:9/277: mkdir d3/d5/da/d34/d3c/d5e/d62 0 2024-09-16T08:57:15.998 INFO:tasks.workunit.client.0.smithi031.stdout:9/278: chown d3/d5/l12 47 1 2024-09-16T08:57:16.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/226: rename d0/d2/l8 to d0/d2/l47 0 2024-09-16T08:57:16.003 INFO:tasks.workunit.client.0.smithi031.stdout:1/251: mknod d2/d4/d36/c56 0 2024-09-16T08:57:16.004 INFO:tasks.workunit.client.0.smithi031.stdout:1/252: truncate d2/d4/f30 2788999 0 2024-09-16T08:57:16.004 INFO:tasks.workunit.client.0.smithi031.stdout:1/253: fsync d2/d4/d22/f45 0 2024-09-16T08:57:16.004 INFO:tasks.workunit.client.0.smithi031.stdout:1/254: stat d2/d15/d4a 0 2024-09-16T08:57:16.004 INFO:tasks.workunit.client.0.smithi031.stdout:1/255: truncate d2/f3c 986738 0 2024-09-16T08:57:16.004 INFO:tasks.workunit.client.0.smithi031.stdout:6/384: dread d0/d10/f60 [0,4194304] 0 2024-09-16T08:57:16.005 INFO:tasks.workunit.client.0.smithi031.stdout:6/385: creat d0/dd/d35/d59/d56/f78 x:0 0 0 2024-09-16T08:57:16.005 INFO:tasks.workunit.client.0.smithi031.stdout:0/264: mkdir d6/d12/d47/d5b 0 2024-09-16T08:57:16.005 INFO:tasks.workunit.client.0.smithi031.stdout:0/265: fsync d6/d12/d13/d3c/f45 0 2024-09-16T08:57:16.006 INFO:tasks.workunit.client.1.smithi097.stdout:1/787: write d5/db/dc/f41 [2251346,125793] 0 2024-09-16T08:57:16.007 INFO:tasks.workunit.client.1.smithi097.stdout:1/788: chown d5/d17/f90 1759838183 1 2024-09-16T08:57:16.007 INFO:tasks.workunit.client.1.smithi097.stdout:1/789: dread - d5/d2c/dcb/de2/fe5 zero size 2024-09-16T08:57:16.009 INFO:tasks.workunit.client.0.smithi031.stdout:3/393: rename d3/fa to d3/d14/d37/d1a/d1d/d3b/d3f/f74 0 2024-09-16T08:57:16.009 INFO:tasks.workunit.client.0.smithi031.stdout:3/394: fdatasync d3/d14/d37/f10 0 2024-09-16T08:57:16.009 INFO:tasks.workunit.client.0.smithi031.stdout:3/395: dread - d3/d14/d37/d1a/d24/f40 zero size 2024-09-16T08:57:16.017 INFO:tasks.workunit.client.0.smithi031.stdout:9/279: symlink d3/d5/da/db/d21/l63 0 2024-09-16T08:57:16.017 INFO:tasks.workunit.client.0.smithi031.stdout:9/280: chown d3/fc 2906587 1 2024-09-16T08:57:16.017 INFO:tasks.workunit.client.0.smithi031.stdout:0/266: write d6/d12/f2d [177120,39588] 0 2024-09-16T08:57:16.019 INFO:tasks.workunit.client.1.smithi097.stdout:5/777: unlink db/d1a/d6d/l60 0 2024-09-16T08:57:16.019 INFO:tasks.workunit.client.1.smithi097.stdout:5/778: readlink db/d22/d58/lfc 0 2024-09-16T08:57:16.020 INFO:tasks.workunit.client.0.smithi031.stdout:4/227: mknod d0/d2/d16/c48 0 2024-09-16T08:57:16.023 INFO:tasks.workunit.client.0.smithi031.stdout:1/256: link d2/d4/l3a d2/d4/d36/l57 0 2024-09-16T08:57:16.023 INFO:tasks.workunit.client.1.smithi097.stdout:9/664: dwrite d2/d5/db0/dc8/d74/d8c/d94/f99 [0,4194304] 0 2024-09-16T08:57:16.024 INFO:tasks.workunit.client.0.smithi031.stdout:7/364: dwrite d2/f42 [0,4194304] 0 2024-09-16T08:57:16.024 INFO:tasks.workunit.client.1.smithi097.stdout:4/679: dwrite d0/d23/f3a [4194304,4194304] 0 2024-09-16T08:57:16.024 INFO:tasks.workunit.client.1.smithi097.stdout:4/680: creat d0/d35/da4/dd1/fee x:0 0 0 2024-09-16T08:57:16.025 INFO:tasks.workunit.client.1.smithi097.stdout:1/790: symlink d5/d17/d54/d6b/da8/dc5/lfb 0 2024-09-16T08:57:16.025 INFO:tasks.workunit.client.0.smithi031.stdout:8/227: dwrite d4/f27 [0,4194304] 0 2024-09-16T08:57:16.025 INFO:tasks.workunit.client.0.smithi031.stdout:8/228: readlink d4/l12 0 2024-09-16T08:57:16.026 INFO:tasks.workunit.client.1.smithi097.stdout:4/681: dread d0/d7/d52/d26/d3d/f76 [0,4194304] 0 2024-09-16T08:57:16.027 INFO:tasks.workunit.client.0.smithi031.stdout:3/396: dwrite d3/d14/d37/d47/f66 [0,4194304] 0 2024-09-16T08:57:16.029 INFO:tasks.workunit.client.0.smithi031.stdout:6/386: mkdir d0/dd/d74/d5e/d79 0 2024-09-16T08:57:16.031 INFO:tasks.workunit.client.1.smithi097.stdout:5/779: read db/d18/d1c/d42/d46/d7e/d8f/fd8 [1567442,125873] 0 2024-09-16T08:57:16.031 INFO:tasks.workunit.client.1.smithi097.stdout:5/780: read - db/d18/d1c/d42/d46/d7e/d76/f8e zero size 2024-09-16T08:57:16.040 INFO:tasks.workunit.client.0.smithi031.stdout:0/267: rename d6/lf to d6/d12/d47/d5b/l5c 0 2024-09-16T08:57:16.040 INFO:tasks.workunit.client.0.smithi031.stdout:1/257: rename d2 to d2/d15/d32/d58 22 2024-09-16T08:57:16.040 INFO:tasks.workunit.client.0.smithi031.stdout:1/258: chown d2/d4/f5 5 1 2024-09-16T08:57:16.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/791: getdents d5/d2c 0 2024-09-16T08:57:16.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/792: fdatasync d5/d14/d95/ff8 0 2024-09-16T08:57:16.046 INFO:tasks.workunit.client.1.smithi097.stdout:1/793: chown d5/dce/ce1 14 1 2024-09-16T08:57:16.047 INFO:tasks.workunit.client.0.smithi031.stdout:8/229: creat d4/d7/f51 x:0 0 0 2024-09-16T08:57:16.048 INFO:tasks.workunit.client.1.smithi097.stdout:4/682: creat d0/d7/df/d15/d1f/fef x:0 0 0 2024-09-16T08:57:16.048 INFO:tasks.workunit.client.1.smithi097.stdout:4/683: fdatasync d0/d7/d52/d26/d3d/f82 0 2024-09-16T08:57:16.052 INFO:tasks.workunit.client.0.smithi031.stdout:6/387: creat d0/dd/d74/d67/f7a x:0 0 0 2024-09-16T08:57:16.053 INFO:tasks.workunit.client.0.smithi031.stdout:6/388: creat d0/dd/d74/d5e/f7b x:0 0 0 2024-09-16T08:57:16.054 INFO:tasks.workunit.client.1.smithi097.stdout:1/794: rename d5/d17/d54/ca9 to d5/d17/d54/d6b/d71/cfc 0 2024-09-16T08:57:16.054 INFO:tasks.workunit.client.1.smithi097.stdout:1/795: chown d5/d17/d54/d6b/d71/d94/f99 37 1 2024-09-16T08:57:16.055 INFO:tasks.workunit.client.1.smithi097.stdout:4/684: mknod d0/d7/df/d15/cf0 0 2024-09-16T08:57:16.055 INFO:tasks.workunit.client.0.smithi031.stdout:9/281: rmdir d3/d5/da/db/d21/d3b 39 2024-09-16T08:57:16.056 INFO:tasks.workunit.client.0.smithi031.stdout:9/282: stat d3/f5b 0 2024-09-16T08:57:16.056 INFO:tasks.workunit.client.1.smithi097.stdout:5/781: getdents db/d18/d1c/d42/d46/d7e/d76/dc4 0 2024-09-16T08:57:16.057 INFO:tasks.workunit.client.1.smithi097.stdout:1/796: rmdir d5/d70/d85 39 2024-09-16T08:57:16.057 INFO:tasks.workunit.client.1.smithi097.stdout:1/797: write d5/d2c/d3b/d7f/fdb [269805,89070] 0 2024-09-16T08:57:16.057 INFO:tasks.workunit.client.1.smithi097.stdout:1/798: fdatasync d5/f9 0 2024-09-16T08:57:16.058 INFO:tasks.workunit.client.0.smithi031.stdout:0/268: truncate d6/d12/d13/f3e 1285672 0 2024-09-16T08:57:16.060 INFO:tasks.workunit.client.1.smithi097.stdout:3/725: sync 2024-09-16T08:57:16.060 INFO:tasks.workunit.client.1.smithi097.stdout:8/676: sync 2024-09-16T08:57:16.061 INFO:tasks.workunit.client.1.smithi097.stdout:7/631: sync 2024-09-16T08:57:16.061 INFO:tasks.workunit.client.1.smithi097.stdout:8/677: fdatasync d8/dd/d15/d2c/d4a/f87 0 2024-09-16T08:57:16.061 INFO:tasks.workunit.client.1.smithi097.stdout:3/726: chown d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/f91 376000235 1 2024-09-16T08:57:16.061 INFO:tasks.workunit.client.1.smithi097.stdout:1/799: symlink d5/d14/d95/lfd 0 2024-09-16T08:57:16.063 INFO:tasks.workunit.client.0.smithi031.stdout:8/230: unlink d4/f19 0 2024-09-16T08:57:16.063 INFO:tasks.workunit.client.0.smithi031.stdout:8/231: write d4/f6 [9202978,70137] 0 2024-09-16T08:57:16.063 INFO:tasks.workunit.client.0.smithi031.stdout:8/232: truncate d4/d7/f51 665494 0 2024-09-16T08:57:16.063 INFO:tasks.workunit.client.0.smithi031.stdout:8/233: stat d4/d7/d21/d37/d46 0 2024-09-16T08:57:16.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/234: chown d4/l1d 269352908 1 2024-09-16T08:57:16.064 INFO:tasks.workunit.client.1.smithi097.stdout:4/685: mknod d0/d7/d52/cf1 0 2024-09-16T08:57:16.064 INFO:tasks.workunit.client.1.smithi097.stdout:4/686: read - d0/d7/d52/dbf/fc6 zero size 2024-09-16T08:57:16.067 INFO:tasks.workunit.client.1.smithi097.stdout:3/727: link d7/de/d8d/d90/d92/cab d7/de/d9c/daf/cfb 0 2024-09-16T08:57:16.068 INFO:tasks.workunit.client.0.smithi031.stdout:6/389: truncate d0/dd/d74/f1e 6644082 0 2024-09-16T08:57:16.071 INFO:tasks.workunit.client.1.smithi097.stdout:1/800: creat d5/d2c/ffe x:0 0 0 2024-09-16T08:57:16.071 INFO:tasks.workunit.client.0.smithi031.stdout:9/283: symlink d3/d5/da/d34/d3c/d42/l64 0 2024-09-16T08:57:16.072 INFO:tasks.workunit.client.1.smithi097.stdout:8/678: dread d8/dd/d15/f29 [0,4194304] 0 2024-09-16T08:57:16.073 INFO:tasks.workunit.client.0.smithi031.stdout:8/235: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:16.073 INFO:tasks.workunit.client.1.smithi097.stdout:4/687: symlink d0/d36/lf2 0 2024-09-16T08:57:16.074 INFO:tasks.workunit.client.0.smithi031.stdout:9/284: mkdir d3/d5/da/d34/d3c/d5e/d62/d65 0 2024-09-16T08:57:16.075 INFO:tasks.workunit.client.0.smithi031.stdout:7/365: dwrite d2/f48 [0,4194304] 0 2024-09-16T08:57:16.076 INFO:tasks.workunit.client.1.smithi097.stdout:7/632: write d4/d7/d77/f9e [3148841,111701] 0 2024-09-16T08:57:16.076 INFO:tasks.workunit.client.1.smithi097.stdout:1/801: rename d5/d17/d54/d6b/d71/d94/cbc to d5/d17/cff 0 2024-09-16T08:57:16.079 INFO:tasks.workunit.client.0.smithi031.stdout:3/397: dwrite d3/d14/d37/d1a/d1d/f34 [4194304,4194304] 0 2024-09-16T08:57:16.079 INFO:tasks.workunit.client.0.smithi031.stdout:3/398: truncate d3/d14/d37/d33/f6d 853151 0 2024-09-16T08:57:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/228: dwrite d0/f38 [0,4194304] 0 2024-09-16T08:57:16.080 INFO:tasks.workunit.client.0.smithi031.stdout:0/269: dwrite d6/d12/d47/f4e [0,4194304] 0 2024-09-16T08:57:16.080 INFO:tasks.workunit.client.1.smithi097.stdout:9/665: dwrite d2/d5/db0/dc8/f32 [0,4194304] 0 2024-09-16T08:57:16.080 INFO:tasks.workunit.client.1.smithi097.stdout:9/666: write d2/d5/f11 [4844883,87039] 0 2024-09-16T08:57:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:8/236: rename d4/d7/d21/d24/f2a to d4/d2f/f52 0 2024-09-16T08:57:16.081 INFO:tasks.workunit.client.0.smithi031.stdout:4/229: dread d0/fa [0,4194304] 0 2024-09-16T08:57:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/230: chown d0/d2/d16/f30 100 1 2024-09-16T08:57:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/231: chown d0/c15 0 1 2024-09-16T08:57:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/232: stat d0/d2/f10 0 2024-09-16T08:57:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/233: fsync d0/d2/f1a 0 2024-09-16T08:57:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/234: write d0/d2/d16/d1e/f34 [595050,31587] 0 2024-09-16T08:57:16.082 INFO:tasks.workunit.client.0.smithi031.stdout:4/235: chown d0/d2/d16/f21 141394159 1 2024-09-16T08:57:16.082 INFO:tasks.workunit.client.1.smithi097.stdout:8/679: mknod d8/dd/d15/d2c/d4a/cd0 0 2024-09-16T08:57:16.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/237: read d4/f29 [210057,86167] 0 2024-09-16T08:57:16.088 INFO:tasks.workunit.client.1.smithi097.stdout:4/688: unlink d0/d23/d5f/d7d/dbc/cd2 0 2024-09-16T08:57:16.092 INFO:tasks.workunit.client.1.smithi097.stdout:3/728: rmdir d7/d28/d97 39 2024-09-16T08:57:16.094 INFO:tasks.workunit.client.0.smithi031.stdout:7/366: symlink d2/d8/d11/d5d/d6e/l80 0 2024-09-16T08:57:16.097 INFO:tasks.workunit.client.1.smithi097.stdout:7/633: symlink d4/d7/d7b/d43/ld6 0 2024-09-16T08:57:16.099 INFO:tasks.workunit.client.1.smithi097.stdout:1/802: mknod d5/d17/d54/d6b/da8/dc5/dc7/df3/c100 0 2024-09-16T08:57:16.099 INFO:tasks.workunit.client.1.smithi097.stdout:1/803: read d5/db/dc/fad [128125,19644] 0 2024-09-16T08:57:16.107 INFO:tasks.workunit.client.0.smithi031.stdout:3/399: creat d3/d14/d37/d1a/d1d/f75 x:0 0 0 2024-09-16T08:57:16.109 INFO:tasks.workunit.client.1.smithi097.stdout:9/667: rmdir d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79 39 2024-09-16T08:57:16.110 INFO:tasks.workunit.client.1.smithi097.stdout:8/680: creat d8/dd/d15/d2c/d4a/d4e/db2/fd1 x:0 0 0 2024-09-16T08:57:16.111 INFO:tasks.workunit.client.0.smithi031.stdout:9/285: rename d3/d5/da/d34/d3c/d42/f53 to d3/d5/da/d34/d4d/f66 0 2024-09-16T08:57:16.112 INFO:tasks.workunit.client.0.smithi031.stdout:4/236: dwrite d0/d2/f13 [0,4194304] 0 2024-09-16T08:57:16.113 INFO:tasks.workunit.client.1.smithi097.stdout:4/689: link d0/d23/d5f/c49 d0/d23/d5f/cf3 0 2024-09-16T08:57:16.113 INFO:tasks.workunit.client.1.smithi097.stdout:4/690: chown d0/f64 9521325 1 2024-09-16T08:57:16.113 INFO:tasks.workunit.client.1.smithi097.stdout:4/691: creat d0/d7/ff4 x:0 0 0 2024-09-16T08:57:16.113 INFO:tasks.workunit.client.1.smithi097.stdout:4/692: creat d0/d7/df/d83/ff5 x:0 0 0 2024-09-16T08:57:16.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/270: dwrite d6/fc [0,4194304] 0 2024-09-16T08:57:16.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/271: fsync d6/f8 0 2024-09-16T08:57:16.115 INFO:tasks.workunit.client.0.smithi031.stdout:0/272: write d6/f4f [1713390,68492] 0 2024-09-16T08:57:16.116 INFO:tasks.workunit.client.0.smithi031.stdout:8/238: stat d4/d34/c42 0 2024-09-16T08:57:16.117 INFO:tasks.workunit.client.1.smithi097.stdout:3/729: creat d7/de/d3c/d3f/d60/dcb/d35/d3d/ffc x:0 0 0 2024-09-16T08:57:16.120 INFO:tasks.workunit.client.0.smithi031.stdout:7/367: mkdir d2/d8/d11/d81 0 2024-09-16T08:57:16.120 INFO:tasks.workunit.client.0.smithi031.stdout:7/368: write d2/d8/f32 [838828,30514] 0 2024-09-16T08:57:16.121 INFO:tasks.workunit.client.1.smithi097.stdout:7/634: link d4/d7/dab/cb9 d4/cd7 0 2024-09-16T08:57:16.121 INFO:tasks.workunit.client.1.smithi097.stdout:7/635: fsync d4/d9/dc/d8e/f92 0 2024-09-16T08:57:16.122 INFO:tasks.workunit.client.0.smithi031.stdout:3/400: rmdir d3/d14/d37/d1a/d24/d63 0 2024-09-16T08:57:16.125 INFO:tasks.workunit.client.1.smithi097.stdout:1/804: symlink d5/l101 0 2024-09-16T08:57:16.126 INFO:tasks.workunit.client.0.smithi031.stdout:9/286: mkdir d3/d5/da/db/d21/d67 0 2024-09-16T08:57:16.129 INFO:tasks.workunit.client.1.smithi097.stdout:9/668: rename d2/d5/db0/dc8/d63/f77 to d2/d5/d8d/db1/fd7 0 2024-09-16T08:57:16.131 INFO:tasks.workunit.client.0.smithi031.stdout:9/287: read d3/d5/da/db/fe [80214,118851] 0 2024-09-16T08:57:16.131 INFO:tasks.workunit.client.1.smithi097.stdout:8/681: rmdir d8/dd/d15/d2c/d48 39 2024-09-16T08:57:16.133 INFO:tasks.workunit.client.0.smithi031.stdout:9/288: read d3/d5/da/f2c [227033,23840] 0 2024-09-16T08:57:16.133 INFO:tasks.workunit.client.0.smithi031.stdout:9/289: write d3/d5/da/db/fe [3939745,75412] 0 2024-09-16T08:57:16.133 INFO:tasks.workunit.client.0.smithi031.stdout:8/239: dread d4/f8 [0,4194304] 0 2024-09-16T08:57:16.133 INFO:tasks.workunit.client.0.smithi031.stdout:8/240: write d4/d7/d21/d35/d40/f4c [706362,16629] 0 2024-09-16T08:57:16.133 INFO:tasks.workunit.client.0.smithi031.stdout:8/241: dread - d4/f43 zero size 2024-09-16T08:57:16.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/369: mkdir d2/d8/d69/d82 0 2024-09-16T08:57:16.133 INFO:tasks.workunit.client.0.smithi031.stdout:7/370: stat d2/c70 0 2024-09-16T08:57:16.135 INFO:tasks.workunit.client.1.smithi097.stdout:7/636: write d4/d7/d7b/d43/f44 [8203,1452] 0 2024-09-16T08:57:16.138 INFO:tasks.workunit.client.0.smithi031.stdout:3/401: unlink d3/d14/d37/d1a/d1d/d1e/l3e 0 2024-09-16T08:57:16.138 INFO:tasks.workunit.client.0.smithi031.stdout:3/402: fsync d3/d14/d37/d1a/d1d/d1e/d67/f6b 0 2024-09-16T08:57:16.139 INFO:tasks.workunit.client.0.smithi031.stdout:4/237: dwrite d0/f2d [0,4194304] 0 2024-09-16T08:57:16.139 INFO:tasks.workunit.client.0.smithi031.stdout:0/273: dwrite d6/d12/d47/f55 [0,4194304] 0 2024-09-16T08:57:16.144 INFO:tasks.workunit.client.1.smithi097.stdout:4/693: creat d0/d23/ddc/ff6 x:0 0 0 2024-09-16T08:57:16.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.149+0000 7f66b291b640 1 -- 172.21.15.31:0/1380228104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 msgr2=0x7f66ac107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:16.151 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.149+0000 7f66b291b640 1 --2- 172.21.15.31:0/1380228104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac107930 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6694009880 tx=0x7f669403b4e0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.152 INFO:tasks.workunit.client.1.smithi097.stdout:1/805: symlink d5/d70/d85/l102 0 2024-09-16T08:57:16.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:15 smithi031 ceph-mon[27844]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:16.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:15 smithi031 ceph-mon[27844]: pgmap v9: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 224 MiB/s rd, 295 MiB/s wr, 494 op/s 2024-09-16T08:57:16.152 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:15 smithi031 ceph-mon[27844]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.150+0000 7f66b291b640 1 -- 172.21.15.31:0/1380228104 shutdown_connections 2024-09-16T08:57:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.150+0000 7f66b291b640 1 --2- 172.21.15.31:0/1380228104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac107930 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.150+0000 7f66b291b640 1 --2- 172.21.15.31:0/1380228104 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac071430 0x7f66ac071830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.150+0000 7f66b291b640 1 -- 172.21.15.31:0/1380228104 >> 172.21.15.31:0/1380228104 conn(0x7f66ac06cc10 msgr2=0x7f66ac06f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:16.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.151+0000 7f66b291b640 1 -- 172.21.15.31:0/1380228104 shutdown_connections 2024-09-16T08:57:16.153 INFO:tasks.workunit.client.0.smithi031.stdout:7/371: dread d2/fb [0,4194304] 0 2024-09-16T08:57:16.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.154+0000 7f66b291b640 1 -- 172.21.15.31:0/1380228104 wait complete. 2024-09-16T08:57:16.157 INFO:tasks.workunit.client.1.smithi097.stdout:9/669: mkdir d2/d5/d56/d6b/dcd/dd8 0 2024-09-16T08:57:16.157 INFO:tasks.workunit.client.1.smithi097.stdout:9/670: read - d2/d5/d3a/d70/da6/fcb zero size 2024-09-16T08:57:16.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.155+0000 7f66b291b640 1 Processor -- start 2024-09-16T08:57:16.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.155+0000 7f66b291b640 1 -- start start 2024-09-16T08:57:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.155+0000 7f66b291b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac071430 0x7f66ac19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.155+0000 7f66b291b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.155+0000 7f66b291b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac19c800 con 0x7f66ac1074b0 2024-09-16T08:57:16.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.155+0000 7f66b291b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac19c970 con 0x7f66ac071430 2024-09-16T08:57:16.158 INFO:tasks.workunit.client.0.smithi031.stdout:3/403: dread d3/d14/d6f/f73 [4194304,4194304] 0 2024-09-16T08:57:16.158 INFO:tasks.workunit.client.0.smithi031.stdout:3/404: fsync d3/d14/d37/d33/f6d 0 2024-09-16T08:57:16.160 INFO:tasks.workunit.client.1.smithi097.stdout:8/682: rename d8/dd/d15/d2c/l60 to d8/dd/d15/d2c/d4a/d4d/d3d/db9/ld2 0 2024-09-16T08:57:16.160 INFO:tasks.workunit.client.1.smithi097.stdout:3/730: dwrite d7/de/d3c/d3f/d60/dcb/d35/d3d/f59 [0,4194304] 0 2024-09-16T08:57:16.160 INFO:tasks.workunit.client.1.smithi097.stdout:8/683: truncate d8/dd/d15/d2c/d48/d56/f6c 4916195 0 2024-09-16T08:57:16.160 INFO:tasks.workunit.client.1.smithi097.stdout:8/684: readlink d8/dd/d15/d2c/d4a/d4d/d62/l64 0 2024-09-16T08:57:16.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.158+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:16.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.158+0000 7f66a37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac19c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34846/0 (socket says 172.21.15.31:34846) 2024-09-16T08:57:16.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.158+0000 7f66a37fe640 1 -- 172.21.15.31:0/3537088789 learned_addr learned my addr 172.21.15.31:0/3537088789 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:16.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.160+0000 7f66a37fe640 1 -- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac071430 msgr2=0x7f66ac19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:16.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.160+0000 7f66a37fe640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac071430 0x7f66ac19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.160+0000 7f66a37fe640 1 -- 172.21.15.31:0/3537088789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6694009560 con 0x7f66ac1074b0 2024-09-16T08:57:16.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.160+0000 7f66a37fe640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac19c230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6694009850 tx=0x7f6694005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:16.162 INFO:tasks.workunit.client.1.smithi097.stdout:9/671: fdatasync d2/d5/f11 0 2024-09-16T08:57:16.163 INFO:tasks.workunit.client.0.smithi031.stdout:6/390: write d0/dd/d74/f1e [6312619,54995] 0 2024-09-16T08:57:16.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.163+0000 7f66a9ffb640 1 -- 172.21.15.31:0/3537088789 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6694004be0 con 0x7f66ac1074b0 2024-09-16T08:57:16.165 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.163+0000 7f66a9ffb640 1 -- 172.21.15.31:0/3537088789 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6694004d40 con 0x7f66ac1074b0 2024-09-16T08:57:16.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.163+0000 7f66a9ffb640 1 -- 172.21.15.31:0/3537088789 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6694003b50 con 0x7f66ac1074b0 2024-09-16T08:57:16.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.163+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac1a13a0 con 0x7f66ac1074b0 2024-09-16T08:57:16.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.164+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac1a1910 con 0x7f66ac1074b0 2024-09-16T08:57:16.167 INFO:tasks.workunit.client.0.smithi031.stdout:6/391: dread d0/dd/d2b/f73 [0,4194304] 0 2024-09-16T08:57:16.168 INFO:tasks.workunit.client.0.smithi031.stdout:0/274: creat d6/d12/d13/f5d x:0 0 0 2024-09-16T08:57:16.168 INFO:tasks.workunit.client.0.smithi031.stdout:0/275: dread - d6/f31 zero size 2024-09-16T08:57:16.169 INFO:tasks.workunit.client.0.smithi031.stdout:0/276: read - d6/d12/d13/f21 zero size 2024-09-16T08:57:16.169 INFO:tasks.workunit.client.0.smithi031.stdout:0/277: stat d6/fa 0 2024-09-16T08:57:16.169 INFO:tasks.workunit.client.0.smithi031.stdout:0/278: fsync d6/d12/f37 0 2024-09-16T08:57:16.169 INFO:tasks.workunit.client.1.smithi097.stdout:1/806: mkdir d5/d103 0 2024-09-16T08:57:16.169 INFO:tasks.workunit.client.1.smithi097.stdout:1/807: chown d5/f35 27 1 2024-09-16T08:57:16.169 INFO:tasks.workunit.client.1.smithi097.stdout:1/808: fsync d5/d2c/d3b/f61 0 2024-09-16T08:57:16.169 INFO:tasks.workunit.client.1.smithi097.stdout:8/685: read d8/f10 [1845518,78603] 0 2024-09-16T08:57:16.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.166+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6690005280 con 0x7f66ac1074b0 2024-09-16T08:57:16.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.167+0000 7f66a9ffb640 1 -- 172.21.15.31:0/3537088789 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f6694003cb0 con 0x7f66ac1074b0 2024-09-16T08:57:16.170 INFO:tasks.workunit.client.0.smithi031.stdout:7/372: mknod d2/d2d/d6d/c83 0 2024-09-16T08:57:16.170 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.168+0000 7f66a9ffb640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f668c073dd0 0x7f668c076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:16.170 INFO:tasks.workunit.client.0.smithi031.stdout:7/373: write d2/d8/fd [659299,69998] 0 2024-09-16T08:57:16.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.168+0000 7f66a9ffb640 1 -- 172.21.15.31:0/3537088789 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f66940c72d0 con 0x7f66ac1074b0 2024-09-16T08:57:16.171 INFO:tasks.workunit.client.0.smithi031.stdout:8/242: dwrite d4/d7/d21/f23 [0,4194304] 0 2024-09-16T08:57:16.171 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.169+0000 7f66abfff640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f668c073dd0 0x7f668c076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:16.174 INFO:tasks.workunit.client.1.smithi097.stdout:7/637: dwrite d4/d9/d1c/f2d [4194304,4194304] 0 2024-09-16T08:57:16.174 INFO:tasks.workunit.client.1.smithi097.stdout:7/638: fdatasync d4/d9/d2b/fa9 0 2024-09-16T08:57:16.174 INFO:tasks.workunit.client.0.smithi031.stdout:3/405: creat d3/d14/d37/d1a/f76 x:0 0 0 2024-09-16T08:57:16.175 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.174+0000 7f66abfff640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f668c073dd0 0x7f668c076290 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66ac19d1d0 tx=0x7f669c006c40 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:16.177 INFO:tasks.workunit.client.1.smithi097.stdout:3/731: mkdir d7/de/d3c/d4f/dfd 0 2024-09-16T08:57:16.177 INFO:tasks.workunit.client.1.smithi097.stdout:3/732: chown d7/d28/d63/dd8 1301 1 2024-09-16T08:57:16.178 INFO:tasks.workunit.client.1.smithi097.stdout:3/733: truncate d7/d28/d30/d88/fa5 1351607 0 2024-09-16T08:57:16.179 INFO:tasks.workunit.client.0.smithi031.stdout:9/290: dwrite d3/d5/da/d34/d4d/f66 [0,4194304] 0 2024-09-16T08:57:16.179 INFO:tasks.workunit.client.0.smithi031.stdout:9/291: creat d3/d48/f68 x:0 0 0 2024-09-16T08:57:16.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.178+0000 7f66a9ffb640 1 -- 172.21.15.31:0/3537088789 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f669408a8f0 con 0x7f66ac1074b0 2024-09-16T08:57:16.180 INFO:tasks.workunit.client.0.smithi031.stdout:0/279: creat d6/d12/d47/d5b/f5e x:0 0 0 2024-09-16T08:57:16.181 INFO:tasks.workunit.client.0.smithi031.stdout:4/238: dwrite d0/d2/f10 [0,4194304] 0 2024-09-16T08:57:16.188 INFO:tasks.workunit.client.0.smithi031.stdout:8/243: symlink d4/d7/d36/l53 0 2024-09-16T08:57:16.189 INFO:tasks.workunit.client.1.smithi097.stdout:2/750: sync 2024-09-16T08:57:16.189 INFO:tasks.workunit.client.1.smithi097.stdout:0/723: sync 2024-09-16T08:57:16.189 INFO:tasks.workunit.client.1.smithi097.stdout:0/724: fsync da/d7e/dc4/dfc/d47/db3/f7b 0 2024-09-16T08:57:16.189 INFO:tasks.workunit.client.1.smithi097.stdout:6/738: sync 2024-09-16T08:57:16.191 INFO:tasks.workunit.client.1.smithi097.stdout:9/672: creat d2/d5/d3a/d70/da6/fd9 x:0 0 0 2024-09-16T08:57:16.191 INFO:tasks.workunit.client.0.smithi031.stdout:3/406: getdents d3/d14/d37/d1a/d1d/d6a 0 2024-09-16T08:57:16.192 INFO:tasks.workunit.client.1.smithi097.stdout:0/725: dread da/d7e/dc4/dfc/d94/d101/f66 [0,4194304] 0 2024-09-16T08:57:16.192 INFO:tasks.workunit.client.1.smithi097.stdout:0/726: truncate da/d7e/dc4/dfc/d47/fee 594929 0 2024-09-16T08:57:16.194 INFO:tasks.workunit.client.1.smithi097.stdout:4/694: dwrite d0/d39/f6f [4194304,4194304] 0 2024-09-16T08:57:16.198 INFO:tasks.workunit.client.1.smithi097.stdout:7/639: creat d4/d9/d51/d8b/fd8 x:0 0 0 2024-09-16T08:57:16.199 INFO:tasks.workunit.client.0.smithi031.stdout:6/392: rename d0/dd/d35/d59 to d0/dd/d74/d5e/d7c 0 2024-09-16T08:57:16.199 INFO:tasks.workunit.client.0.smithi031.stdout:6/393: write d0/dd/d74/f1e [6179165,10271] 0 2024-09-16T08:57:16.201 INFO:tasks.workunit.client.1.smithi097.stdout:3/734: creat d7/d28/d30/d69/ffe x:0 0 0 2024-09-16T08:57:16.204 INFO:tasks.workunit.client.1.smithi097.stdout:3/735: write d7/de/d3c/d3f/d60/dcb/d35/d3d/fb7 [1159772,68274] 0 2024-09-16T08:57:16.205 INFO:tasks.workunit.client.0.smithi031.stdout:9/292: dread d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:16.209 INFO:tasks.workunit.client.0.smithi031.stdout:3/407: dread d3/d14/d6f/f73 [0,4194304] 0 2024-09-16T08:57:16.210 INFO:tasks.workunit.client.0.smithi031.stdout:7/374: dwrite d2/d8/f2c [0,4194304] 0 2024-09-16T08:57:16.212 INFO:tasks.workunit.client.1.smithi097.stdout:2/751: rmdir d3/d4/d68 39 2024-09-16T08:57:16.216 INFO:tasks.workunit.client.1.smithi097.stdout:9/673: creat d2/d5/d3a/d3c/d73/fda x:0 0 0 2024-09-16T08:57:16.217 INFO:tasks.workunit.client.0.smithi031.stdout:4/239: dwrite d0/d2/d16/f35 [0,4194304] 0 2024-09-16T08:57:16.217 INFO:tasks.workunit.client.0.smithi031.stdout:4/240: write d0/d2/d16/f21 [3833,115956] 0 2024-09-16T08:57:16.217 INFO:tasks.workunit.client.0.smithi031.stdout:4/241: readlink d0/d2/l47 0 2024-09-16T08:57:16.218 INFO:tasks.workunit.client.0.smithi031.stdout:0/280: unlink d6/d12/d13/l44 0 2024-09-16T08:57:16.218 INFO:tasks.workunit.client.1.smithi097.stdout:8/686: dwrite d8/fa [4194304,4194304] 0 2024-09-16T08:57:16.218 INFO:tasks.workunit.client.1.smithi097.stdout:3/736: dread d7/de/d3c/dcc/ff3 [0,4194304] 0 2024-09-16T08:57:16.221 INFO:tasks.workunit.client.1.smithi097.stdout:0/727: dwrite da/d7e/dc4/dfc/d47/db3/fd4 [0,4194304] 0 2024-09-16T08:57:16.221 INFO:tasks.workunit.client.1.smithi097.stdout:6/739: dwrite d1/dc/f7d [0,4194304] 0 2024-09-16T08:57:16.221 INFO:tasks.workunit.client.1.smithi097.stdout:6/740: chown d1/dd3/d9e/dca/d81/f88 21 1 2024-09-16T08:57:16.222 INFO:tasks.workunit.client.1.smithi097.stdout:4/695: creat d0/d23/d5f/d7d/dbc/dcc/ff7 x:0 0 0 2024-09-16T08:57:16.224 INFO:tasks.workunit.client.1.smithi097.stdout:7/640: link d4/d9/d1c/fba d4/d1b/d41/fd9 0 2024-09-16T08:57:16.227 INFO:tasks.workunit.client.1.smithi097.stdout:2/752: link d3/d1b/f83 d3/d4/d68/d8b/d7c/d7d/d53/fdc 0 2024-09-16T08:57:16.235 INFO:tasks.workunit.client.0.smithi031.stdout:7/375: dwrite d2/d8/d11/f74 [4194304,4194304] 0 2024-09-16T08:57:16.235 INFO:tasks.workunit.client.0.smithi031.stdout:7/376: write d2/d8/fd [1008194,1031] 0 2024-09-16T08:57:16.235 INFO:tasks.workunit.client.1.smithi097.stdout:3/737: unlink d7/de/d3c/d3f/d60/dcb/d35/d3d/fb7 0 2024-09-16T08:57:16.243 INFO:tasks.workunit.client.1.smithi097.stdout:7/641: write d4/d7/d30/f5b [348084,52047] 0 2024-09-16T08:57:16.244 INFO:tasks.workunit.client.1.smithi097.stdout:6/741: creat d1/dd3/d9e/dca/d16/d4f/ff4 x:0 0 0 2024-09-16T08:57:16.246 INFO:tasks.workunit.client.0.smithi031.stdout:8/244: rename d4/d34/c42 to d4/d7/d21/d35/d50/c54 0 2024-09-16T08:57:16.246 INFO:tasks.workunit.client.0.smithi031.stdout:8/245: readlink d4/d7/d36/l53 0 2024-09-16T08:57:16.246 INFO:tasks.workunit.client.0.smithi031.stdout:8/246: chown c3 8561673 1 2024-09-16T08:57:16.249 INFO:tasks.workunit.client.0.smithi031.stdout:6/394: getdents d0/dd 0 2024-09-16T08:57:16.252 INFO:tasks.workunit.client.1.smithi097.stdout:2/753: creat d3/d4/d8/fdd x:0 0 0 2024-09-16T08:57:16.252 INFO:tasks.workunit.client.1.smithi097.stdout:2/754: truncate d3/d4/d9/d78/f9e 2368009 0 2024-09-16T08:57:16.253 INFO:tasks.workunit.client.0.smithi031.stdout:9/293: mknod d3/d5/da/c69 0 2024-09-16T08:57:16.254 INFO:tasks.workunit.client.1.smithi097.stdout:9/674: getdents d2/d5/d3a 0 2024-09-16T08:57:16.254 INFO:tasks.workunit.client.1.smithi097.stdout:9/675: dread - d2/d5/db0/dc8/d74/d8c/d94/d96/dac/f87 zero size 2024-09-16T08:57:16.255 INFO:tasks.workunit.client.1.smithi097.stdout:7/642: dread d4/d9/d1c/f2c [0,4194304] 0 2024-09-16T08:57:16.256 INFO:tasks.workunit.client.0.smithi031.stdout:3/408: link d3/d14/d37/d1a/f26 d3/d14/d37/d1a/d24/f77 0 2024-09-16T08:57:16.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:15 smithi097 ceph-mon[35478]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:16.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:15 smithi097 ceph-mon[35478]: pgmap v9: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 224 MiB/s rd, 295 MiB/s wr, 494 op/s 2024-09-16T08:57:16.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:15 smithi097 ceph-mon[35478]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:16.263 INFO:tasks.workunit.client.0.smithi031.stdout:4/242: creat d0/d2/d3a/f49 x:0 0 0 2024-09-16T08:57:16.265 INFO:tasks.workunit.client.0.smithi031.stdout:0/281: mknod d6/d12/d47/d4d/c5f 0 2024-09-16T08:57:16.267 INFO:tasks.workunit.client.1.smithi097.stdout:8/687: rmdir d8/dd/d15/d2c/d4a/d4d/d62 39 2024-09-16T08:57:16.268 INFO:tasks.workunit.client.1.smithi097.stdout:8/688: chown d8/dd/d15/d16/d7c/da2/lc5 1 1 2024-09-16T08:57:16.268 INFO:tasks.workunit.client.0.smithi031.stdout:7/377: dwrite d2/f42 [0,4194304] 0 2024-09-16T08:57:16.268 INFO:tasks.workunit.client.0.smithi031.stdout:7/378: chown d2/d8/d11/d5d/f73 14 1 2024-09-16T08:57:16.270 INFO:tasks.workunit.client.0.smithi031.stdout:4/243: dread d0/f39 [0,4194304] 0 2024-09-16T08:57:16.271 INFO:tasks.workunit.client.1.smithi097.stdout:3/738: mknod d7/d15/d52/d81/cff 0 2024-09-16T08:57:16.271 INFO:tasks.workunit.client.1.smithi097.stdout:3/739: write d7/de/d3c/d4f/f7c [946109,12570] 0 2024-09-16T08:57:16.276 INFO:tasks.workunit.client.1.smithi097.stdout:6/742: rename d1/dd3/d9e/dca/d81/dad/d5a to d1/dd3/d9e/dca/d81/dad/df5 0 2024-09-16T08:57:16.280 INFO:tasks.workunit.client.0.smithi031.stdout:8/247: dwrite d4/d7/f1a [0,4194304] 0 2024-09-16T08:57:16.284 INFO:tasks.workunit.client.1.smithi097.stdout:2/755: unlink d3/d4/d9/d67/db2/lc2 0 2024-09-16T08:57:16.284 INFO:tasks.workunit.client.1.smithi097.stdout:2/756: write d3/d4/d9/d78/f9e [2933989,92495] 0 2024-09-16T08:57:16.284 INFO:tasks.workunit.client.1.smithi097.stdout:2/757: chown d3/d4/d68/cb3 3020360 1 2024-09-16T08:57:16.284 INFO:tasks.workunit.client.1.smithi097.stdout:2/758: dread - d3/d4/d68/d8b/d7c/d7d/fb5 zero size 2024-09-16T08:57:16.284 INFO:tasks.workunit.client.1.smithi097.stdout:6/743: write d1/dd3/d9e/dca/d16/f2e [385834,56052] 0 2024-09-16T08:57:16.285 INFO:tasks.workunit.client.1.smithi097.stdout:6/744: fsync d1/dc/fcb 0 2024-09-16T08:57:16.285 INFO:tasks.workunit.client.0.smithi031.stdout:4/244: dread d0/f38 [0,4194304] 0 2024-09-16T08:57:16.285 INFO:tasks.workunit.client.1.smithi097.stdout:9/676: mknod d2/d5/db0/dc8/d63/da8/dca/cdb 0 2024-09-16T08:57:16.285 INFO:tasks.workunit.client.1.smithi097.stdout:9/677: dread - d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/fbe zero size 2024-09-16T08:57:16.286 INFO:tasks.workunit.client.0.smithi031.stdout:9/294: getdents d3/d5/da/d34/d3c/d42 0 2024-09-16T08:57:16.286 INFO:tasks.workunit.client.0.smithi031.stdout:3/409: dwrite d3/d14/d37/d1a/d1d/d3b/f43 [0,4194304] 0 2024-09-16T08:57:16.288 INFO:tasks.workunit.client.1.smithi097.stdout:7/643: link d4/d7/fbd d4/d9/fda 0 2024-09-16T08:57:16.289 INFO:tasks.workunit.client.0.smithi031.stdout:0/282: mknod d6/d12/d47/c60 0 2024-09-16T08:57:16.291 INFO:tasks.workunit.client.1.smithi097.stdout:4/696: rename d0/d23/d5f/d7d to d0/d7/df/d83/df8 0 2024-09-16T08:57:16.293 INFO:tasks.workunit.client.1.smithi097.stdout:2/759: mknod d3/d4/d68/cde 0 2024-09-16T08:57:16.293 INFO:tasks.workunit.client.1.smithi097.stdout:2/760: dread - d3/d4/d68/d8b/fda zero size 2024-09-16T08:57:16.297 INFO:tasks.workunit.client.1.smithi097.stdout:9/678: creat d2/d5/db0/dc8/fdc x:0 0 0 2024-09-16T08:57:16.297 INFO:tasks.workunit.client.1.smithi097.stdout:9/679: chown d2/d5/db0/dc8/d74/d8c/d94/ld1 1745669 1 2024-09-16T08:57:16.300 INFO:tasks.workunit.client.1.smithi097.stdout:6/745: rename d1/dd3/d9e/dca/d16/lc5 to d1/dd3/d9e/lf6 0 2024-09-16T08:57:16.301 INFO:tasks.workunit.client.1.smithi097.stdout:6/746: fdatasync d1/f22 0 2024-09-16T08:57:16.301 INFO:tasks.workunit.client.1.smithi097.stdout:6/747: fsync d1/dd3/d9e/dca/d16/d4f/f55 0 2024-09-16T08:57:16.302 INFO:tasks.workunit.client.0.smithi031.stdout:6/395: getdents d0 0 2024-09-16T08:57:16.302 INFO:tasks.workunit.client.0.smithi031.stdout:6/396: fdatasync d0/dd/d35/f64 0 2024-09-16T08:57:16.305 INFO:tasks.workunit.client.1.smithi097.stdout:2/761: truncate d3/d4/d68/d8b/d7c/d7d/f6d 3850382 0 2024-09-16T08:57:16.305 INFO:tasks.workunit.client.1.smithi097.stdout:3/740: dwrite d7/de/d3c/f4d [4194304,4194304] 0 2024-09-16T08:57:16.305 INFO:tasks.workunit.client.1.smithi097.stdout:3/741: readlink d7/de/d3c/d3f/d60/d7a/l7e 0 2024-09-16T08:57:16.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/379: dwrite d2/f22 [4194304,4194304] 0 2024-09-16T08:57:16.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/380: readlink d2/d2d/l38 0 2024-09-16T08:57:16.306 INFO:tasks.workunit.client.1.smithi097.stdout:8/689: dwrite d8/f31 [0,4194304] 0 2024-09-16T08:57:16.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/248: dread d4/f8 [8388608,4194304] 0 2024-09-16T08:57:16.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/249: dread - d4/f43 zero size 2024-09-16T08:57:16.307 INFO:tasks.workunit.client.0.smithi031.stdout:4/245: symlink d0/d3c/l4a 0 2024-09-16T08:57:16.309 INFO:tasks.workunit.client.0.smithi031.stdout:9/295: mkdir d3/d5/d6a 0 2024-09-16T08:57:16.309 INFO:tasks.workunit.client.0.smithi031.stdout:9/296: truncate f2 1927049 0 2024-09-16T08:57:16.311 INFO:tasks.workunit.client.0.smithi031.stdout:3/410: dread d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 [0,4194304] 0 2024-09-16T08:57:16.314 INFO:tasks.workunit.client.0.smithi031.stdout:6/397: creat d0/dd/d74/d5e/d7c/d3f/f7d x:0 0 0 2024-09-16T08:57:16.314 INFO:tasks.workunit.client.0.smithi031.stdout:6/398: write d0/dd/f3e [700554,50059] 0 2024-09-16T08:57:16.318 INFO:tasks.workunit.client.1.smithi097.stdout:2/762: rename d3/d4/d8/f2b to d3/d1b/da7/fdf 0 2024-09-16T08:57:16.320 INFO:tasks.workunit.client.0.smithi031.stdout:5/359: sync 2024-09-16T08:57:16.320 INFO:tasks.workunit.client.0.smithi031.stdout:5/360: dread - d2/dc/d25/d21/f73 zero size 2024-09-16T08:57:16.320 INFO:tasks.workunit.client.0.smithi031.stdout:5/361: readlink d2/dc/l6f 0 2024-09-16T08:57:16.320 INFO:tasks.workunit.client.0.smithi031.stdout:3/411: write d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 [3734357,415] 0 2024-09-16T08:57:16.321 INFO:tasks.workunit.client.0.smithi031.stdout:2/232: sync 2024-09-16T08:57:16.323 INFO:tasks.workunit.client.0.smithi031.stdout:7/381: creat d2/d8/d11/d5d/d7e/f84 x:0 0 0 2024-09-16T08:57:16.323 INFO:tasks.workunit.client.0.smithi031.stdout:7/382: chown d2/d8/l10 0 1 2024-09-16T08:57:16.324 INFO:tasks.workunit.client.0.smithi031.stdout:7/383: fdatasync d2/d8/f63 0 2024-09-16T08:57:16.324 INFO:tasks.workunit.client.0.smithi031.stdout:7/384: write d2/d8/f2f [4411209,50556] 0 2024-09-16T08:57:16.324 INFO:tasks.workunit.client.1.smithi097.stdout:5/782: sync 2024-09-16T08:57:16.325 INFO:tasks.workunit.client.0.smithi031.stdout:6/399: read d0/dd/f2f [1996435,99384] 0 2024-09-16T08:57:16.327 INFO:tasks.workunit.client.0.smithi031.stdout:4/246: mknod d0/d2/d16/d1e/c4b 0 2024-09-16T08:57:16.327 INFO:tasks.workunit.client.0.smithi031.stdout:4/247: creat d0/f4c x:0 0 0 2024-09-16T08:57:16.329 INFO:tasks.workunit.client.0.smithi031.stdout:9/297: rmdir d3/d5/da/d34/d3c/d5e/d62 39 2024-09-16T08:57:16.329 INFO:tasks.workunit.client.0.smithi031.stdout:9/298: read - d3/d48/f4e zero size 2024-09-16T08:57:16.329 INFO:tasks.workunit.client.0.smithi031.stdout:9/299: creat d3/d5/da/f6b x:0 0 0 2024-09-16T08:57:16.329 INFO:tasks.workunit.client.0.smithi031.stdout:9/300: readlink d3/d5/da/d34/d3c/d42/l64 0 2024-09-16T08:57:16.332 INFO:tasks.workunit.client.0.smithi031.stdout:1/259: sync 2024-09-16T08:57:16.333 INFO:tasks.workunit.client.0.smithi031.stdout:3/412: write d3/f4 [3864234,102248] 0 2024-09-16T08:57:16.333 INFO:tasks.workunit.client.0.smithi031.stdout:3/413: readlink d3/d14/d37/d33/l4b 0 2024-09-16T08:57:16.337 INFO:tasks.workunit.client.1.smithi097.stdout:7/644: dwrite d4/d9/dc/f72 [0,4194304] 0 2024-09-16T08:57:16.337 INFO:tasks.workunit.client.1.smithi097.stdout:7/645: write d4/d1b/d41/fd9 [767037,44770] 0 2024-09-16T08:57:16.337 INFO:tasks.workunit.client.1.smithi097.stdout:7/646: creat d4/d9/d1c/d5f/fdb x:0 0 0 2024-09-16T08:57:16.337 INFO:tasks.workunit.client.1.smithi097.stdout:7/647: chown d4/d12/f4f 48 1 2024-09-16T08:57:16.337 INFO:tasks.workunit.client.0.smithi031.stdout:7/385: mknod d2/d8/d69/d82/c85 0 2024-09-16T08:57:16.338 INFO:tasks.workunit.client.1.smithi097.stdout:8/690: creat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/fd3 x:0 0 0 2024-09-16T08:57:16.340 INFO:tasks.workunit.client.1.smithi097.stdout:5/783: symlink db/d1a/d6d/l103 0 2024-09-16T08:57:16.340 INFO:tasks.workunit.client.1.smithi097.stdout:5/784: chown db/d18/d20/l3f 1262343 1 2024-09-16T08:57:16.340 INFO:tasks.workunit.client.1.smithi097.stdout:5/785: chown db/c1b 93063 1 2024-09-16T08:57:16.341 INFO:tasks.workunit.client.0.smithi031.stdout:6/400: creat d0/dd/d74/f7e x:0 0 0 2024-09-16T08:57:16.343 INFO:tasks.workunit.client.1.smithi097.stdout:7/648: dread d4/d9/d1c/f34 [0,4194304] 0 2024-09-16T08:57:16.347 INFO:tasks.workunit.client.0.smithi031.stdout:0/283: dwrite d6/d12/d13/f41 [0,4194304] 0 2024-09-16T08:57:16.348 INFO:tasks.workunit.client.1.smithi097.stdout:7/649: creat d4/d7/fdc x:0 0 0 2024-09-16T08:57:16.348 INFO:tasks.workunit.client.1.smithi097.stdout:7/650: fdatasync d4/d9/d2b/f2e 0 2024-09-16T08:57:16.353 INFO:tasks.workunit.client.0.smithi031.stdout:5/362: rmdir d2/dc/d3d 39 2024-09-16T08:57:16.353 INFO:tasks.workunit.client.0.smithi031.stdout:1/260: rename d2/d4/l3a to d2/d15/d32/l59 0 2024-09-16T08:57:16.353 INFO:tasks.workunit.client.0.smithi031.stdout:3/414: symlink d3/d14/d37/l78 0 2024-09-16T08:57:16.354 INFO:tasks.workunit.client.0.smithi031.stdout:7/386: mkdir d2/d8/d86 0 2024-09-16T08:57:16.354 INFO:tasks.workunit.client.0.smithi031.stdout:7/387: write d2/d8/fd [2580951,32984] 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:2/763: fdatasync d3/d4/d9/d78/f9e 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:9/680: dwrite d2/d5/d56/f8a [4194304,4194304] 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:3/742: dwrite d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/f91 [0,4194304] 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:4/697: dwrite d0/d7/d52/f88 [0,4194304] 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:7/651: mkdir d4/d7/d19/ddd 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:7/652: write d4/d9/dc/f6b [1010277,61873] 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:7/653: creat d4/d9/dc/d8e/fde x:0 0 0 2024-09-16T08:57:16.355 INFO:tasks.workunit.client.1.smithi097.stdout:2/764: symlink d3/d4/d8/le0 0 2024-09-16T08:57:16.356 INFO:tasks.workunit.client.1.smithi097.stdout:1/809: sync 2024-09-16T08:57:16.356 INFO:tasks.workunit.client.0.smithi031.stdout:6/401: creat d0/f7f x:0 0 0 2024-09-16T08:57:16.358 INFO:tasks.workunit.client.1.smithi097.stdout:3/743: mknod d7/d28/c100 0 2024-09-16T08:57:16.358 INFO:tasks.workunit.client.1.smithi097.stdout:3/744: truncate d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/fb4 1340625 0 2024-09-16T08:57:16.360 INFO:tasks.workunit.client.0.smithi031.stdout:2/233: dwrite d6/d20/f3f [0,4194304] 0 2024-09-16T08:57:16.362 INFO:tasks.workunit.client.0.smithi031.stdout:0/284: mkdir d6/d61 0 2024-09-16T08:57:16.362 INFO:tasks.workunit.client.0.smithi031.stdout:0/285: chown d6/d12/f37 1186 1 2024-09-16T08:57:16.362 INFO:tasks.workunit.client.0.smithi031.stdout:0/286: chown d6 6956167 1 2024-09-16T08:57:16.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/248: rmdir d0/d2/d3a 39 2024-09-16T08:57:16.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/249: truncate d0/d2/d16/f21 467810 0 2024-09-16T08:57:16.364 INFO:tasks.workunit.client.0.smithi031.stdout:9/301: dwrite d3/d48/f68 [0,4194304] 0 2024-09-16T08:57:16.364 INFO:tasks.workunit.client.0.smithi031.stdout:9/302: readlink d3/d5/da/db/d21/d3b/l57 0 2024-09-16T08:57:16.365 INFO:tasks.workunit.client.0.smithi031.stdout:5/363: mkdir d2/dc/d36/d76 0 2024-09-16T08:57:16.365 INFO:tasks.workunit.client.0.smithi031.stdout:5/364: chown d2/dc/d3d/l3e 4548181 1 2024-09-16T08:57:16.367 INFO:tasks.workunit.client.1.smithi097.stdout:7/654: rename d4/d9/d51/d88 to d4/d7/d30/ddf 0 2024-09-16T08:57:16.367 INFO:tasks.workunit.client.1.smithi097.stdout:5/786: dread db/d1a/f57 [0,4194304] 0 2024-09-16T08:57:16.369 INFO:tasks.workunit.client.0.smithi031.stdout:1/261: mknod d2/d15/c5a 0 2024-09-16T08:57:16.369 INFO:tasks.workunit.client.0.smithi031.stdout:1/262: stat d2/d15/d4a 0 2024-09-16T08:57:16.369 INFO:tasks.workunit.client.0.smithi031.stdout:1/263: write d2/d4/f12 [818639,3188] 0 2024-09-16T08:57:16.371 INFO:tasks.workunit.client.1.smithi097.stdout:2/765: creat d3/d1b/da7/fe1 x:0 0 0 2024-09-16T08:57:16.371 INFO:tasks.workunit.client.1.smithi097.stdout:2/766: write d3/d4/d68/d8b/d7c/d7d/f81 [569022,29448] 0 2024-09-16T08:57:16.371 INFO:tasks.workunit.client.0.smithi031.stdout:3/415: mkdir d3/d14/d37/d1a/d1d/d3b/d3f/d79 0 2024-09-16T08:57:16.371 INFO:tasks.workunit.client.0.smithi031.stdout:3/416: creat d3/d14/d37/d33/f7a x:0 0 0 2024-09-16T08:57:16.372 INFO:tasks.workunit.client.0.smithi031.stdout:3/417: write d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 [831274,10997] 0 2024-09-16T08:57:16.373 INFO:tasks.workunit.client.1.smithi097.stdout:1/810: link d5/db/dc/f41 d5/db/dc/d19/f104 0 2024-09-16T08:57:16.375 INFO:tasks.workunit.client.1.smithi097.stdout:8/691: dwrite d8/dd/f82 [0,4194304] 0 2024-09-16T08:57:16.375 INFO:tasks.workunit.client.1.smithi097.stdout:3/745: mkdir d7/d28/d63/d70/d101 0 2024-09-16T08:57:16.376 INFO:tasks.workunit.client.1.smithi097.stdout:9/681: dwrite d2/f46 [4194304,4194304] 0 2024-09-16T08:57:16.376 INFO:tasks.workunit.client.1.smithi097.stdout:3/746: fsync d7/de/d3c/d3f/d60/d6b/f86 0 2024-09-16T08:57:16.380 INFO:tasks.workunit.client.1.smithi097.stdout:5/787: write db/d18/d20/d2b/f6f [1499931,72373] 0 2024-09-16T08:57:16.384 INFO:tasks.workunit.client.0.smithi031.stdout:0/287: write d6/d12/f16 [3687188,94698] 0 2024-09-16T08:57:16.384 INFO:tasks.workunit.client.0.smithi031.stdout:4/250: read d0/f38 [1636347,15328] 0 2024-09-16T08:57:16.385 INFO:tasks.workunit.client.1.smithi097.stdout:1/811: dread d5/d2c/fc3 [0,4194304] 0 2024-09-16T08:57:16.385 INFO:tasks.workunit.client.1.smithi097.stdout:1/812: write d5/fb9 [258500,130734] 0 2024-09-16T08:57:16.387 INFO:tasks.workunit.client.1.smithi097.stdout:9/682: dread d2/d5/db0/dc8/f3b [0,4194304] 0 2024-09-16T08:57:16.387 INFO:tasks.workunit.client.0.smithi031.stdout:6/402: rmdir d0/dd/d74/d5e/d79 0 2024-09-16T08:57:16.396 INFO:tasks.workunit.client.1.smithi097.stdout:7/655: dwrite d4/d12/d1f/fb6 [0,4194304] 0 2024-09-16T08:57:16.396 INFO:tasks.workunit.client.0.smithi031.stdout:2/234: dwrite f1 [0,4194304] 0 2024-09-16T08:57:16.396 INFO:tasks.workunit.client.0.smithi031.stdout:2/235: dread - d6/d2e/f2f zero size 2024-09-16T08:57:16.396 INFO:tasks.workunit.client.0.smithi031.stdout:2/236: fdatasync f1 0 2024-09-16T08:57:16.397 INFO:tasks.workunit.client.1.smithi097.stdout:5/788: write db/d18/d20/d2b/f7c [91067,30944] 0 2024-09-16T08:57:16.397 INFO:tasks.workunit.client.1.smithi097.stdout:9/683: write d2/d5/d72/fab [604466,130045] 0 2024-09-16T08:57:16.397 INFO:tasks.workunit.client.1.smithi097.stdout:5/789: chown db/d18/d1c/d42/d46/d7e/d76/fc0 2 1 2024-09-16T08:57:16.401 INFO:tasks.workunit.client.1.smithi097.stdout:2/767: rename d3/d4/d68/d8b/d61/cc9 to d3/d4/d68/d8b/d61/dac/dd9/ce2 0 2024-09-16T08:57:16.403 INFO:tasks.workunit.client.0.smithi031.stdout:9/303: mknod d3/d5/c6c 0 2024-09-16T08:57:16.407 INFO:tasks.workunit.client.0.smithi031.stdout:5/365: symlink d2/dc/d36/d76/l77 0 2024-09-16T08:57:16.409 INFO:tasks.workunit.client.0.smithi031.stdout:1/264: creat d2/d15/d4a/f5b x:0 0 0 2024-09-16T08:57:16.410 INFO:tasks.workunit.client.1.smithi097.stdout:9/684: write d2/d5/db0/dc8/d74/d8c/d94/fa9 [618976,39032] 0 2024-09-16T08:57:16.415 INFO:tasks.workunit.client.0.smithi031.stdout:9/304: read d3/d5/f6 [3229533,98818] 0 2024-09-16T08:57:16.419 INFO:tasks.workunit.client.1.smithi097.stdout:0/728: sync 2024-09-16T08:57:16.420 INFO:tasks.workunit.client.1.smithi097.stdout:9/685: write d2/f31 [7926224,129831] 0 2024-09-16T08:57:16.421 INFO:tasks.workunit.client.0.smithi031.stdout:3/418: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d79/l7b 0 2024-09-16T08:57:16.421 INFO:tasks.workunit.client.0.smithi031.stdout:3/419: chown d3/d14/d37/d1a/d1d/d3b/d3f/l5d 233 1 2024-09-16T08:57:16.422 INFO:tasks.workunit.client.0.smithi031.stdout:7/388: rmdir d2/d8/d11/d5d/d6e 39 2024-09-16T08:57:16.423 INFO:tasks.workunit.client.1.smithi097.stdout:8/692: link d8/dd/d42/f51 d8/dd/d15/d2c/d4a/d4d/d3d/db9/fd4 0 2024-09-16T08:57:16.424 INFO:tasks.workunit.client.0.smithi031.stdout:0/288: creat d6/d61/f62 x:0 0 0 2024-09-16T08:57:16.425 INFO:tasks.workunit.client.1.smithi097.stdout:3/747: creat d7/de/d3c/d4f/dfd/f102 x:0 0 0 2024-09-16T08:57:16.426 INFO:tasks.workunit.client.1.smithi097.stdout:3/748: dread d7/de/d3c/f99 [0,4194304] 0 2024-09-16T08:57:16.426 INFO:tasks.workunit.client.0.smithi031.stdout:4/251: rename d0/d2/d16/c48 to d0/d2/d16/d2b/c4d 0 2024-09-16T08:57:16.427 INFO:tasks.workunit.client.0.smithi031.stdout:4/252: truncate d0/d2/d16/f30 748990 0 2024-09-16T08:57:16.427 INFO:tasks.workunit.client.0.smithi031.stdout:4/253: write d0/f4c [155942,43162] 0 2024-09-16T08:57:16.429 INFO:tasks.workunit.client.1.smithi097.stdout:1/813: creat d5/d17/d54/d6b/da8/dc5/dc7/df3/f105 x:0 0 0 2024-09-16T08:57:16.429 INFO:tasks.workunit.client.1.smithi097.stdout:1/814: write d5/db/dc/ff6 [3654257,44676] 0 2024-09-16T08:57:16.432 INFO:tasks.workunit.client.0.smithi031.stdout:2/237: rmdir d6/da/d37/d44 0 2024-09-16T08:57:16.433 INFO:tasks.workunit.client.0.smithi031.stdout:1/265: dwrite d2/de/f2f [0,4194304] 0 2024-09-16T08:57:16.436 INFO:tasks.workunit.client.0.smithi031.stdout:5/366: unlink d2/dc/d25/d21/f65 0 2024-09-16T08:57:16.436 INFO:tasks.workunit.client.0.smithi031.stdout:5/367: fdatasync d2/fa 0 2024-09-16T08:57:16.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/420: read f1 [4129728,97874] 0 2024-09-16T08:57:16.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/421: readlink d3/d14/d37/d1a/d1d/d3b/d3f/l46 0 2024-09-16T08:57:16.438 INFO:tasks.workunit.client.0.smithi031.stdout:3/422: chown d3/d14/d37/d1a/d24/c36 134063449 1 2024-09-16T08:57:16.441 INFO:tasks.workunit.client.0.smithi031.stdout:7/389: creat d2/d8/f87 x:0 0 0 2024-09-16T08:57:16.442 INFO:tasks.workunit.client.0.smithi031.stdout:7/390: dread d2/d8/d69/f6c [0,4194304] 0 2024-09-16T08:57:16.443 INFO:tasks.workunit.client.1.smithi097.stdout:7/656: unlink d4/d9/d51/cd5 0 2024-09-16T08:57:16.446 INFO:tasks.workunit.client.0.smithi031.stdout:1/266: dread d2/d4/d22/f45 [0,4194304] 0 2024-09-16T08:57:16.452 INFO:tasks.workunit.client.1.smithi097.stdout:2/768: creat d3/d4/d68/d8b/d7c/d7d/d53/da3/fe3 x:0 0 0 2024-09-16T08:57:16.454 INFO:tasks.workunit.client.0.smithi031.stdout:4/254: creat d0/d2/d16/f4e x:0 0 0 2024-09-16T08:57:16.456 INFO:tasks.workunit.client.1.smithi097.stdout:0/729: truncate da/d1a/d41/f50 165798 0 2024-09-16T08:57:16.456 INFO:tasks.workunit.client.1.smithi097.stdout:0/730: fdatasync da/d7e/dc4/dfc/d47/faf 0 2024-09-16T08:57:16.457 INFO:tasks.workunit.client.1.smithi097.stdout:0/731: truncate da/df/d8a/fda 876046 0 2024-09-16T08:57:16.457 INFO:tasks.workunit.client.1.smithi097.stdout:0/732: chown da/d7e/dc4/dfc/d47/db3/d60/d73/f77 10 1 2024-09-16T08:57:16.458 INFO:tasks.workunit.client.0.smithi031.stdout:0/289: rename d6/d12/d13/l54 to d6/d12/d47/d5b/l63 0 2024-09-16T08:57:16.462 INFO:tasks.workunit.client.1.smithi097.stdout:9/686: mkdir d2/d5/d72/dc1/ddd 0 2024-09-16T08:57:16.462 INFO:tasks.workunit.client.1.smithi097.stdout:9/687: dread - d2/d5/db0/dc8/d74/d8c/d94/d96/dac/f87 zero size 2024-09-16T08:57:16.464 INFO:tasks.workunit.client.1.smithi097.stdout:8/693: symlink d8/dd/d15/d2c/d4a/d4e/d85/ld5 0 2024-09-16T08:57:16.464 INFO:tasks.workunit.client.1.smithi097.stdout:8/694: creat d8/dd/d15/d2c/d4a/d4e/d85/fd6 x:0 0 0 2024-09-16T08:57:16.464 INFO:tasks.workunit.client.1.smithi097.stdout:3/749: mknod d7/d28/d30/c103 0 2024-09-16T08:57:16.464 INFO:tasks.workunit.client.1.smithi097.stdout:1/815: mknod d5/d17/df5/c106 0 2024-09-16T08:57:16.466 INFO:tasks.workunit.client.0.smithi031.stdout:7/391: dwrite d2/d8/f50 [0,4194304] 0 2024-09-16T08:57:16.466 INFO:tasks.workunit.client.0.smithi031.stdout:7/392: fdatasync d2/f48 0 2024-09-16T08:57:16.466 INFO:tasks.workunit.client.0.smithi031.stdout:7/393: fsync d2/d8/d11/f5c 0 2024-09-16T08:57:16.467 INFO:tasks.workunit.client.0.smithi031.stdout:6/403: rmdir d0 39 2024-09-16T08:57:16.467 INFO:tasks.workunit.client.0.smithi031.stdout:6/404: fdatasync d0/f7f 0 2024-09-16T08:57:16.468 INFO:tasks.workunit.client.0.smithi031.stdout:2/238: mkdir d6/da/d4b 0 2024-09-16T08:57:16.469 INFO:tasks.workunit.client.0.smithi031.stdout:5/368: truncate d2/dc/d25/f2f 4133239 0 2024-09-16T08:57:16.469 INFO:tasks.workunit.client.1.smithi097.stdout:2/769: symlink d3/d4/d9/d78/le4 0 2024-09-16T08:57:16.472 INFO:tasks.workunit.client.0.smithi031.stdout:3/423: unlink d3/d14/d37/d1a/d1d/d3b/d3f/l6c 0 2024-09-16T08:57:16.476 INFO:tasks.workunit.client.1.smithi097.stdout:9/688: rename d2/d5/db0/dc8/d74/d8c/d94/d96/cc2 to d2/d5/d72/dc1/cde 0 2024-09-16T08:57:16.479 INFO:tasks.workunit.client.0.smithi031.stdout:7/394: write d2/d8/d11/f31 [1411348,52784] 0 2024-09-16T08:57:16.479 INFO:tasks.workunit.client.1.smithi097.stdout:3/750: truncate d7/de/d6c/f66 1675339 0 2024-09-16T08:57:16.481 INFO:tasks.workunit.client.1.smithi097.stdout:1/816: mknod d5/d2c/dcb/c107 0 2024-09-16T08:57:16.481 INFO:tasks.workunit.client.1.smithi097.stdout:7/657: getdents d4/d7/d77 0 2024-09-16T08:57:16.481 INFO:tasks.workunit.client.1.smithi097.stdout:7/658: readlink d4/d7/d7b/l7f 0 2024-09-16T08:57:16.482 INFO:tasks.workunit.client.0.smithi031.stdout:0/290: dread d6/d12/d47/f55 [0,4194304] 0 2024-09-16T08:57:16.484 INFO:tasks.workunit.client.1.smithi097.stdout:3/751: creat d7/d28/d63/dd8/f104 x:0 0 0 2024-09-16T08:57:16.485 INFO:tasks.workunit.client.1.smithi097.stdout:3/752: fsync d7/de/d3c/d3f/d60/dcb/d35/f22 0 2024-09-16T08:57:16.485 INFO:tasks.workunit.client.1.smithi097.stdout:3/753: getdents d7/de/d3c/d3f/d60/dcb 0 2024-09-16T08:57:16.487 INFO:tasks.workunit.client.1.smithi097.stdout:1/817: creat d5/dce/f108 x:0 0 0 2024-09-16T08:57:16.487 INFO:tasks.workunit.client.1.smithi097.stdout:1/818: stat d5/d14/d95/fa0 0 2024-09-16T08:57:16.487 INFO:tasks.workunit.client.1.smithi097.stdout:1/819: creat d5/d70/d9d/f109 x:0 0 0 2024-09-16T08:57:16.493 INFO:tasks.workunit.client.1.smithi097.stdout:1/820: rmdir d5/d17/df5 39 2024-09-16T08:57:16.494 INFO:tasks.workunit.client.1.smithi097.stdout:7/659: write d4/d12/d1f/d22/f3d [862065,51033] 0 2024-09-16T08:57:16.496 INFO:tasks.workunit.client.0.smithi031.stdout:6/405: mknod d0/dd/d74/d5e/d7c/d3f/c80 0 2024-09-16T08:57:16.496 INFO:tasks.workunit.client.0.smithi031.stdout:6/406: chown d0/dd/d74/d67/f7a 199120 1 2024-09-16T08:57:16.496 INFO:tasks.workunit.client.0.smithi031.stdout:6/407: write d0/dd/d74/d19/f32 [120892,11791] 0 2024-09-16T08:57:16.496 INFO:tasks.workunit.client.1.smithi097.stdout:1/821: unlink d5/d14/ce6 0 2024-09-16T08:57:16.497 INFO:tasks.workunit.client.1.smithi097.stdout:3/754: dread d7/de/d3c/d3f/f6d [0,4194304] 0 2024-09-16T08:57:16.497 INFO:tasks.workunit.client.1.smithi097.stdout:3/755: readlink d7/de/d8d/lef 0 2024-09-16T08:57:16.497 INFO:tasks.workunit.client.1.smithi097.stdout:3/756: chown d7/d15/d52/d81 552624075 1 2024-09-16T08:57:16.498 INFO:tasks.workunit.client.0.smithi031.stdout:2/239: mkdir d6/da/d4b/d4c 0 2024-09-16T08:57:16.498 INFO:tasks.workunit.client.0.smithi031.stdout:2/240: stat d6/da/c1c 0 2024-09-16T08:57:16.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/424: write d3/f59 [4613725,69031] 0 2024-09-16T08:57:16.501 INFO:tasks.workunit.client.1.smithi097.stdout:1/822: mknod d5/d2c/dcb/c10a 0 2024-09-16T08:57:16.501 INFO:tasks.workunit.client.1.smithi097.stdout:1/823: stat d5/db/dc/d19/lec 0 2024-09-16T08:57:16.501 INFO:tasks.workunit.client.1.smithi097.stdout:1/824: write d5/d2c/fc3 [31156,35181] 0 2024-09-16T08:57:16.501 INFO:tasks.workunit.client.1.smithi097.stdout:1/825: dread - d5/d2c/dcb/de2/fe5 zero size 2024-09-16T08:57:16.502 INFO:tasks.workunit.client.0.smithi031.stdout:7/395: creat d2/d8/d86/f88 x:0 0 0 2024-09-16T08:57:16.503 INFO:tasks.workunit.client.1.smithi097.stdout:3/757: symlink d7/de/d9c/daf/l105 0 2024-09-16T08:57:16.506 INFO:tasks.workunit.client.1.smithi097.stdout:2/770: dwrite d3/d4/d68/d8b/d61/dac/fd8 [0,4194304] 0 2024-09-16T08:57:16.506 INFO:tasks.workunit.client.0.smithi031.stdout:4/255: dwrite d0/f4c [0,4194304] 0 2024-09-16T08:57:16.506 INFO:tasks.workunit.client.0.smithi031.stdout:4/256: creat d0/d2/d16/f4f x:0 0 0 2024-09-16T08:57:16.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/369: dwrite d2/dc/f43 [0,4194304] 0 2024-09-16T08:57:16.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/370: fsync d2/dc/d36/f41 0 2024-09-16T08:57:16.507 INFO:tasks.workunit.client.0.smithi031.stdout:0/291: rename d6/d12/l1c to d6/d12/d47/d5b/l64 0 2024-09-16T08:57:16.507 INFO:tasks.workunit.client.1.smithi097.stdout:1/826: rename d5/l27 to d5/d17/df5/l10b 0 2024-09-16T08:57:16.508 INFO:tasks.workunit.client.1.smithi097.stdout:1/827: write d5/d70/d9d/fd0 [648407,97739] 0 2024-09-16T08:57:16.509 INFO:tasks.workunit.client.1.smithi097.stdout:1/828: write f4 [2067671,48954] 0 2024-09-16T08:57:16.509 INFO:tasks.workunit.client.1.smithi097.stdout:9/689: dwrite d2/d5/f71 [0,4194304] 0 2024-09-16T08:57:16.513 INFO:tasks.workunit.client.0.smithi031.stdout:6/408: mkdir d0/dd/d35/d6c/d81 0 2024-09-16T08:57:16.515 INFO:tasks.workunit.client.1.smithi097.stdout:7/660: dread d4/d7/f11 [0,4194304] 0 2024-09-16T08:57:16.518 INFO:tasks.workunit.client.1.smithi097.stdout:2/771: symlink d3/d4/d68/d8b/d61/dac/dd9/le5 0 2024-09-16T08:57:16.518 INFO:tasks.workunit.client.1.smithi097.stdout:2/772: fsync d3/f32 0 2024-09-16T08:57:16.519 INFO:tasks.workunit.client.1.smithi097.stdout:1/829: dread d5/db/f34 [4194304,4194304] 0 2024-09-16T08:57:16.524 INFO:tasks.workunit.client.1.smithi097.stdout:3/758: rename d7/de/d8d/d90/d92/fc6 to d7/de/d3c/d3f/d60/dcb/f106 0 2024-09-16T08:57:16.525 INFO:tasks.workunit.client.0.smithi031.stdout:1/267: dwrite f0 [0,4194304] 0 2024-09-16T08:57:16.525 INFO:tasks.workunit.client.0.smithi031.stdout:2/241: dread d6/d41/f2b [0,4194304] 0 2024-09-16T08:57:16.525 INFO:tasks.workunit.client.0.smithi031.stdout:2/242: creat d6/f4d x:0 0 0 2024-09-16T08:57:16.527 INFO:tasks.workunit.client.1.smithi097.stdout:7/661: write d4/d7/d30/f68 [4084484,30713] 0 2024-09-16T08:57:16.527 INFO:tasks.workunit.client.0.smithi031.stdout:0/292: dread d6/d12/d13/f24 [0,4194304] 0 2024-09-16T08:57:16.528 INFO:tasks.workunit.client.0.smithi031.stdout:6/409: symlink d0/dd/d74/d5e/d7c/d3f/l82 0 2024-09-16T08:57:16.529 INFO:tasks.workunit.client.0.smithi031.stdout:3/425: dwrite d3/d14/d37/d33/f6d [0,4194304] 0 2024-09-16T08:57:16.535 INFO:tasks.workunit.client.1.smithi097.stdout:4/698: sync 2024-09-16T08:57:16.535 INFO:tasks.workunit.client.1.smithi097.stdout:6/748: sync 2024-09-16T08:57:16.535 INFO:tasks.workunit.client.1.smithi097.stdout:6/749: chown d1/dd3/d9e/dca/d16/d86/l8f 245 1 2024-09-16T08:57:16.536 INFO:tasks.workunit.client.0.smithi031.stdout:1/268: mkdir d2/d4/d1b/d5c 0 2024-09-16T08:57:16.536 INFO:tasks.workunit.client.0.smithi031.stdout:1/269: read - d2/f53 zero size 2024-09-16T08:57:16.539 INFO:tasks.workunit.client.0.smithi031.stdout:4/257: dwrite d0/fb [0,4194304] 0 2024-09-16T08:57:16.540 INFO:tasks.workunit.client.0.smithi031.stdout:5/371: dread d2/dc/d25/d21/f6d [0,4194304] 0 2024-09-16T08:57:16.540 INFO:tasks.workunit.client.0.smithi031.stdout:5/372: stat d2/dc/c10 0 2024-09-16T08:57:16.541 INFO:tasks.workunit.client.0.smithi031.stdout:2/243: rename d6/d20/l48 to d6/d41/d2c/l4e 0 2024-09-16T08:57:16.546 INFO:tasks.workunit.client.1.smithi097.stdout:2/773: mkdir d3/d4/d68/d8b/d61/dac/dd9/de6 0 2024-09-16T08:57:16.547 INFO:tasks.workunit.client.1.smithi097.stdout:2/774: dread - d3/d4/d68/d7f/fbd zero size 2024-09-16T08:57:16.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/426: rmdir d3/d14/d37/d1a/d1d/d3b/d3f/d79 39 2024-09-16T08:57:16.548 INFO:tasks.workunit.client.1.smithi097.stdout:1/830: unlink d5/d14/la4 0 2024-09-16T08:57:16.548 INFO:tasks.workunit.client.1.smithi097.stdout:1/831: chown d5/d70/d85/db5 29342783 1 2024-09-16T08:57:16.549 INFO:tasks.workunit.client.0.smithi031.stdout:1/270: getdents d2/d15 0 2024-09-16T08:57:16.550 INFO:tasks.workunit.client.0.smithi031.stdout:7/396: dwrite d2/d8/f2c [0,4194304] 0 2024-09-16T08:57:16.551 INFO:tasks.workunit.client.0.smithi031.stdout:7/397: dread - d2/d8/d11/f55 zero size 2024-09-16T08:57:16.551 INFO:tasks.workunit.client.1.smithi097.stdout:9/690: dwrite d2/d5/db0/dc8/d74/d8c/d94/f99 [4194304,4194304] 0 2024-09-16T08:57:16.552 INFO:tasks.workunit.client.1.smithi097.stdout:3/759: rename d7/d28/d63/l65 to d7/d28/d30/d88/df7/l107 0 2024-09-16T08:57:16.553 INFO:tasks.workunit.client.0.smithi031.stdout:6/410: dwrite d0/dd/d74/d5e/d7c/d3f/f70 [0,4194304] 0 2024-09-16T08:57:16.553 INFO:tasks.workunit.client.0.smithi031.stdout:6/411: truncate d0/dd/d74/d5e/d7c/d3f/d65/f68 1584169 0 2024-09-16T08:57:16.554 INFO:tasks.workunit.client.0.smithi031.stdout:4/258: dread d0/f39 [0,4194304] 0 2024-09-16T08:57:16.554 INFO:tasks.workunit.client.0.smithi031.stdout:4/259: stat d0/d2/d16/d2b/l40 0 2024-09-16T08:57:16.558 INFO:tasks.workunit.client.1.smithi097.stdout:1/832: dread d5/d70/fcc [0,4194304] 0 2024-09-16T08:57:16.558 INFO:tasks.workunit.client.1.smithi097.stdout:7/662: mkdir d4/d1b/da4/de0 0 2024-09-16T08:57:16.558 INFO:tasks.workunit.client.1.smithi097.stdout:1/833: write d5/db/dc/d19/ff1 [23534,129899] 0 2024-09-16T08:57:16.560 INFO:tasks.workunit.client.0.smithi031.stdout:1/271: dread f0 [0,4194304] 0 2024-09-16T08:57:16.560 INFO:tasks.workunit.client.0.smithi031.stdout:1/272: getdents d2/d15 0 2024-09-16T08:57:16.560 INFO:tasks.workunit.client.0.smithi031.stdout:2/244: rename d6/f8 to d6/d20/f4f 0 2024-09-16T08:57:16.563 INFO:tasks.workunit.client.1.smithi097.stdout:4/699: mkdir d0/d23/ddc/de5/df9 0 2024-09-16T08:57:16.563 INFO:tasks.workunit.client.1.smithi097.stdout:4/700: chown d0/d7/df/f37 86970 1 2024-09-16T08:57:16.565 INFO:tasks.workunit.client.1.smithi097.stdout:6/750: mkdir d1/dd3/d9e/dca/d81/dc4/df7 0 2024-09-16T08:57:16.567 INFO:tasks.workunit.client.0.smithi031.stdout:3/427: mkdir d3/d14/d37/d1a/d1d/d3b/d3f/d79/d7c 0 2024-09-16T08:57:16.569 INFO:tasks.workunit.client.1.smithi097.stdout:2/775: symlink d3/d4/d68/d8b/d61/dac/le7 0 2024-09-16T08:57:16.570 INFO:tasks.workunit.client.1.smithi097.stdout:6/751: write d1/dd3/d9e/dca/d16/f7b [635671,37012] 0 2024-09-16T08:57:16.572 INFO:tasks.workunit.client.0.smithi031.stdout:0/293: getdents d6/d12/d47/d5b 0 2024-09-16T08:57:16.575 INFO:tasks.workunit.client.1.smithi097.stdout:9/691: link d2/d5/db0/dc8/d74/d8c/d94/d96/dac/l80 d2/d5/d3a/d3c/ldf 0 2024-09-16T08:57:16.575 INFO:tasks.workunit.client.1.smithi097.stdout:9/692: write d2/d5/d3a/d70/da6/fd9 [351965,31236] 0 2024-09-16T08:57:16.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/398: dwrite d2/d8/f2f [4194304,4194304] 0 2024-09-16T08:57:16.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/399: creat d2/d2d/f89 x:0 0 0 2024-09-16T08:57:16.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/400: fsync d2/f6a 0 2024-09-16T08:57:16.577 INFO:tasks.workunit.client.1.smithi097.stdout:1/834: dwrite d5/d2c/f4e [4194304,4194304] 0 2024-09-16T08:57:16.577 INFO:tasks.workunit.client.1.smithi097.stdout:1/835: write d5/d14/d95/fd8 [705601,51265] 0 2024-09-16T08:57:16.577 INFO:tasks.workunit.client.1.smithi097.stdout:1/836: fsync d5/d70/d85/db5/fef 0 2024-09-16T08:57:16.577 INFO:tasks.workunit.client.1.smithi097.stdout:3/760: mkdir d7/de/d3c/d3f/d108 0 2024-09-16T08:57:16.578 INFO:tasks.workunit.client.0.smithi031.stdout:6/412: mkdir d0/dd/d35/d6c/d83 0 2024-09-16T08:57:16.579 INFO:tasks.workunit.client.1.smithi097.stdout:4/701: mkdir d0/d7/d52/d26/d8c/db0/dfa 0 2024-09-16T08:57:16.580 INFO:tasks.workunit.client.0.smithi031.stdout:4/260: mkdir d0/d2/d16/d50 0 2024-09-16T08:57:16.581 INFO:tasks.workunit.client.1.smithi097.stdout:2/776: mkdir d3/d4/d68/d8b/d7c/d7d/d53/da3/de8 0 2024-09-16T08:57:16.583 INFO:tasks.workunit.client.0.smithi031.stdout:5/373: link d2/dc/d25/d21/d2e/l35 d2/dc/d3d/d4c/l78 0 2024-09-16T08:57:16.583 INFO:tasks.workunit.client.0.smithi031.stdout:5/374: creat d2/d4b/f79 x:0 0 0 2024-09-16T08:57:16.584 INFO:tasks.workunit.client.1.smithi097.stdout:6/752: mkdir d1/dd3/d9e/dca/d16/d4f/df8 0 2024-09-16T08:57:16.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.583+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6690005800 con 0x7f66ac1074b0 2024-09-16T08:57:16.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.583+0000 7f66a9ffb640 1 -- 172.21.15.31:0/3537088789 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f669408da10 con 0x7f66ac1074b0 2024-09-16T08:57:16.585 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:57:16.585 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:57:16.586 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:57:16.587 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:16.588 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:57:16.589 INFO:tasks.workunit.client.0.smithi031.stdout:2/245: symlink d6/da/d37/d42/l50 0 2024-09-16T08:57:16.589 INFO:tasks.workunit.client.1.smithi097.stdout:9/693: mkdir d2/d5/db0/dc8/d74/de0 0 2024-09-16T08:57:16.589 INFO:tasks.workunit.client.1.smithi097.stdout:9/694: fsync d2/f4 0 2024-09-16T08:57:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f668c073dd0 msgr2=0x7f668c076290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f668c073dd0 0x7f668c076290 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f66ac19d1d0 tx=0x7f669c006c40 comp rx=0 tx=0).stop 2024-09-16T08:57:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 msgr2=0x7f66ac19c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac19c230 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6694009850 tx=0x7f6694005630 comp rx=0 tx=0).stop 2024-09-16T08:57:16.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 shutdown_connections 2024-09-16T08:57:16.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f668c073dd0 0x7f668c076290 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66ac1074b0 0x7f66ac19c230 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 --2- 172.21.15.31:0/3537088789 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f66ac071430 0x7f66ac19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.588+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 >> 172.21.15.31:0/3537088789 conn(0x7f66ac06cc10 msgr2=0x7f66ac10a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:16.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.589+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 shutdown_connections 2024-09-16T08:57:16.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.589+0000 7f66b291b640 1 -- 172.21.15.31:0/3537088789 wait complete. 2024-09-16T08:57:16.591 INFO:tasks.workunit.client.0.smithi031.stdout:3/428: mkdir d3/d14/d6f/d7d 0 2024-09-16T08:57:16.592 INFO:tasks.workunit.client.1.smithi097.stdout:3/761: dwrite d7/d28/d30/d88/fa5 [0,4194304] 0 2024-09-16T08:57:16.592 INFO:tasks.workunit.client.1.smithi097.stdout:3/762: stat d7/de/d9c/dca 0 2024-09-16T08:57:16.593 INFO:tasks.workunit.client.1.smithi097.stdout:5/790: sync 2024-09-16T08:57:16.596 INFO:tasks.workunit.client.1.smithi097.stdout:1/837: symlink d5/db/deb/l10c 0 2024-09-16T08:57:16.596 INFO:tasks.workunit.client.1.smithi097.stdout:1/838: readlink d5/d2c/d3b/d7f/laf 0 2024-09-16T08:57:16.599 INFO:tasks.workunit.client.1.smithi097.stdout:4/702: link d0/d39/d71/fa6 d0/d7/d52/d26/d8c/ffb 0 2024-09-16T08:57:16.600 INFO:tasks.workunit.client.0.smithi031.stdout:7/401: mknod d2/d8/d11/d5d/d7e/c8a 0 2024-09-16T08:57:16.600 INFO:tasks.workunit.client.0.smithi031.stdout:7/402: stat d2/d8/d11/f40 0 2024-09-16T08:57:16.600 INFO:tasks.workunit.client.0.smithi031.stdout:7/403: truncate d2/d2d/f89 294253 0 2024-09-16T08:57:16.600 INFO:tasks.workunit.client.0.smithi031.stdout:7/404: creat d2/d8/f8b x:0 0 0 2024-09-16T08:57:16.600 INFO:tasks.workunit.client.0.smithi031.stdout:7/405: dread - d2/d2d/f58 zero size 2024-09-16T08:57:16.601 INFO:tasks.workunit.client.1.smithi097.stdout:4/703: write d0/d7/d52/d26/d3d/f76 [287137,30445] 0 2024-09-16T08:57:16.601 INFO:tasks.workunit.client.1.smithi097.stdout:4/704: chown d0/d23/d5f/fc9 7523125 1 2024-09-16T08:57:16.601 INFO:tasks.workunit.client.1.smithi097.stdout:4/705: readlink d0/d7/df/l4a 0 2024-09-16T08:57:16.603 INFO:tasks.workunit.client.0.smithi031.stdout:6/413: creat d0/f84 x:0 0 0 2024-09-16T08:57:16.603 INFO:tasks.workunit.client.0.smithi031.stdout:6/414: write d0/ff [1752362,41460] 0 2024-09-16T08:57:16.603 INFO:tasks.workunit.client.0.smithi031.stdout:6/415: write d0/f84 [619895,13702] 0 2024-09-16T08:57:16.605 INFO:tasks.workunit.client.1.smithi097.stdout:3/763: dread d7/de/d3c/d3f/d60/dcb/d35/f22 [0,4194304] 0 2024-09-16T08:57:16.606 INFO:tasks.workunit.client.0.smithi031.stdout:4/261: unlink d0/d2/l26 0 2024-09-16T08:57:16.606 INFO:tasks.workunit.client.0.smithi031.stdout:4/262: chown d0/d2/f27 57666503 1 2024-09-16T08:57:16.607 INFO:tasks.workunit.client.1.smithi097.stdout:9/695: getdents d2/d5/db0/dc8/d63/da8 0 2024-09-16T08:57:16.612 INFO:tasks.workunit.client.1.smithi097.stdout:6/753: dwrite d1/dd3/d9e/dca/d16/ff1 [0,4194304] 0 2024-09-16T08:57:16.616 INFO:tasks.workunit.client.1.smithi097.stdout:5/791: dwrite db/f8b [0,4194304] 0 2024-09-16T08:57:16.617 INFO:tasks.workunit.client.1.smithi097.stdout:1/839: symlink d5/d17/d54/d6b/d71/dd3/l10d 0 2024-09-16T08:57:16.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/294: getdents d6/d12/d47 0 2024-09-16T08:57:16.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/295: write d6/f9 [4495103,23253] 0 2024-09-16T08:57:16.620 INFO:tasks.workunit.client.0.smithi031.stdout:3/429: dwrite d3/d14/f2f [0,4194304] 0 2024-09-16T08:57:16.622 INFO:tasks.workunit.client.1.smithi097.stdout:4/706: rename d0/d7/df/f2c to d0/d23/ddc/ffc 0 2024-09-16T08:57:16.624 INFO:tasks.workunit.client.1.smithi097.stdout:9/696: truncate f0 495553 0 2024-09-16T08:57:16.624 INFO:tasks.workunit.client.1.smithi097.stdout:9/697: write d2/d5/fbd [687799,62032] 0 2024-09-16T08:57:16.624 INFO:tasks.workunit.client.1.smithi097.stdout:9/698: chown d2/d5/c1d 61096 1 2024-09-16T08:57:16.627 INFO:tasks.workunit.client.0.smithi031.stdout:4/263: mknod d0/c51 0 2024-09-16T08:57:16.629 INFO:tasks.workunit.client.0.smithi031.stdout:3/430: dread d3/d14/d37/f10 [4194304,4194304] 0 2024-09-16T08:57:16.631 INFO:tasks.workunit.client.0.smithi031.stdout:1/273: rename d2/d4/d22 to d2/d15/d5d 0 2024-09-16T08:57:16.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/274: chown d2/d4/d36 152 1 2024-09-16T08:57:16.632 INFO:tasks.workunit.client.0.smithi031.stdout:1/275: chown d2/d15/d1f/c21 1561 1 2024-09-16T08:57:16.635 INFO:tasks.workunit.client.0.smithi031.stdout:0/296: mknod d6/d12/d13/d36/c65 0 2024-09-16T08:57:16.636 INFO:tasks.workunit.client.0.smithi031.stdout:0/297: write d6/d12/f37 [602905,128545] 0 2024-09-16T08:57:16.637 INFO:tasks.workunit.client.0.smithi031.stdout:5/375: dwrite d2/dc/d25/d21/f26 [0,4194304] 0 2024-09-16T08:57:16.644 INFO:tasks.workunit.client.1.smithi097.stdout:9/699: dread d2/d5/db0/dc8/f32 [0,4194304] 0 2024-09-16T08:57:16.644 INFO:tasks.workunit.client.1.smithi097.stdout:9/700: chown d2/d5/d3a/d3c 19742464 1 2024-09-16T08:57:16.648 INFO:tasks.workunit.client.1.smithi097.stdout:6/754: truncate d1/dd3/d9e/dca/f7 3925043 0 2024-09-16T08:57:16.648 INFO:tasks.workunit.client.1.smithi097.stdout:6/755: getdents d1/dd3/d9e/dca/d81/da9 0 2024-09-16T08:57:16.649 INFO:tasks.workunit.client.1.smithi097.stdout:1/840: dwrite d5/d2c/d3b/f9a [0,4194304] 0 2024-09-16T08:57:16.656 INFO:tasks.workunit.client.1.smithi097.stdout:5/792: symlink db/d18/d20/l104 0 2024-09-16T08:57:16.656 INFO:tasks.workunit.client.1.smithi097.stdout:5/793: write db/d18/d20/d2b/f7c [4214862,57495] 0 2024-09-16T08:57:16.656 INFO:tasks.workunit.client.1.smithi097.stdout:5/794: write db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 [1206765,127412] 0 2024-09-16T08:57:16.657 INFO:tasks.workunit.client.1.smithi097.stdout:5/795: write db/d1a/d6d/dc6/fef [691536,119535] 0 2024-09-16T08:57:16.658 INFO:tasks.workunit.client.0.smithi031.stdout:4/264: dwrite d0/d2/d16/d1e/f34 [0,4194304] 0 2024-09-16T08:57:16.658 INFO:tasks.workunit.client.0.smithi031.stdout:4/265: chown d0/d2/d16/d1e/f3b 3384055 1 2024-09-16T08:57:16.658 INFO:tasks.workunit.client.0.smithi031.stdout:4/266: truncate d0/d2/d16/d1e/f3f 851312 0 2024-09-16T08:57:16.658 INFO:tasks.workunit.client.0.smithi031.stdout:4/267: truncate d0/f2e 859077 0 2024-09-16T08:57:16.658 INFO:tasks.workunit.client.0.smithi031.stdout:1/276: dwrite d2/de/f2a [4194304,4194304] 0 2024-09-16T08:57:16.659 INFO:tasks.workunit.client.1.smithi097.stdout:3/764: rename d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/l83 to d7/de/d6c/dcf/l109 0 2024-09-16T08:57:16.660 INFO:tasks.workunit.client.1.smithi097.stdout:4/707: symlink d0/d23/ddc/de5/df9/lfd 0 2024-09-16T08:57:16.661 INFO:tasks.workunit.client.1.smithi097.stdout:4/708: read d0/d7/d52/f50 [1208738,24736] 0 2024-09-16T08:57:16.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/406: rename d2/d8/d11/f55 to d2/d8/f8c 0 2024-09-16T08:57:16.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/407: stat d2/d8/c29 0 2024-09-16T08:57:16.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/408: dread - d2/d8/d86/f88 zero size 2024-09-16T08:57:16.663 INFO:tasks.workunit.client.0.smithi031.stdout:7/409: truncate d2/d2d/f77 84652 0 2024-09-16T08:57:16.666 INFO:tasks.workunit.client.0.smithi031.stdout:0/298: truncate d6/f4f 1023258 0 2024-09-16T08:57:16.666 INFO:tasks.workunit.client.0.smithi031.stdout:1/277: dread d2/de/f2f [0,4194304] 0 2024-09-16T08:57:16.667 INFO:tasks.workunit.client.1.smithi097.stdout:9/701: symlink d2/d5/db0/dc8/d74/le1 0 2024-09-16T08:57:16.668 INFO:tasks.workunit.client.1.smithi097.stdout:9/702: dread d2/d5/fb [8388608,4194304] 0 2024-09-16T08:57:16.670 INFO:tasks.workunit.client.1.smithi097.stdout:6/756: symlink d1/lf9 0 2024-09-16T08:57:16.672 INFO:tasks.workunit.client.1.smithi097.stdout:4/709: dread d0/d39/f6a [0,4194304] 0 2024-09-16T08:57:16.673 INFO:tasks.workunit.client.1.smithi097.stdout:1/841: symlink d5/db/dc/l10e 0 2024-09-16T08:57:16.673 INFO:tasks.workunit.client.1.smithi097.stdout:1/842: creat d5/d17/d54/d6b/da8/f10f x:0 0 0 2024-09-16T08:57:16.677 INFO:tasks.workunit.client.0.smithi031.stdout:4/268: link d0/fb d0/d2/d16/d1e/f52 0 2024-09-16T08:57:16.677 INFO:tasks.workunit.client.1.smithi097.stdout:0/733: sync 2024-09-16T08:57:16.677 INFO:tasks.workunit.client.1.smithi097.stdout:8/695: sync 2024-09-16T08:57:16.678 INFO:tasks.workunit.client.1.smithi097.stdout:5/796: mkdir db/d1a/d6d/dce/d105 0 2024-09-16T08:57:16.680 INFO:tasks.workunit.client.1.smithi097.stdout:3/765: link d7/d15/f17 d7/de/d3c/d3f/d60/dcb/f10a 0 2024-09-16T08:57:16.687 INFO:tasks.workunit.client.1.smithi097.stdout:9/703: creat d2/d5/d56/d6b/dcd/dd8/fe2 x:0 0 0 2024-09-16T08:57:16.688 INFO:tasks.workunit.client.1.smithi097.stdout:9/704: write d2/d5/d72/f89 [770285,74620] 0 2024-09-16T08:57:16.688 INFO:tasks.workunit.client.0.smithi031.stdout:7/410: mknod d2/d2d/d68/c8d 0 2024-09-16T08:57:16.690 INFO:tasks.workunit.client.1.smithi097.stdout:6/757: unlink d1/f22 0 2024-09-16T08:57:16.691 INFO:tasks.workunit.client.1.smithi097.stdout:6/758: write d1/dd3/d9e/dca/d16/ff1 [4995181,81883] 0 2024-09-16T08:57:16.691 INFO:tasks.workunit.client.1.smithi097.stdout:6/759: creat d1/db7/ffa x:0 0 0 2024-09-16T08:57:16.691 INFO:tasks.workunit.client.1.smithi097.stdout:6/760: write d1/dd3/d9e/dca/fe8 [333962,63190] 0 2024-09-16T08:57:16.691 INFO:tasks.workunit.client.0.smithi031.stdout:0/299: truncate d6/d12/d13/f30 4076086 0 2024-09-16T08:57:16.694 INFO:tasks.workunit.client.0.smithi031.stdout:1/278: link d2/d4/c35 d2/d15/d1f/c5e 0 2024-09-16T08:57:16.697 INFO:tasks.workunit.client.1.smithi097.stdout:0/734: symlink da/d7e/dc4/dfc/d47/db3/dbf/dcc/l102 0 2024-09-16T08:57:16.697 INFO:tasks.workunit.client.0.smithi031.stdout:0/300: write d6/f4f [119998,82237] 0 2024-09-16T08:57:16.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/416: rename d0/dd/d35/f52 to d0/dd/d74/d19/f85 0 2024-09-16T08:57:16.698 INFO:tasks.workunit.client.0.smithi031.stdout:6/417: dread - d0/f72 zero size 2024-09-16T08:57:16.699 INFO:tasks.workunit.client.1.smithi097.stdout:0/735: dread da/f1b [0,4194304] 0 2024-09-16T08:57:16.699 INFO:tasks.workunit.client.1.smithi097.stdout:0/736: creat da/d24/d4b/f103 x:0 0 0 2024-09-16T08:57:16.699 INFO:tasks.workunit.client.1.smithi097.stdout:5/797: creat db/d1a/d6d/dce/f106 x:0 0 0 2024-09-16T08:57:16.701 INFO:tasks.workunit.client.0.smithi031.stdout:7/411: symlink d2/d8/d11/d5d/l8e 0 2024-09-16T08:57:16.701 INFO:tasks.workunit.client.0.smithi031.stdout:7/412: dread - d2/d2d/d68/f6b zero size 2024-09-16T08:57:16.702 INFO:tasks.workunit.client.1.smithi097.stdout:9/705: symlink d2/d5/db0/dc8/d74/d8c/d94/d96/dac/le3 0 2024-09-16T08:57:16.705 INFO:tasks.workunit.client.0.smithi031.stdout:4/269: rmdir d0/d2 39 2024-09-16T08:57:16.706 INFO:tasks.workunit.client.0.smithi031.stdout:8/250: sync 2024-09-16T08:57:16.706 INFO:tasks.workunit.client.0.smithi031.stdout:9/305: sync 2024-09-16T08:57:16.706 INFO:tasks.workunit.client.0.smithi031.stdout:6/418: dread d0/dd/d74/d19/f85 [0,4194304] 0 2024-09-16T08:57:16.707 INFO:tasks.workunit.client.1.smithi097.stdout:0/737: write da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/f72 [7060654,100450] 0 2024-09-16T08:57:16.707 INFO:tasks.workunit.client.1.smithi097.stdout:6/761: mknod d1/dd3/d9e/dca/d81/dc4/df7/cfb 0 2024-09-16T08:57:16.707 INFO:tasks.workunit.client.1.smithi097.stdout:6/762: fdatasync d1/db7/ffa 0 2024-09-16T08:57:16.707 INFO:tasks.workunit.client.1.smithi097.stdout:6/763: write d1/dc/d3c/f76 [4379790,6872] 0 2024-09-16T08:57:16.708 INFO:tasks.workunit.client.0.smithi031.stdout:0/301: symlink d6/d12/l66 0 2024-09-16T08:57:16.709 INFO:tasks.workunit.client.1.smithi097.stdout:1/843: getdents d5/d2c/dcb 0 2024-09-16T08:57:16.709 INFO:tasks.workunit.client.1.smithi097.stdout:1/844: readlink d5/l51 0 2024-09-16T08:57:16.709 INFO:tasks.workunit.client.1.smithi097.stdout:1/845: truncate d5/d17/d54/d6b/f9c 1092747 0 2024-09-16T08:57:16.709 INFO:tasks.workunit.client.1.smithi097.stdout:1/846: dread - d5/d17/d54/d6b/d71/d94/fbb zero size 2024-09-16T08:57:16.710 INFO:tasks.workunit.client.0.smithi031.stdout:3/431: rename d3/d14/f2f to d3/d14/d37/d1a/d1d/d3b/d60/f7e 0 2024-09-16T08:57:16.711 INFO:tasks.workunit.client.1.smithi097.stdout:7/663: sync 2024-09-16T08:57:16.711 INFO:tasks.workunit.client.1.smithi097.stdout:2/777: sync 2024-09-16T08:57:16.711 INFO:tasks.workunit.client.1.smithi097.stdout:7/664: read d4/d12/d1f/d22/f3d [1668814,125147] 0 2024-09-16T08:57:16.711 INFO:tasks.workunit.client.1.smithi097.stdout:7/665: truncate d4/f23 5075125 0 2024-09-16T08:57:16.711 INFO:tasks.workunit.client.0.smithi031.stdout:7/413: truncate d2/d2d/f6f 162565 0 2024-09-16T08:57:16.713 INFO:tasks.workunit.client.1.smithi097.stdout:5/798: mknod db/d18/d1c/c107 0 2024-09-16T08:57:16.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/306: symlink d3/d5/da/d34/d3c/d5e/d62/d65/l6d 0 2024-09-16T08:57:16.715 INFO:tasks.workunit.client.1.smithi097.stdout:9/706: mkdir d2/d5/db0/dc8/d74/d8c/d94/de4 0 2024-09-16T08:57:16.717 INFO:tasks.workunit.client.0.smithi031.stdout:6/419: symlink d0/dd/l86 0 2024-09-16T08:57:16.724 INFO:tasks.workunit.client.1.smithi097.stdout:0/738: rename da/d24/d4c/f9d to da/d24/d4c/d80/d3b/d86/f104 0 2024-09-16T08:57:16.725 INFO:tasks.workunit.client.1.smithi097.stdout:6/764: truncate d1/dd3/d9e/dca/d16/d86/f8b 2264132 0 2024-09-16T08:57:16.725 INFO:tasks.workunit.client.1.smithi097.stdout:2/778: dread d3/d4/d68/d8b/d7c/d7d/f6b [0,4194304] 0 2024-09-16T08:57:16.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.725+0000 7f090970b640 1 -- 172.21.15.31:0/823582417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 msgr2=0x7f09040feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.725+0000 7f090970b640 1 --2- 172.21.15.31:0/823582417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f09040feef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009880 tx=0x7f08f402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.725+0000 7f090970b640 1 -- 172.21.15.31:0/823582417 shutdown_connections 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.725+0000 7f090970b640 1 --2- 172.21.15.31:0/823582417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ffcf0 0x7f0904100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.725+0000 7f090970b640 1 --2- 172.21.15.31:0/823582417 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f09040feef0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.725+0000 7f090970b640 1 -- 172.21.15.31:0/823582417 >> 172.21.15.31:0/823582417 conn(0x7f09040fa280 msgr2=0x7f09040fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.726+0000 7f090970b640 1 -- 172.21.15.31:0/823582417 shutdown_connections 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.726+0000 7f090970b640 1 -- 172.21.15.31:0/823582417 wait complete. 2024-09-16T08:57:16.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.726+0000 7f090970b640 1 Processor -- start 2024-09-16T08:57:16.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.726+0000 7f090970b640 1 -- start start 2024-09-16T08:57:16.729 INFO:tasks.workunit.client.0.smithi031.stdout:1/279: rename d2/d4/f30 to d2/d15/d32/f5f 0 2024-09-16T08:57:16.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.727+0000 7f090970b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f0904195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:16.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.727+0000 7f090970b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ffcf0 0x7f0904195c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:16.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.727+0000 7f090970b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904196250 con 0x7f09040feaf0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.727+0000 7f090970b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041963c0 con 0x7f09040ffcf0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.727+0000 7f0902ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f0904195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.727+0000 7f0902ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f0904195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34864/0 (socket says 172.21.15.31:34864) 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.727+0000 7f0902ffd640 1 -- 172.21.15.31:0/1629198732 learned_addr learned my addr 172.21.15.31:0/1629198732 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.728+0000 7f0902ffd640 1 -- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ffcf0 msgr2=0x7f0904195c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.728+0000 7f0902ffd640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ffcf0 0x7f0904195c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.728+0000 7f0902ffd640 1 -- 172.21.15.31:0/1629198732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4009560 con 0x7f09040feaf0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.728+0000 7f0902ffd640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f0904195740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005610 tx=0x7f08f4005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.728+0000 7f08ebfff640 1 -- 172.21.15.31:0/1629198732 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f4004ba0 con 0x7f09040feaf0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.728+0000 7f08ebfff640 1 -- 172.21.15.31:0/1629198732 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08f4004d00 con 0x7f09040feaf0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.728+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0904100f10 con 0x7f09040feaf0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.729+0000 7f08ebfff640 1 -- 172.21.15.31:0/1629198732 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f4003bd0 con 0x7f09040feaf0 2024-09-16T08:57:16.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.729+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09041013c0 con 0x7f09040feaf0 2024-09-16T08:57:16.732 INFO:tasks.workunit.client.1.smithi097.stdout:1/847: symlink d5/d2c/dcb/l110 0 2024-09-16T08:57:16.732 INFO:tasks.workunit.client.0.smithi031.stdout:3/432: mkdir d3/d14/d37/d1a/d1d/d1e/d67/d7f 0 2024-09-16T08:57:16.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.730+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0904107760 con 0x7f09040feaf0 2024-09-16T08:57:16.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.731+0000 7f08ebfff640 1 -- 172.21.15.31:0/1629198732 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f08f4031640 con 0x7f09040feaf0 2024-09-16T08:57:16.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.732+0000 7f08ebfff640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f08ec073d40 0x7f08ec076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:16.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.732+0000 7f08ebfff640 1 -- 172.21.15.31:0/1629198732 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f08f40b62e0 con 0x7f09040feaf0 2024-09-16T08:57:16.735 INFO:tasks.workunit.client.1.smithi097.stdout:7/666: link d4/d9/d1c/d5f/f6c d4/d9/dc/fe1 0 2024-09-16T08:57:16.735 INFO:tasks.workunit.client.1.smithi097.stdout:7/667: fdatasync d4/d7/d77/f84 0 2024-09-16T08:57:16.736 INFO:tasks.workunit.client.0.smithi031.stdout:7/414: creat d2/d8/d69/d82/f8f x:0 0 0 2024-09-16T08:57:16.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.734+0000 7f09027fc640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f08ec073d40 0x7f08ec076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:16.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.739+0000 7f09027fc640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f08ec073d40 0x7f08ec076200 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0904196c60 tx=0x7f08f8006c90 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:16.741 INFO:tasks.workunit.client.1.smithi097.stdout:0/739: mknod da/d24/d4b/dfb/c105 0 2024-09-16T08:57:16.741 INFO:tasks.workunit.client.0.smithi031.stdout:9/307: mknod d3/d5/da/d34/d3c/d5e/d62/d65/c6e 0 2024-09-16T08:57:16.742 INFO:tasks.workunit.client.1.smithi097.stdout:8/696: read d8/dd/d71/d9c/fae [1568739,11676] 0 2024-09-16T08:57:16.742 INFO:tasks.workunit.client.1.smithi097.stdout:8/697: fdatasync d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/fd3 0 2024-09-16T08:57:16.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:16.740+0000 7f08ebfff640 1 -- 172.21.15.31:0/1629198732 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f08f4079920 con 0x7f09040feaf0 2024-09-16T08:57:16.746 INFO:tasks.workunit.client.1.smithi097.stdout:2/779: link d3/d4/d9/f9c d3/d4/d9/fe9 0 2024-09-16T08:57:16.747 INFO:tasks.workunit.client.1.smithi097.stdout:5/799: dwrite db/d22/f6a [0,4194304] 0 2024-09-16T08:57:16.747 INFO:tasks.workunit.client.0.smithi031.stdout:0/302: rmdir d6/d12/d13/d36 39 2024-09-16T08:57:16.748 INFO:tasks.workunit.client.0.smithi031.stdout:8/251: dwrite d4/d7/d21/d24/f31 [0,4194304] 0 2024-09-16T08:57:16.749 INFO:tasks.workunit.client.1.smithi097.stdout:7/668: mkdir d4/d7/d77/da8/de2 0 2024-09-16T08:57:16.750 INFO:tasks.workunit.client.0.smithi031.stdout:4/270: rename d0/d2/c44 to d0/d2/d3a/c53 0 2024-09-16T08:57:16.752 INFO:tasks.workunit.client.1.smithi097.stdout:0/740: mknod da/d7e/dde/c106 0 2024-09-16T08:57:16.753 INFO:tasks.workunit.client.0.smithi031.stdout:1/280: link d2/d4/l7 d2/d4/d52/l60 0 2024-09-16T08:57:16.755 INFO:tasks.workunit.client.1.smithi097.stdout:2/780: mknod d3/d4/d68/d8b/d61/dac/dd9/de6/cea 0 2024-09-16T08:57:16.756 INFO:tasks.workunit.client.1.smithi097.stdout:3/766: dread d7/d28/d30/d88/fa5 [0,4194304] 0 2024-09-16T08:57:16.757 INFO:tasks.workunit.client.0.smithi031.stdout:0/303: rmdir d6/d12 39 2024-09-16T08:57:16.757 INFO:tasks.workunit.client.0.smithi031.stdout:0/304: readlink d6/d12/d47/d5b/l64 0 2024-09-16T08:57:16.757 INFO:tasks.workunit.client.0.smithi031.stdout:0/305: dread - d6/d12/d47/d5b/f5e zero size 2024-09-16T08:57:16.758 INFO:tasks.workunit.client.1.smithi097.stdout:8/698: read d8/dd/d71/f73 [368094,110157] 0 2024-09-16T08:57:16.760 INFO:tasks.workunit.client.1.smithi097.stdout:1/848: dread d5/f77 [0,4194304] 0 2024-09-16T08:57:16.761 INFO:tasks.workunit.client.0.smithi031.stdout:8/252: rename d4/l1f to d4/d7/d21/d35/d40/l55 0 2024-09-16T08:57:16.764 INFO:tasks.workunit.client.1.smithi097.stdout:5/800: link db/fe db/d1a/d6d/f108 0 2024-09-16T08:57:16.765 INFO:tasks.workunit.client.1.smithi097.stdout:5/801: write db/d18/d1c/d42/d46/d7e/d76/f8e [979904,74536] 0 2024-09-16T08:57:16.765 INFO:tasks.workunit.client.1.smithi097.stdout:5/802: write db/d18/d1c/d42/d46/d7e/fe8 [29574,116783] 0 2024-09-16T08:57:16.765 INFO:tasks.workunit.client.1.smithi097.stdout:5/803: read - db/d18/d20/fc5 zero size 2024-09-16T08:57:16.766 INFO:tasks.workunit.client.1.smithi097.stdout:1/849: dread d5/d17/d54/d6b/da8/fc8 [0,4194304] 0 2024-09-16T08:57:16.766 INFO:tasks.workunit.client.1.smithi097.stdout:0/741: creat da/d7e/dc4/dfc/d47/db3/d60/d68/f107 x:0 0 0 2024-09-16T08:57:16.766 INFO:tasks.workunit.client.1.smithi097.stdout:0/742: chown da/d7e/dc4/dfc/d94/dc9/dcb/ff8 22142 1 2024-09-16T08:57:16.768 INFO:tasks.workunit.client.1.smithi097.stdout:2/781: rename d3/d4/d68/cde to d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4/ceb 0 2024-09-16T08:57:16.768 INFO:tasks.workunit.client.1.smithi097.stdout:2/782: fdatasync d3/d4/d8/f25 0 2024-09-16T08:57:16.770 INFO:tasks.workunit.client.1.smithi097.stdout:3/767: creat d7/d15/d52/d81/f10b x:0 0 0 2024-09-16T08:57:16.771 INFO:tasks.workunit.client.0.smithi031.stdout:6/420: dwrite d0/dd/d2b/f31 [0,4194304] 0 2024-09-16T08:57:16.771 INFO:tasks.workunit.client.0.smithi031.stdout:1/281: link d2/d4/l16 d2/d15/d32/l61 0 2024-09-16T08:57:16.772 INFO:tasks.workunit.client.0.smithi031.stdout:0/306: read d6/d12/d13/d36/f51 [4078826,39125] 0 2024-09-16T08:57:16.773 INFO:tasks.workunit.client.0.smithi031.stdout:0/307: write d6/d12/d13/f21 [898633,83151] 0 2024-09-16T08:57:16.773 INFO:tasks.workunit.client.0.smithi031.stdout:7/415: rename d2/d8/d69/d7a to d2/d8/d69/d90 0 2024-09-16T08:57:16.774 INFO:tasks.workunit.client.1.smithi097.stdout:5/804: creat db/d6e/d79/f109 x:0 0 0 2024-09-16T08:57:16.775 INFO:tasks.workunit.client.1.smithi097.stdout:0/743: symlink da/df/d8a/l108 0 2024-09-16T08:57:16.775 INFO:tasks.workunit.client.0.smithi031.stdout:1/282: creat d2/d15/d1f/f62 x:0 0 0 2024-09-16T08:57:16.778 INFO:tasks.workunit.client.1.smithi097.stdout:2/783: rename d3/d4/f6e to d3/d1b/dbf/fec 0 2024-09-16T08:57:16.780 INFO:tasks.workunit.client.0.smithi031.stdout:5/376: dwrite d2/dc/d25/d21/f45 [4194304,4194304] 0 2024-09-16T08:57:16.780 INFO:tasks.workunit.client.0.smithi031.stdout:7/416: mkdir d2/d2d/d6d/d91 0 2024-09-16T08:57:16.782 INFO:tasks.workunit.client.1.smithi097.stdout:5/805: symlink db/d22/l10a 0 2024-09-16T08:57:16.782 INFO:tasks.workunit.client.1.smithi097.stdout:5/806: chown db/d22/f4b 43738331 1 2024-09-16T08:57:16.782 INFO:tasks.workunit.client.0.smithi031.stdout:1/283: creat d2/d4/d52/f63 x:0 0 0 2024-09-16T08:57:16.782 INFO:tasks.workunit.client.0.smithi031.stdout:1/284: write d2/f3e [459915,117474] 0 2024-09-16T08:57:16.783 INFO:tasks.workunit.client.1.smithi097.stdout:0/744: mknod da/d7e/dc4/dfc/d47/c109 0 2024-09-16T08:57:16.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/699: rename d8/dd/d15/d2c/d4a/d4d/d62/c94 to d8/dd/cd7 0 2024-09-16T08:57:16.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/700: creat d8/dd/d15/d16/d7c/da2/fd8 x:0 0 0 2024-09-16T08:57:16.785 INFO:tasks.workunit.client.1.smithi097.stdout:5/807: dread db/d1a/f5b [0,4194304] 0 2024-09-16T08:57:16.785 INFO:tasks.workunit.client.1.smithi097.stdout:5/808: chown db/d1a/d6d/c71 0 1 2024-09-16T08:57:16.786 INFO:tasks.workunit.client.0.smithi031.stdout:0/308: rmdir d6/d12/d13 39 2024-09-16T08:57:16.786 INFO:tasks.workunit.client.1.smithi097.stdout:3/768: getdents d7/d28/d63/dd8 0 2024-09-16T08:57:16.787 INFO:tasks.workunit.client.1.smithi097.stdout:3/769: fsync d7/d28/d30/f53 0 2024-09-16T08:57:16.787 INFO:tasks.workunit.client.1.smithi097.stdout:3/770: chown d7/de/d6c/f5a 631462716 1 2024-09-16T08:57:16.787 INFO:tasks.workunit.client.1.smithi097.stdout:3/771: write d7/de/d3c/dde/ff0 [446233,41788] 0 2024-09-16T08:57:16.787 INFO:tasks.workunit.client.1.smithi097.stdout:3/772: fsync d7/de/d3c/d4f/f80 0 2024-09-16T08:57:16.787 INFO:tasks.workunit.client.1.smithi097.stdout:3/773: chown d7/de/d3c/d3f/d60/dcb/d35/d3d/c58 10 1 2024-09-16T08:57:16.788 INFO:tasks.workunit.client.0.smithi031.stdout:5/377: rename d2/d4b/f51 to d2/dc/d3d/d4c/f7a 0 2024-09-16T08:57:16.789 INFO:tasks.workunit.client.0.smithi031.stdout:7/417: truncate d2/d8/f2f 3372182 0 2024-09-16T08:57:16.792 INFO:tasks.workunit.client.0.smithi031.stdout:1/285: rename d2/d4/d1b/c37 to d2/d4/d1b/d5c/c64 0 2024-09-16T08:57:16.792 INFO:tasks.workunit.client.0.smithi031.stdout:1/286: creat d2/d4/d1b/f65 x:0 0 0 2024-09-16T08:57:16.793 INFO:tasks.workunit.client.1.smithi097.stdout:8/701: mknod d8/dd/d15/d2c/d4a/d4e/d97/cd9 0 2024-09-16T08:57:16.793 INFO:tasks.workunit.client.1.smithi097.stdout:8/702: write d8/dd/d15/d2c/f4b [2711600,87253] 0 2024-09-16T08:57:16.793 INFO:tasks.workunit.client.0.smithi031.stdout:5/378: symlink d2/dc/d1d/l7b 0 2024-09-16T08:57:16.794 INFO:tasks.workunit.client.0.smithi031.stdout:5/379: chown d2/dc/d3d/l3e 336 1 2024-09-16T08:57:16.794 INFO:tasks.workunit.client.0.smithi031.stdout:5/380: truncate d2/dc/d36/f52 4532862 0 2024-09-16T08:57:16.794 INFO:tasks.workunit.client.0.smithi031.stdout:5/381: chown d2/dc/d3d 21819 1 2024-09-16T08:57:16.794 INFO:tasks.workunit.client.1.smithi097.stdout:5/809: link db/d1a/f3a db/d22/d58/f10b 0 2024-09-16T08:57:16.795 INFO:tasks.workunit.client.0.smithi031.stdout:5/382: dread d2/dc/d36/f52 [4194304,4194304] 0 2024-09-16T08:57:16.795 INFO:tasks.workunit.client.0.smithi031.stdout:5/383: truncate d2/dc/d36/f41 97643 0 2024-09-16T08:57:16.795 INFO:tasks.workunit.client.0.smithi031.stdout:5/384: stat d2/dc/d25 0 2024-09-16T08:57:16.795 INFO:tasks.workunit.client.1.smithi097.stdout:3/774: truncate d7/de/d3c/f4d 1021447 0 2024-09-16T08:57:16.797 INFO:tasks.workunit.client.0.smithi031.stdout:0/309: rename d6/f31 to d6/d12/d47/d5b/f67 0 2024-09-16T08:57:16.797 INFO:tasks.workunit.client.0.smithi031.stdout:0/310: readlink d6/l25 0 2024-09-16T08:57:16.797 INFO:tasks.workunit.client.1.smithi097.stdout:8/703: creat d8/dd/d15/d2c/d4a/d4d/d3d/db9/fda x:0 0 0 2024-09-16T08:57:16.798 INFO:tasks.workunit.client.1.smithi097.stdout:0/745: read da/d1a/f49 [3674686,90427] 0 2024-09-16T08:57:16.798 INFO:tasks.workunit.client.1.smithi097.stdout:0/746: stat da/d7e/dc4/dfc/d47/d97/d84/l9e 0 2024-09-16T08:57:16.798 INFO:tasks.workunit.client.1.smithi097.stdout:3/775: unlink d7/d15/d52/d81/f10b 0 2024-09-16T08:57:16.799 INFO:tasks.workunit.client.0.smithi031.stdout:1/287: creat d2/d4/d1b/d5c/f66 x:0 0 0 2024-09-16T08:57:16.800 INFO:tasks.workunit.client.1.smithi097.stdout:3/776: dread d7/d28/d97/fd3 [0,4194304] 0 2024-09-16T08:57:16.800 INFO:tasks.workunit.client.1.smithi097.stdout:3/777: stat d7/de/d8d/d90/d92/fdd 0 2024-09-16T08:57:16.800 INFO:tasks.workunit.client.1.smithi097.stdout:0/747: mknod da/c10a 0 2024-09-16T08:57:16.801 INFO:tasks.workunit.client.0.smithi031.stdout:5/385: symlink d2/dc/l7c 0 2024-09-16T08:57:16.801 INFO:tasks.workunit.client.0.smithi031.stdout:7/418: dread d2/f1a [0,4194304] 0 2024-09-16T08:57:16.801 INFO:tasks.workunit.client.1.smithi097.stdout:8/704: dread d8/dd/d15/d2c/d4a/d4d/d62/f76 [0,4194304] 0 2024-09-16T08:57:16.801 INFO:tasks.workunit.client.1.smithi097.stdout:8/705: dread - d8/dd/d71/d9c/fca zero size 2024-09-16T08:57:16.801 INFO:tasks.workunit.client.1.smithi097.stdout:8/706: write d8/dd/d15/d2c/d4a/d4d/d62/fc2 [296027,81129] 0 2024-09-16T08:57:16.802 INFO:tasks.workunit.client.0.smithi031.stdout:0/311: creat d6/d12/d13/f68 x:0 0 0 2024-09-16T08:57:16.802 INFO:tasks.workunit.client.1.smithi097.stdout:3/778: rename d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/dc3 to d7/de/d8d/d90/d92/d10c 0 2024-09-16T08:57:16.804 INFO:tasks.workunit.client.0.smithi031.stdout:7/419: rename d2/d8/f32 to d2/f92 0 2024-09-16T08:57:16.804 INFO:tasks.workunit.client.0.smithi031.stdout:7/420: truncate d2/d8/d11/f31 3376900 0 2024-09-16T08:57:16.805 INFO:tasks.workunit.client.0.smithi031.stdout:7/421: chown d2/d8/d11/l19 113 1 2024-09-16T08:57:16.805 INFO:tasks.workunit.client.1.smithi097.stdout:0/748: rmdir da/d7e/dc4/dfc/d47/db3/d60/d73/d85 39 2024-09-16T08:57:16.806 INFO:tasks.workunit.client.1.smithi097.stdout:8/707: stat d8/dd/d15/d2c/d4a/d4e/c67 0 2024-09-16T08:57:16.808 INFO:tasks.workunit.client.1.smithi097.stdout:0/749: symlink da/d7e/l10b 0 2024-09-16T08:57:16.809 INFO:tasks.workunit.client.1.smithi097.stdout:8/708: symlink d8/dd/d15/d2c/ldb 0 2024-09-16T08:57:16.810 INFO:tasks.workunit.client.1.smithi097.stdout:8/709: fsync d8/dd/d15/d2c/d4a/d4d/fbe 0 2024-09-16T08:57:16.810 INFO:tasks.workunit.client.1.smithi097.stdout:8/710: creat d8/dd/d15/d16/d7c/da2/fdc x:0 0 0 2024-09-16T08:57:16.810 INFO:tasks.workunit.client.1.smithi097.stdout:5/810: write db/d22/f6a [1043184,53057] 0 2024-09-16T08:57:16.810 INFO:tasks.workunit.client.1.smithi097.stdout:5/811: fsync db/f56 0 2024-09-16T08:57:16.810 INFO:tasks.workunit.client.1.smithi097.stdout:5/812: read db/d6e/d95/fa3 [1281538,20329] 0 2024-09-16T08:57:16.810 INFO:tasks.workunit.client.1.smithi097.stdout:5/813: getdents db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f 0 2024-09-16T08:57:16.810 INFO:tasks.workunit.client.0.smithi031.stdout:7/422: dread d2/d8/d11/f31 [0,4194304] 0 2024-09-16T08:57:16.817 INFO:tasks.workunit.client.0.smithi031.stdout:5/386: dread d2/dc/d25/d21/f6d [0,4194304] 0 2024-09-16T08:57:16.817 INFO:tasks.workunit.client.0.smithi031.stdout:5/387: creat d2/f7d x:0 0 0 2024-09-16T08:57:16.817 INFO:tasks.workunit.client.1.smithi097.stdout:5/814: creat db/d22/f10c x:0 0 0 2024-09-16T08:57:16.818 INFO:tasks.workunit.client.1.smithi097.stdout:9/707: dwrite d2/d5/d56/f8a [0,4194304] 0 2024-09-16T08:57:16.818 INFO:tasks.workunit.client.1.smithi097.stdout:5/815: creat db/d6e/d79/dba/f10d x:0 0 0 2024-09-16T08:57:16.818 INFO:tasks.workunit.client.1.smithi097.stdout:6/765: dwrite d1/dd3/d9e/dca/fea [0,4194304] 0 2024-09-16T08:57:16.819 INFO:tasks.workunit.client.1.smithi097.stdout:9/708: dread d2/d5/d3a/d3c/d73/f7b [0,4194304] 0 2024-09-16T08:57:16.819 INFO:tasks.workunit.client.1.smithi097.stdout:9/709: truncate d2/d5/d56/d6b/dcd/dd8/fe2 365515 0 2024-09-16T08:57:16.819 INFO:tasks.workunit.client.1.smithi097.stdout:9/710: chown d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/c7e 3819820 1 2024-09-16T08:57:16.823 INFO:tasks.workunit.client.0.smithi031.stdout:5/388: rename d2/dc/d25/d21/f73 to d2/dc/d3d/f7e 0 2024-09-16T08:57:16.823 INFO:tasks.workunit.client.0.smithi031.stdout:5/389: stat d2/dc/d3d/d4c/f62 0 2024-09-16T08:57:16.823 INFO:tasks.workunit.client.0.smithi031.stdout:5/390: read - d2/dc/d3d/f74 zero size 2024-09-16T08:57:16.824 INFO:tasks.workunit.client.1.smithi097.stdout:7/669: dwrite d4/f5 [0,4194304] 0 2024-09-16T08:57:16.824 INFO:tasks.workunit.client.1.smithi097.stdout:7/670: stat d4/d9/c13 0 2024-09-16T08:57:16.824 INFO:tasks.workunit.client.1.smithi097.stdout:7/671: read - d4/d9/dc/d8e/fde zero size 2024-09-16T08:57:16.824 INFO:tasks.workunit.client.1.smithi097.stdout:7/672: chown d4/d7/d79/f82 482518 1 2024-09-16T08:57:16.825 INFO:tasks.workunit.client.1.smithi097.stdout:5/816: write db/d18/f68 [246755,48451] 0 2024-09-16T08:57:16.825 INFO:tasks.workunit.client.1.smithi097.stdout:5/817: chown db/d18/d1c/fad 1386058 1 2024-09-16T08:57:16.827 INFO:tasks.workunit.client.0.smithi031.stdout:4/271: dwrite d0/d2/d16/f30 [0,4194304] 0 2024-09-16T08:57:16.827 INFO:tasks.workunit.client.0.smithi031.stdout:3/433: dwrite d3/f59 [0,4194304] 0 2024-09-16T08:57:16.831 INFO:tasks.workunit.client.0.smithi031.stdout:9/308: dwrite f2 [0,4194304] 0 2024-09-16T08:57:16.834 INFO:tasks.workunit.client.0.smithi031.stdout:5/391: write d2/f1a [1144706,90193] 0 2024-09-16T08:57:16.836 INFO:tasks.workunit.client.0.smithi031.stdout:3/434: creat d3/d14/d37/d1a/d1d/d1e/f80 x:0 0 0 2024-09-16T08:57:16.836 INFO:tasks.workunit.client.0.smithi031.stdout:3/435: creat d3/d14/d37/d1a/d1d/d1e/f81 x:0 0 0 2024-09-16T08:57:16.838 INFO:tasks.workunit.client.1.smithi097.stdout:1/850: dwrite d5/d70/d9d/fd0 [0,4194304] 0 2024-09-16T08:57:16.840 INFO:tasks.workunit.client.0.smithi031.stdout:8/253: dwrite d4/d7/f1c [0,4194304] 0 2024-09-16T08:57:16.840 INFO:tasks.workunit.client.0.smithi031.stdout:4/272: dread d0/d2/d16/d1e/f34 [0,4194304] 0 2024-09-16T08:57:16.841 INFO:tasks.workunit.client.0.smithi031.stdout:1/288: dwrite d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:16.843 INFO:tasks.workunit.client.1.smithi097.stdout:7/673: dread d4/d7/d30/ddf/f9b [0,4194304] 0 2024-09-16T08:57:16.843 INFO:tasks.workunit.client.1.smithi097.stdout:6/766: link d1/dd3/d9e/dca/d16/d4f/c7e d1/dd3/db0/cfc 0 2024-09-16T08:57:16.844 INFO:tasks.workunit.client.1.smithi097.stdout:3/779: dwrite d7/de/d3c/d3f/d60/fe1 [0,4194304] 0 2024-09-16T08:57:16.844 INFO:tasks.workunit.client.1.smithi097.stdout:3/780: creat d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa/f10d x:0 0 0 2024-09-16T08:57:16.844 INFO:tasks.workunit.client.0.smithi031.stdout:7/423: dwrite d2/d8/d69/f6c [0,4194304] 0 2024-09-16T08:57:16.846 INFO:tasks.workunit.client.1.smithi097.stdout:0/750: dwrite da/d1a/d41/db2/fd6 [0,4194304] 0 2024-09-16T08:57:16.846 INFO:tasks.workunit.client.1.smithi097.stdout:0/751: fsync da/d1a/d9c/fb9 0 2024-09-16T08:57:16.846 INFO:tasks.workunit.client.1.smithi097.stdout:0/752: chown da/d7e/dc4/dfc/d47/d97/fa2 11568 1 2024-09-16T08:57:16.848 INFO:tasks.workunit.client.0.smithi031.stdout:5/392: unlink d2/f1c 0 2024-09-16T08:57:16.850 INFO:tasks.workunit.client.1.smithi097.stdout:0/753: dread da/d1a/f58 [0,4194304] 0 2024-09-16T08:57:16.850 INFO:tasks.workunit.client.1.smithi097.stdout:9/711: creat d2/d5/db0/dc8/d63/da8/fe5 x:0 0 0 2024-09-16T08:57:16.856 INFO:tasks.workunit.client.0.smithi031.stdout:3/436: mkdir d3/d14/d6f/d82 0 2024-09-16T08:57:16.858 INFO:tasks.workunit.client.1.smithi097.stdout:7/674: dread d4/d9/dc/f90 [0,4194304] 0 2024-09-16T08:57:16.867 INFO:tasks.workunit.client.1.smithi097.stdout:1/851: mknod d5/c111 0 2024-09-16T08:57:16.867 INFO:tasks.workunit.client.1.smithi097.stdout:1/852: fdatasync d5/d70/d9d/fd0 0 2024-09-16T08:57:16.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/309: dwrite f2 [0,4194304] 0 2024-09-16T08:57:16.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/310: dread - d3/d5/da/db/d21/f41 zero size 2024-09-16T08:57:16.875 INFO:tasks.workunit.client.0.smithi031.stdout:8/254: dwrite d4/d7/d36/d38/f3c [0,4194304] 0 2024-09-16T08:57:16.875 INFO:tasks.workunit.client.0.smithi031.stdout:8/255: chown d4/d7/d21/d35/d40 2 1 2024-09-16T08:57:16.875 INFO:tasks.workunit.client.0.smithi031.stdout:1/289: dwrite d2/d15/d32/f55 [0,4194304] 0 2024-09-16T08:57:16.875 INFO:tasks.workunit.client.1.smithi097.stdout:6/767: rename d1/dd3/d9e/dca/d81/dad/df5 to d1/db7/dfd 0 2024-09-16T08:57:16.876 INFO:tasks.workunit.client.0.smithi031.stdout:5/393: dwrite d2/dc/d25/d21/d2e/f72 [0,4194304] 0 2024-09-16T08:57:16.876 INFO:tasks.workunit.client.0.smithi031.stdout:5/394: creat d2/f7f x:0 0 0 2024-09-16T08:57:16.883 INFO:tasks.workunit.client.1.smithi097.stdout:8/711: getdents d8/dd/d15/d2c/d4a/d4d/d62 0 2024-09-16T08:57:16.883 INFO:tasks.workunit.client.1.smithi097.stdout:8/712: fdatasync d8/dd/d15/d2c/d4a/d4d/d3d/db9/fda 0 2024-09-16T08:57:16.885 INFO:tasks.workunit.client.1.smithi097.stdout:3/781: creat d7/d28/d30/d88/df7/f10e x:0 0 0 2024-09-16T08:57:16.887 INFO:tasks.workunit.client.0.smithi031.stdout:8/256: write d4/d2d/f3a [3017119,14127] 0 2024-09-16T08:57:16.895 INFO:tasks.workunit.client.0.smithi031.stdout:5/395: dwrite d2/dc/d36/f38 [0,4194304] 0 2024-09-16T08:57:16.896 INFO:tasks.workunit.client.1.smithi097.stdout:0/754: symlink da/d1a/d9c/l10c 0 2024-09-16T08:57:16.897 INFO:tasks.workunit.client.0.smithi031.stdout:4/273: rename d0/d2/d16/d50 to d0/d2/d3a/d54 0 2024-09-16T08:57:16.899 INFO:tasks.workunit.client.1.smithi097.stdout:9/712: unlink d2/d5/db0/dc8/d74/d8c/d94/d96/dac/f87 0 2024-09-16T08:57:16.899 INFO:tasks.workunit.client.1.smithi097.stdout:9/713: readlink d2/d5/db0/dc8/d74/d8c/d94/d96/dac/l66 0 2024-09-16T08:57:16.902 INFO:tasks.workunit.client.1.smithi097.stdout:5/818: getdents db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf 0 2024-09-16T08:57:16.905 INFO:tasks.workunit.client.0.smithi031.stdout:0/312: fsync d6/d12/d47/d5b/f67 0 2024-09-16T08:57:16.905 INFO:tasks.workunit.client.0.smithi031.stdout:0/313: fdatasync d6/d12/f16 0 2024-09-16T08:57:16.905 INFO:tasks.workunit.client.1.smithi097.stdout:8/713: dread d8/dd/d15/d2c/d4a/d4e/db2/fb3 [0,4194304] 0 2024-09-16T08:57:16.909 INFO:tasks.workunit.client.1.smithi097.stdout:1/853: truncate d5/fd9 82858 0 2024-09-16T08:57:16.910 INFO:tasks.workunit.client.0.smithi031.stdout:0/314: dread d6/d12/d13/f21 [0,4194304] 0 2024-09-16T08:57:16.910 INFO:tasks.workunit.client.0.smithi031.stdout:0/315: chown d6/d12/f22 188653515 1 2024-09-16T08:57:16.910 INFO:tasks.workunit.client.1.smithi097.stdout:7/675: rename d4/d7/d79/cbe to d4/d7/d30/ce3 0 2024-09-16T08:57:16.911 INFO:tasks.workunit.client.1.smithi097.stdout:6/768: mknod d1/cfe 0 2024-09-16T08:57:16.911 INFO:tasks.workunit.client.1.smithi097.stdout:6/769: chown d1/dd3/d9e/dca/d81/dad/c5e 191 1 2024-09-16T08:57:16.912 INFO:tasks.workunit.client.1.smithi097.stdout:3/782: dwrite d7/d28/fd4 [4194304,4194304] 0 2024-09-16T08:57:16.916 INFO:tasks.workunit.client.1.smithi097.stdout:9/714: dread d2/f28 [0,4194304] 0 2024-09-16T08:57:16.916 INFO:tasks.workunit.client.1.smithi097.stdout:9/715: truncate d2/d5/f69 317788 0 2024-09-16T08:57:16.919 INFO:tasks.workunit.client.1.smithi097.stdout:9/716: dread d2/f4 [0,4194304] 0 2024-09-16T08:57:16.919 INFO:tasks.workunit.client.1.smithi097.stdout:9/717: write d2/d5/f11 [3104396,68142] 0 2024-09-16T08:57:16.925 INFO:tasks.workunit.client.1.smithi097.stdout:0/755: dread da/d24/d4c/d80/d3b/f4e [0,4194304] 0 2024-09-16T08:57:16.925 INFO:tasks.workunit.client.1.smithi097.stdout:0/756: write da/d24/d4c/d80/d3b/f5a [871868,108803] 0 2024-09-16T08:57:16.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/783: write d7/de/d3c/dcc/ff3 [3164995,65110] 0 2024-09-16T08:57:16.925 INFO:tasks.workunit.client.1.smithi097.stdout:3/784: truncate d7/de/d3c/d4f/f7c 1441133 0 2024-09-16T08:57:16.926 INFO:tasks.workunit.client.1.smithi097.stdout:3/785: creat d7/de/d9c/daf/f10f x:0 0 0 2024-09-16T08:57:16.927 INFO:tasks.workunit.client.1.smithi097.stdout:8/714: mkdir d8/dd/d15/d2c/d4a/d4d/ddd 0 2024-09-16T08:57:16.930 INFO:tasks.workunit.client.1.smithi097.stdout:1/854: truncate d5/d14/d95/fa0 1136713 0 2024-09-16T08:57:16.932 INFO:tasks.workunit.client.1.smithi097.stdout:6/770: stat d1/dd3/cbc 0 2024-09-16T08:57:16.933 INFO:tasks.workunit.client.0.smithi031.stdout:8/257: dwrite d4/d2d/f3a [0,4194304] 0 2024-09-16T08:57:16.934 INFO:tasks.workunit.client.1.smithi097.stdout:1/855: read d5/d70/fcc [1333006,6802] 0 2024-09-16T08:57:16.934 INFO:tasks.workunit.client.1.smithi097.stdout:1/856: write d5/d14/d95/fb8 [173426,110768] 0 2024-09-16T08:57:16.934 INFO:tasks.workunit.client.1.smithi097.stdout:1/857: readlink d5/db/l50 0 2024-09-16T08:57:16.934 INFO:tasks.workunit.client.1.smithi097.stdout:1/858: chown d5/d2c/dcb/f8d 698331496 1 2024-09-16T08:57:16.935 INFO:tasks.workunit.client.1.smithi097.stdout:1/859: truncate d5/db/dc/fde 723123 0 2024-09-16T08:57:16.935 INFO:tasks.workunit.client.1.smithi097.stdout:1/860: write d5/d2c/f5e [953542,89390] 0 2024-09-16T08:57:16.935 INFO:tasks.workunit.client.1.smithi097.stdout:1/861: chown d5/d2c/dcb/l31 19152884 1 2024-09-16T08:57:16.940 INFO:tasks.workunit.client.1.smithi097.stdout:5/819: dwrite db/fe [0,4194304] 0 2024-09-16T08:57:16.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/396: symlink d2/l80 0 2024-09-16T08:57:16.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/397: readlink d2/dc/d25/d21/d2e/l5f 0 2024-09-16T08:57:16.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/398: chown d2/dc/d25/f53 279808 1 2024-09-16T08:57:16.947 INFO:tasks.workunit.client.0.smithi031.stdout:4/274: mkdir d0/d2/d3a/d55 0 2024-09-16T08:57:16.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/424: rename d2/d8/d11/l4d to d2/d8/d86/l93 0 2024-09-16T08:57:16.950 INFO:tasks.workunit.client.0.smithi031.stdout:7/425: creat d2/f94 x:0 0 0 2024-09-16T08:57:16.951 INFO:tasks.workunit.client.0.smithi031.stdout:0/316: creat d6/f69 x:0 0 0 2024-09-16T08:57:16.953 INFO:tasks.workunit.client.1.smithi097.stdout:6/771: rename d1/db7/ffa to d1/dd3/d9e/dca/d16/d86/fff 0 2024-09-16T08:57:16.954 INFO:tasks.workunit.client.0.smithi031.stdout:1/290: mkdir d2/d4/d67 0 2024-09-16T08:57:16.955 INFO:tasks.workunit.client.1.smithi097.stdout:1/862: link d5/d17/d54/d6b/da8/dc5/lfb d5/ddd/l112 0 2024-09-16T08:57:16.955 INFO:tasks.workunit.client.1.smithi097.stdout:1/863: readlink d5/db/dc/l6d 0 2024-09-16T08:57:16.956 INFO:tasks.workunit.client.1.smithi097.stdout:1/864: chown d5/db/dc/d19/l9e 459 1 2024-09-16T08:57:16.957 INFO:tasks.workunit.client.1.smithi097.stdout:7/676: dwrite d4/d9/fda [0,4194304] 0 2024-09-16T08:57:16.957 INFO:tasks.workunit.client.0.smithi031.stdout:4/275: rename d0/c2a to d0/d2/d16/d1e/c56 0 2024-09-16T08:57:16.959 INFO:tasks.workunit.client.1.smithi097.stdout:5/820: mknod db/d18/d1c/d42/d46/d7e/c10e 0 2024-09-16T08:57:16.962 INFO:tasks.workunit.client.0.smithi031.stdout:0/317: creat d6/d12/f6a x:0 0 0 2024-09-16T08:57:16.962 INFO:tasks.workunit.client.0.smithi031.stdout:4/276: dread d0/d2/f27 [0,4194304] 0 2024-09-16T08:57:16.962 INFO:tasks.workunit.client.0.smithi031.stdout:4/277: stat d0/c42 0 2024-09-16T08:57:16.962 INFO:tasks.workunit.client.0.smithi031.stdout:4/278: readlink d0/d2/d16/d1e/l3e 0 2024-09-16T08:57:16.963 INFO:tasks.workunit.client.0.smithi031.stdout:8/258: dwrite d4/d7/d21/f3b [0,4194304] 0 2024-09-16T08:57:16.964 INFO:tasks.workunit.client.0.smithi031.stdout:5/399: dwrite d2/dc/d25/d21/f26 [4194304,4194304] 0 2024-09-16T08:57:16.965 INFO:tasks.workunit.client.1.smithi097.stdout:9/718: dwrite d2/d5/d3a/d3c/f5f [0,4194304] 0 2024-09-16T08:57:16.965 INFO:tasks.workunit.client.1.smithi097.stdout:9/719: chown d2/d5/d3a/fa4 39 1 2024-09-16T08:57:16.965 INFO:tasks.workunit.client.1.smithi097.stdout:9/720: chown d2/d5/db0/dc8/f5a 22122422 1 2024-09-16T08:57:16.965 INFO:tasks.workunit.client.1.smithi097.stdout:0/757: dwrite da/d7e/dc4/dfc/d94/d101/f66 [0,4194304] 0 2024-09-16T08:57:16.965 INFO:tasks.workunit.client.1.smithi097.stdout:7/677: dread d4/d7/d30/f5c [0,4194304] 0 2024-09-16T08:57:16.966 INFO:tasks.workunit.client.0.smithi031.stdout:1/291: symlink d2/d15/d5d/l68 0 2024-09-16T08:57:16.966 INFO:tasks.workunit.client.0.smithi031.stdout:1/292: readlink d2/d15/d5d/l48 0 2024-09-16T08:57:16.966 INFO:tasks.workunit.client.0.smithi031.stdout:1/293: chown d2/d4/d1b 46325 1 2024-09-16T08:57:16.966 INFO:tasks.workunit.client.1.smithi097.stdout:7/678: write d4/d1b/f69 [255075,79772] 0 2024-09-16T08:57:16.968 INFO:tasks.workunit.client.1.smithi097.stdout:7/679: dread d4/d9/d2b/fa9 [0,4194304] 0 2024-09-16T08:57:16.970 INFO:tasks.workunit.client.0.smithi031.stdout:4/279: dread d0/f38 [0,4194304] 0 2024-09-16T08:57:16.971 INFO:tasks.workunit.client.1.smithi097.stdout:0/758: dread da/d24/d4c/d80/d3b/f6a [0,4194304] 0 2024-09-16T08:57:16.977 INFO:tasks.workunit.client.1.smithi097.stdout:5/821: dwrite db/d18/d1c/d42/d46/d7e/d64/ff6 [0,4194304] 0 2024-09-16T08:57:16.981 INFO:tasks.workunit.client.0.smithi031.stdout:0/318: rmdir d6/d12/d13 39 2024-09-16T08:57:16.986 INFO:tasks.workunit.client.0.smithi031.stdout:4/280: symlink d0/d2/d16/d2b/l57 0 2024-09-16T08:57:16.988 INFO:tasks.workunit.client.0.smithi031.stdout:0/319: dread d6/d12/d13/d36/f51 [0,4194304] 0 2024-09-16T08:57:16.988 INFO:tasks.workunit.client.0.smithi031.stdout:1/294: link d2/d15/d1f/c51 d2/d4/c69 0 2024-09-16T08:57:16.989 INFO:tasks.workunit.client.0.smithi031.stdout:1/295: truncate d2/d4/d1b/d5c/f66 402255 0 2024-09-16T08:57:16.990 INFO:tasks.workunit.client.1.smithi097.stdout:6/772: rename d1/dd3/d9e/dca/d81/f93 to d1/dd3/d9e/f100 0 2024-09-16T08:57:16.992 INFO:tasks.workunit.client.1.smithi097.stdout:1/865: creat d5/d17/d54/f113 x:0 0 0 2024-09-16T08:57:16.993 INFO:tasks.workunit.client.1.smithi097.stdout:8/715: rmdir d8/dd/d15/d2c/d72 0 2024-09-16T08:57:16.996 INFO:tasks.workunit.client.1.smithi097.stdout:9/721: mkdir d2/d5/d72/dc1/de6 0 2024-09-16T08:57:16.997 INFO:tasks.workunit.client.0.smithi031.stdout:0/320: dread d6/f8 [0,4194304] 0 2024-09-16T08:57:16.997 INFO:tasks.workunit.client.0.smithi031.stdout:0/321: creat d6/d61/f6b x:0 0 0 2024-09-16T08:57:16.997 INFO:tasks.workunit.client.0.smithi031.stdout:0/322: readlink d6/d12/l2b 0 2024-09-16T08:57:16.997 INFO:tasks.workunit.client.0.smithi031.stdout:0/323: read d6/d12/d13/f23 [2006411,117918] 0 2024-09-16T08:57:16.999 INFO:tasks.workunit.client.1.smithi097.stdout:0/759: mknod da/d24/d76/c10d 0 2024-09-16T08:57:16.999 INFO:tasks.workunit.client.0.smithi031.stdout:2/246: sync 2024-09-16T08:57:17.000 INFO:tasks.workunit.client.0.smithi031.stdout:2/247: dread - d6/d2e/f2f zero size 2024-09-16T08:57:17.001 INFO:tasks.workunit.client.1.smithi097.stdout:7/680: dwrite d4/d7/d19/f71 [0,4194304] 0 2024-09-16T08:57:17.002 INFO:tasks.workunit.client.1.smithi097.stdout:5/822: mknod db/d1a/d6d/dc6/c10f 0 2024-09-16T08:57:17.008 INFO:tasks.workunit.client.1.smithi097.stdout:7/681: dread d4/d9/dc/f49 [0,4194304] 0 2024-09-16T08:57:17.008 INFO:tasks.workunit.client.0.smithi031.stdout:8/259: dwrite d4/d7/f51 [0,4194304] 0 2024-09-16T08:57:17.010 INFO:tasks.workunit.client.1.smithi097.stdout:8/716: dread d8/dd/d15/d2c/d4a/d4d/f35 [0,4194304] 0 2024-09-16T08:57:17.010 INFO:tasks.workunit.client.0.smithi031.stdout:2/248: mkdir d6/d20/d51 0 2024-09-16T08:57:17.011 INFO:tasks.workunit.client.0.smithi031.stdout:2/249: dread d6/d20/f32 [0,4194304] 0 2024-09-16T08:57:17.011 INFO:tasks.workunit.client.1.smithi097.stdout:0/760: mkdir da/d7e/dc4/dfc/d47/db3/d60/d68/d10e 0 2024-09-16T08:57:17.012 INFO:tasks.workunit.client.1.smithi097.stdout:0/761: chown da/df/dd8 9906 1 2024-09-16T08:57:17.014 INFO:tasks.workunit.client.0.smithi031.stdout:4/281: dwrite d0/d2/d16/d1e/f3b [0,4194304] 0 2024-09-16T08:57:17.014 INFO:tasks.workunit.client.0.smithi031.stdout:4/282: chown d0/d2/d3a/c53 12863019 1 2024-09-16T08:57:17.016 INFO:tasks.workunit.client.1.smithi097.stdout:6/773: dwrite d1/db7/dfd/f47 [0,4194304] 0 2024-09-16T08:57:17.021 INFO:tasks.workunit.client.0.smithi031.stdout:8/260: rmdir d4/d7/d21/d37/d46 39 2024-09-16T08:57:17.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/682: rename d4/d9/dc/d8e/lc8 to d4/d7/d7b/le4 0 2024-09-16T08:57:17.021 INFO:tasks.workunit.client.1.smithi097.stdout:7/683: read d4/d7/f11 [2499210,33899] 0 2024-09-16T08:57:17.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/762: symlink da/d24/d4b/dfb/l10f 0 2024-09-16T08:57:17.022 INFO:tasks.workunit.client.1.smithi097.stdout:0/763: chown da/d7e/dc4/dfc/d47/db3/dbf 1204 1 2024-09-16T08:57:17.023 INFO:tasks.workunit.client.0.smithi031.stdout:2/250: mknod d6/d20/c52 0 2024-09-16T08:57:17.024 INFO:tasks.workunit.client.1.smithi097.stdout:2/784: sync 2024-09-16T08:57:17.024 INFO:tasks.workunit.client.1.smithi097.stdout:4/710: sync 2024-09-16T08:57:17.025 INFO:tasks.workunit.client.1.smithi097.stdout:6/774: getdents d1/dd3/d9e/dca/dbb 0 2024-09-16T08:57:17.028 INFO:tasks.workunit.client.1.smithi097.stdout:7/684: mknod d4/d9/d1c/ce5 0 2024-09-16T08:57:17.029 INFO:tasks.workunit.client.0.smithi031.stdout:4/283: truncate d0/d2/d16/d1e/f52 832011 0 2024-09-16T08:57:17.032 INFO:tasks.workunit.client.1.smithi097.stdout:0/764: truncate da/d1a/d9c/fb9 103515 0 2024-09-16T08:57:17.032 INFO:tasks.workunit.client.1.smithi097.stdout:0/765: chown da/d1a/d9c/l10c 7156650 1 2024-09-16T08:57:17.032 INFO:tasks.workunit.client.0.smithi031.stdout:8/261: truncate d4/d7/d21/d24/f31 2214795 0 2024-09-16T08:57:17.035 INFO:tasks.workunit.client.1.smithi097.stdout:4/711: rename d0/d7/d52/d26/d3d/feb to d0/d7/d52/d26/d8c/db0/ffe 0 2024-09-16T08:57:17.035 INFO:tasks.workunit.client.1.smithi097.stdout:4/712: stat d0/d23/fd0 0 2024-09-16T08:57:17.035 INFO:tasks.workunit.client.1.smithi097.stdout:4/713: readlink d0/d39/d71/l5a 0 2024-09-16T08:57:17.037 INFO:tasks.workunit.client.0.smithi031.stdout:2/251: unlink d6/c1a 0 2024-09-16T08:57:17.039 INFO:tasks.workunit.client.1.smithi097.stdout:6/775: mknod d1/dd3/d9e/db3/c101 0 2024-09-16T08:57:17.040 INFO:tasks.workunit.client.1.smithi097.stdout:6/776: write d1/db7/dfd/f83 [184248,30240] 0 2024-09-16T08:57:17.040 INFO:tasks.workunit.client.1.smithi097.stdout:6/777: read d1/dd3/d9e/dca/d81/f88 [3445435,105466] 0 2024-09-16T08:57:17.040 INFO:tasks.workunit.client.1.smithi097.stdout:6/778: chown d1/dd3/cbc 407401144 1 2024-09-16T08:57:17.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/685: mknod d4/d7/d7b/d43/daf/ce6 0 2024-09-16T08:57:17.044 INFO:tasks.workunit.client.1.smithi097.stdout:7/686: chown d4/d1b/c2f 1609 1 2024-09-16T08:57:17.044 INFO:tasks.workunit.client.1.smithi097.stdout:2/785: dwrite d3/d4/d8/f25 [0,4194304] 0 2024-09-16T08:57:17.044 INFO:tasks.workunit.client.1.smithi097.stdout:2/786: dread - d3/d4/d8/d51/fc7 zero size 2024-09-16T08:57:17.045 INFO:tasks.workunit.client.0.smithi031.stdout:8/262: creat d4/d7/d21/d37/d46/f56 x:0 0 0 2024-09-16T08:57:17.045 INFO:tasks.workunit.client.0.smithi031.stdout:8/263: stat d4/l1d 0 2024-09-16T08:57:17.045 INFO:tasks.workunit.client.0.smithi031.stdout:8/264: stat d4/l1d 0 2024-09-16T08:57:17.045 INFO:tasks.workunit.client.1.smithi097.stdout:9/722: dwrite d2/f38 [0,4194304] 0 2024-09-16T08:57:17.046 INFO:tasks.workunit.client.1.smithi097.stdout:0/766: rename da/d24/d4c/d80/d42/f100 to da/d7e/dc4/f110 0 2024-09-16T08:57:17.049 INFO:tasks.workunit.client.1.smithi097.stdout:4/714: mknod d0/d39/cff 0 2024-09-16T08:57:17.049 INFO:tasks.workunit.client.1.smithi097.stdout:5/823: dwrite db/d18/d1c/d42/d46/f47 [0,4194304] 0 2024-09-16T08:57:17.049 INFO:tasks.workunit.client.1.smithi097.stdout:5/824: fsync db/d18/d1c/d42/d46/f9b 0 2024-09-16T08:57:17.049 INFO:tasks.workunit.client.1.smithi097.stdout:5/825: truncate db/d18/d20/d2b/f7c 4603917 0 2024-09-16T08:57:17.050 INFO:tasks.workunit.client.0.smithi031.stdout:8/265: rename d4/l32 to d4/d7/d21/d24/l57 0 2024-09-16T08:57:17.056 INFO:tasks.workunit.client.0.smithi031.stdout:4/284: getdents d0/d2/d16/d1e 0 2024-09-16T08:57:17.058 INFO:tasks.workunit.client.1.smithi097.stdout:0/767: truncate da/d7e/dc4/dfc/f59 122140 0 2024-09-16T08:57:17.060 INFO:tasks.workunit.client.1.smithi097.stdout:8/717: dwrite d8/f13 [0,4194304] 0 2024-09-16T08:57:17.061 INFO:tasks.workunit.client.1.smithi097.stdout:2/787: rename d3/d4/d68/d8b/d61/dac/cdb to d3/d4/d68/d8b/d61/ced 0 2024-09-16T08:57:17.062 INFO:tasks.workunit.client.0.smithi031.stdout:8/266: dread d4/d2f/f41 [0,4194304] 0 2024-09-16T08:57:17.062 INFO:tasks.workunit.client.0.smithi031.stdout:0/324: dwrite d6/d12/f37 [0,4194304] 0 2024-09-16T08:57:17.062 INFO:tasks.workunit.client.0.smithi031.stdout:0/325: readlink d6/d12/l28 0 2024-09-16T08:57:17.065 INFO:tasks.workunit.client.0.smithi031.stdout:4/285: getdents d0 0 2024-09-16T08:57:17.068 INFO:tasks.workunit.client.1.smithi097.stdout:6/779: rmdir d1/dd8 39 2024-09-16T08:57:17.069 INFO:tasks.workunit.client.1.smithi097.stdout:1/866: dwrite d5/d14/d95/fa0 [0,4194304] 0 2024-09-16T08:57:17.070 INFO:tasks.workunit.client.0.smithi031.stdout:2/252: dwrite d6/d41/d28/f35 [0,4194304] 0 2024-09-16T08:57:17.073 INFO:tasks.workunit.client.1.smithi097.stdout:7/687: dwrite d4/d1b/d41/fb1 [0,4194304] 0 2024-09-16T08:57:17.076 INFO:tasks.workunit.client.1.smithi097.stdout:4/715: rename d0/d7/f89 to d0/d7/d52/dbf/f100 0 2024-09-16T08:57:17.078 INFO:tasks.workunit.client.1.smithi097.stdout:2/788: mknod d3/d4/d68/d8b/d61/dac/cee 0 2024-09-16T08:57:17.078 INFO:tasks.workunit.client.1.smithi097.stdout:7/688: dread d4/d9/d1c/f34 [0,4194304] 0 2024-09-16T08:57:17.080 INFO:tasks.workunit.client.1.smithi097.stdout:6/780: write d1/dd3/d9e/dca/d16/d4f/d9f/fdb [3307396,61024] 0 2024-09-16T08:57:17.081 INFO:tasks.workunit.client.0.smithi031.stdout:4/286: symlink d0/d2/d3a/d55/l58 0 2024-09-16T08:57:17.081 INFO:tasks.workunit.client.0.smithi031.stdout:2/253: dread d6/d20/f3f [0,4194304] 0 2024-09-16T08:57:17.087 INFO:tasks.workunit.client.0.smithi031.stdout:4/287: mkdir d0/d2/d16/d59 0 2024-09-16T08:57:17.087 INFO:tasks.workunit.client.0.smithi031.stdout:4/288: chown d0/d2/d16/f21 3095254 1 2024-09-16T08:57:17.087 INFO:tasks.workunit.client.0.smithi031.stdout:4/289: fdatasync d0/d2/d16/f19 0 2024-09-16T08:57:17.088 INFO:tasks.workunit.client.1.smithi097.stdout:1/867: creat d5/d70/d85/f114 x:0 0 0 2024-09-16T08:57:17.088 INFO:tasks.workunit.client.1.smithi097.stdout:7/689: write d4/f9d [4203435,96261] 0 2024-09-16T08:57:17.089 INFO:tasks.workunit.client.1.smithi097.stdout:7/690: stat d4 0 2024-09-16T08:57:17.089 INFO:tasks.workunit.client.0.smithi031.stdout:2/254: mkdir d6/d41/d2c/d53 0 2024-09-16T08:57:17.089 INFO:tasks.workunit.client.0.smithi031.stdout:2/255: readlink d6/da/l4a 0 2024-09-16T08:57:17.090 INFO:tasks.workunit.client.0.smithi031.stdout:8/267: dwrite d4/d2f/f41 [0,4194304] 0 2024-09-16T08:57:17.092 INFO:tasks.workunit.client.0.smithi031.stdout:4/290: rename d0/c14 to d0/d2/d3a/c5a 0 2024-09-16T08:57:17.092 INFO:tasks.workunit.client.0.smithi031.stdout:4/291: chown d0/d2/l47 4 1 2024-09-16T08:57:17.092 INFO:tasks.workunit.client.1.smithi097.stdout:5/826: rmdir db/d1a 39 2024-09-16T08:57:17.093 INFO:tasks.workunit.client.0.smithi031.stdout:8/268: rmdir d4/d2f 39 2024-09-16T08:57:17.093 INFO:tasks.workunit.client.0.smithi031.stdout:8/269: creat d4/d7/d21/d35/d40/f58 x:0 0 0 2024-09-16T08:57:17.095 INFO:tasks.workunit.client.0.smithi031.stdout:0/326: dwrite d6/f3b [0,4194304] 0 2024-09-16T08:57:17.097 INFO:tasks.workunit.client.0.smithi031.stdout:0/327: dread d6/d12/f17 [0,4194304] 0 2024-09-16T08:57:17.099 INFO:tasks.workunit.client.0.smithi031.stdout:0/328: write d6/d12/d13/f21 [97223,75883] 0 2024-09-16T08:57:17.100 INFO:tasks.workunit.client.0.smithi031.stdout:8/270: link d4/d7/c11 d4/d7/d21/d35/c59 0 2024-09-16T08:57:17.100 INFO:tasks.workunit.client.1.smithi097.stdout:7/691: dread d4/d12/d1f/fbb [4194304,4194304] 0 2024-09-16T08:57:17.100 INFO:tasks.workunit.client.1.smithi097.stdout:7/692: dread - d4/d9/d2b/f2e zero size 2024-09-16T08:57:17.103 INFO:tasks.workunit.client.0.smithi031.stdout:2/256: write d6/d41/d28/f35 [128935,19510] 0 2024-09-16T08:57:17.103 INFO:tasks.workunit.client.0.smithi031.stdout:2/257: truncate d6/d41/d2c/f34 842292 0 2024-09-16T08:57:17.105 INFO:tasks.workunit.client.0.smithi031.stdout:0/329: dread d6/f4f [0,4194304] 0 2024-09-16T08:57:17.109 INFO:tasks.workunit.client.1.smithi097.stdout:4/716: unlink d0/d39/fcd 0 2024-09-16T08:57:17.111 INFO:tasks.workunit.client.1.smithi097.stdout:2/789: symlink d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4/lef 0 2024-09-16T08:57:17.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/330: unlink d6/f42 0 2024-09-16T08:57:17.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/331: write d6/d61/f6b [393242,72499] 0 2024-09-16T08:57:17.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/332: write d6/d12/d13/f46 [628957,20122] 0 2024-09-16T08:57:17.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/333: creat d6/d12/f6c x:0 0 0 2024-09-16T08:57:17.117 INFO:tasks.workunit.client.1.smithi097.stdout:6/781: creat d1/dd3/d9e/dca/d81/dc4/ddc/f102 x:0 0 0 2024-09-16T08:57:17.117 INFO:tasks.workunit.client.1.smithi097.stdout:6/782: dread - d1/db7/dfd/fa2 zero size 2024-09-16T08:57:17.118 INFO:tasks.workunit.client.1.smithi097.stdout:1/868: creat d5/d103/f115 x:0 0 0 2024-09-16T08:57:17.118 INFO:tasks.workunit.client.1.smithi097.stdout:1/869: creat d5/d70/f116 x:0 0 0 2024-09-16T08:57:17.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.121+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09040fc470 con 0x7f08ec073d40 2024-09-16T08:57:17.124 INFO:tasks.workunit.client.1.smithi097.stdout:5/827: mknod db/d18/d1c/d42/d46/d7e/d64/d7f/d89/c110 0 2024-09-16T08:57:17.126 INFO:tasks.workunit.client.1.smithi097.stdout:4/717: mkdir d0/d39/d101 0 2024-09-16T08:57:17.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.125+0000 7f08ebfff640 1 -- 172.21.15.31:0/1629198732 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f09040fc470 con 0x7f08ec073d40 2024-09-16T08:57:17.127 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:57:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:57:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:57:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:57:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-09-16T08:57:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "1/23 daemons upgraded", 2024-09-16T08:57:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-09-16T08:57:17.128 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:57:17.128 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:57:17.130 INFO:tasks.workunit.client.1.smithi097.stdout:5/828: symlink db/d1a/d6d/dce/l111 0 2024-09-16T08:57:17.130 INFO:tasks.workunit.client.1.smithi097.stdout:5/829: fsync db/d18/d1c/d42/d46/d7e/f67 0 2024-09-16T08:57:17.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f08ec073d40 msgr2=0x7f08ec076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:17.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f08ec073d40 0x7f08ec076200 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0904196c60 tx=0x7f08f8006c90 comp rx=0 tx=0).stop 2024-09-16T08:57:17.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 msgr2=0x7f0904195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:17.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f0904195740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005610 tx=0x7f08f4005640 comp rx=0 tx=0).stop 2024-09-16T08:57:17.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 shutdown_connections 2024-09-16T08:57:17.131 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f08ec073d40 0x7f08ec076200 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f09040ffcf0 0x7f0904195c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 --2- 172.21.15.31:0/1629198732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040feaf0 0x7f0904195740 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 >> 172.21.15.31:0/1629198732 conn(0x7f09040fa280 msgr2=0x7f09040fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:17.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 shutdown_connections 2024-09-16T08:57:17.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.129+0000 7f090970b640 1 -- 172.21.15.31:0/1629198732 wait complete. 2024-09-16T08:57:17.132 INFO:tasks.workunit.client.1.smithi097.stdout:4/718: mknod d0/d7/d52/d26/d8c/c102 0 2024-09-16T08:57:17.132 INFO:tasks.workunit.client.1.smithi097.stdout:4/719: read - d0/d7/fe0 zero size 2024-09-16T08:57:17.133 INFO:tasks.workunit.client.1.smithi097.stdout:5/830: truncate db/d18/f68 686 0 2024-09-16T08:57:17.135 INFO:tasks.workunit.client.1.smithi097.stdout:4/720: mknod d0/d7/df/d15/c103 0 2024-09-16T08:57:17.137 INFO:tasks.workunit.client.1.smithi097.stdout:5/831: rmdir db/d1a 39 2024-09-16T08:57:17.138 INFO:tasks.workunit.client.0.smithi031.stdout:2/258: dwrite d6/f2d [0,4194304] 0 2024-09-16T08:57:17.138 INFO:tasks.workunit.client.1.smithi097.stdout:4/721: write d0/d7/d52/f94 [638206,43335] 0 2024-09-16T08:57:17.138 INFO:tasks.workunit.client.1.smithi097.stdout:4/722: dread - d0/d36/d4c/fd7 zero size 2024-09-16T08:57:17.139 INFO:tasks.workunit.client.1.smithi097.stdout:4/723: write d0/d7/f5b [538145,114898] 0 2024-09-16T08:57:17.141 INFO:tasks.workunit.client.1.smithi097.stdout:7/693: dwrite d4/d7/d77/f84 [0,4194304] 0 2024-09-16T08:57:17.141 INFO:tasks.workunit.client.1.smithi097.stdout:4/724: mknod d0/d7/d52/d26/d8c/c104 0 2024-09-16T08:57:17.150 INFO:tasks.workunit.client.0.smithi031.stdout:2/259: write d6/d41/d28/f35 [5117083,31844] 0 2024-09-16T08:57:17.150 INFO:tasks.workunit.client.0.smithi031.stdout:8/271: dwrite d4/d7/d21/d35/d40/f4c [0,4194304] 0 2024-09-16T08:57:17.150 INFO:tasks.workunit.client.1.smithi097.stdout:2/790: dwrite d3/d4/d68/d8b/d61/dac/fd6 [0,4194304] 0 2024-09-16T08:57:17.151 INFO:tasks.workunit.client.1.smithi097.stdout:2/791: write d3/d4/d9/f95 [342098,104820] 0 2024-09-16T08:57:17.151 INFO:tasks.workunit.client.1.smithi097.stdout:2/792: dread - d3/d4/d68/d7f/fbd zero size 2024-09-16T08:57:17.151 INFO:tasks.workunit.client.1.smithi097.stdout:2/793: truncate d3/d1b/da7/fe1 703966 0 2024-09-16T08:57:17.152 INFO:tasks.workunit.client.1.smithi097.stdout:1/870: dwrite d5/db/f5d [0,4194304] 0 2024-09-16T08:57:17.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/260: creat d6/d2e/d40/f54 x:0 0 0 2024-09-16T08:57:17.157 INFO:tasks.workunit.client.0.smithi031.stdout:4/292: getdents d0 0 2024-09-16T08:57:17.158 INFO:tasks.workunit.client.1.smithi097.stdout:2/794: creat d3/d4/d8/d51/ff0 x:0 0 0 2024-09-16T08:57:17.160 INFO:tasks.workunit.client.0.smithi031.stdout:8/272: symlink d4/l5a 0 2024-09-16T08:57:17.161 INFO:tasks.workunit.client.0.smithi031.stdout:0/334: dwrite d6/d12/d13/f23 [0,4194304] 0 2024-09-16T08:57:17.161 INFO:tasks.workunit.client.0.smithi031.stdout:0/335: truncate d6/d12/f2d 509838 0 2024-09-16T08:57:17.165 INFO:tasks.workunit.client.0.smithi031.stdout:2/261: unlink d6/d20/c52 0 2024-09-16T08:57:17.165 INFO:tasks.workunit.client.0.smithi031.stdout:2/262: dread - d6/d2e/d40/f54 zero size 2024-09-16T08:57:17.166 INFO:tasks.workunit.client.1.smithi097.stdout:2/795: mknod d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4/cf1 0 2024-09-16T08:57:17.166 INFO:tasks.workunit.client.1.smithi097.stdout:3/786: sync 2024-09-16T08:57:17.167 INFO:tasks.workunit.client.0.smithi031.stdout:4/293: mkdir d0/d5b 0 2024-09-16T08:57:17.169 INFO:tasks.workunit.client.1.smithi097.stdout:1/871: getdents d5/d17/d54/d6b/da8/dc5/dc7/df3 0 2024-09-16T08:57:17.170 INFO:tasks.workunit.client.0.smithi031.stdout:8/273: mkdir d4/d5b 0 2024-09-16T08:57:17.172 INFO:tasks.workunit.client.1.smithi097.stdout:2/796: symlink d3/d4/d68/d8b/d7c/d7d/d53/lf2 0 2024-09-16T08:57:17.173 INFO:tasks.workunit.client.1.smithi097.stdout:3/787: mkdir d7/de/d8d/d90/d96/d9f/db5/d110 0 2024-09-16T08:57:17.175 INFO:tasks.workunit.client.1.smithi097.stdout:7/694: dwrite d4/d12/f4f [4194304,4194304] 0 2024-09-16T08:57:17.175 INFO:tasks.workunit.client.1.smithi097.stdout:1/872: symlink d5/d70/d85/db5/l117 0 2024-09-16T08:57:17.178 INFO:tasks.workunit.client.0.smithi031.stdout:0/336: rename d6/d12/d13/f30 to d6/d12/d13/d3c/f6d 0 2024-09-16T08:57:17.178 INFO:tasks.workunit.client.0.smithi031.stdout:0/337: write d6/f7 [4774215,75622] 0 2024-09-16T08:57:17.179 INFO:tasks.workunit.client.0.smithi031.stdout:2/263: symlink d6/da/l55 0 2024-09-16T08:57:17.182 INFO:tasks.workunit.client.1.smithi097.stdout:2/797: creat d3/d4/d68/d8b/d7c/d7d/d53/ff3 x:0 0 0 2024-09-16T08:57:17.184 INFO:tasks.workunit.client.0.smithi031.stdout:8/274: rename d4/l4d to d4/d7/d21/d37/d46/l5c 0 2024-09-16T08:57:17.184 INFO:tasks.workunit.client.0.smithi031.stdout:8/275: chown d4/d7/d21/l22 761554954 1 2024-09-16T08:57:17.185 INFO:tasks.workunit.client.1.smithi097.stdout:3/788: symlink d7/d28/d63/l111 0 2024-09-16T08:57:17.185 INFO:tasks.workunit.client.1.smithi097.stdout:3/789: dread - d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/fb6 zero size 2024-09-16T08:57:17.185 INFO:tasks.workunit.client.1.smithi097.stdout:3/790: write d7/de/d9c/daf/f10f [101370,122447] 0 2024-09-16T08:57:17.185 INFO:tasks.workunit.client.1.smithi097.stdout:3/791: dread - d7/de/d8d/d90/d92/fb1 zero size 2024-09-16T08:57:17.187 INFO:tasks.workunit.client.0.smithi031.stdout:0/338: write d6/f7 [2005121,126906] 0 2024-09-16T08:57:17.187 INFO:tasks.workunit.client.0.smithi031.stdout:2/264: mknod d6/da/d37/c56 0 2024-09-16T08:57:17.187 INFO:tasks.workunit.client.0.smithi031.stdout:0/339: dread - d6/d12/d47/d5b/f67 zero size 2024-09-16T08:57:17.187 INFO:tasks.workunit.client.0.smithi031.stdout:0/340: write d6/d12/d47/d5b/f5e [1037085,5656] 0 2024-09-16T08:57:17.188 INFO:tasks.workunit.client.0.smithi031.stdout:0/341: dread d6/d12/f22 [0,4194304] 0 2024-09-16T08:57:17.191 INFO:tasks.workunit.client.1.smithi097.stdout:7/695: link d4/d9/d2b/fa9 d4/d9/fe7 0 2024-09-16T08:57:17.191 INFO:tasks.workunit.client.1.smithi097.stdout:7/696: fdatasync d4/d12/d1f/f21 0 2024-09-16T08:57:17.194 INFO:tasks.workunit.client.1.smithi097.stdout:1/873: truncate d5/d17/f90 1021989 0 2024-09-16T08:57:17.196 INFO:tasks.workunit.client.0.smithi031.stdout:2/265: mknod d6/d2e/d40/c57 0 2024-09-16T08:57:17.196 INFO:tasks.workunit.client.0.smithi031.stdout:2/266: fdatasync d6/f13 0 2024-09-16T08:57:17.196 INFO:tasks.workunit.client.0.smithi031.stdout:4/294: link d0/l5 d0/d2/d16/d1e/l5c 0 2024-09-16T08:57:17.197 INFO:tasks.workunit.client.1.smithi097.stdout:2/798: mknod d3/d4/d68/d8b/cf4 0 2024-09-16T08:57:17.198 INFO:tasks.workunit.client.0.smithi031.stdout:2/267: getdents d6/d41/d28 0 2024-09-16T08:57:17.198 INFO:tasks.workunit.client.0.smithi031.stdout:2/268: write f4 [3384767,71157] 0 2024-09-16T08:57:17.199 INFO:tasks.workunit.client.0.smithi031.stdout:2/269: fsync d6/d20/f4f 0 2024-09-16T08:57:17.199 INFO:tasks.workunit.client.0.smithi031.stdout:2/270: stat d6/da/fe 0 2024-09-16T08:57:17.201 INFO:tasks.workunit.client.0.smithi031.stdout:4/295: creat d0/d2/d16/d59/f5d x:0 0 0 2024-09-16T08:57:17.203 INFO:tasks.workunit.client.1.smithi097.stdout:1/874: rename d5/d17/df5 to d5/ddd/d118 0 2024-09-16T08:57:17.203 INFO:tasks.workunit.client.1.smithi097.stdout:1/875: write d5/db/dc/d19/f8c [239243,61735] 0 2024-09-16T08:57:17.203 INFO:tasks.workunit.client.1.smithi097.stdout:1/876: stat d5/d17/d54/d6b/d71/dd3 0 2024-09-16T08:57:17.203 INFO:tasks.workunit.client.1.smithi097.stdout:3/792: write d7/de/d3c/f73 [802537,65296] 0 2024-09-16T08:57:17.203 INFO:tasks.workunit.client.0.smithi031.stdout:4/296: creat d0/d2/d3a/d54/f5e x:0 0 0 2024-09-16T08:57:17.204 INFO:tasks.workunit.client.0.smithi031.stdout:4/297: chown d0/d2/f41 309557496 1 2024-09-16T08:57:17.204 INFO:tasks.workunit.client.1.smithi097.stdout:7/697: dread d4/d9/f24 [0,4194304] 0 2024-09-16T08:57:17.204 INFO:tasks.workunit.client.1.smithi097.stdout:2/799: link d3/d4/d68/d8b/d7c/d7d/f6d d3/d4/d77/ff5 0 2024-09-16T08:57:17.206 INFO:tasks.workunit.client.1.smithi097.stdout:1/877: rename d5/ddd/d118/c106 to d5/d17/d54/c119 0 2024-09-16T08:57:17.209 INFO:tasks.workunit.client.1.smithi097.stdout:2/800: creat d3/d4/d9/d78/dcc/ff6 x:0 0 0 2024-09-16T08:57:17.209 INFO:tasks.workunit.client.0.smithi031.stdout:8/276: dwrite d4/d2f/f52 [0,4194304] 0 2024-09-16T08:57:17.210 INFO:tasks.workunit.client.0.smithi031.stdout:0/342: dwrite d6/d12/d47/d5b/f67 [0,4194304] 0 2024-09-16T08:57:17.211 INFO:tasks.workunit.client.0.smithi031.stdout:4/298: rename d0/d2/f27 to d0/f5f 0 2024-09-16T08:57:17.211 INFO:tasks.workunit.client.0.smithi031.stdout:4/299: fsync d0/d2/f33 0 2024-09-16T08:57:17.212 INFO:tasks.workunit.client.1.smithi097.stdout:3/793: rename d7/de/d8d/d90/d92/fdd to d7/de/d3c/d3f/d60/dcb/d35/d3d/f112 0 2024-09-16T08:57:17.212 INFO:tasks.workunit.client.1.smithi097.stdout:3/794: write d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/fb6 [921236,35949] 0 2024-09-16T08:57:17.214 INFO:tasks.workunit.client.1.smithi097.stdout:2/801: rmdir d3/d4/d77 39 2024-09-16T08:57:17.214 INFO:tasks.workunit.client.1.smithi097.stdout:2/802: chown d3/d4/d68/d8b/d7c/d7d/fb5 1596204859 1 2024-09-16T08:57:17.217 INFO:tasks.workunit.client.0.smithi031.stdout:8/277: creat d4/d7/d21/d35/f5d x:0 0 0 2024-09-16T08:57:17.217 INFO:tasks.workunit.client.0.smithi031.stdout:2/271: dread f4 [0,4194304] 0 2024-09-16T08:57:17.217 INFO:tasks.workunit.client.0.smithi031.stdout:2/272: fsync d6/d20/f26 0 2024-09-16T08:57:17.220 INFO:tasks.workunit.client.1.smithi097.stdout:3/795: mkdir d7/de/d8d/d90/d96/d9f/dbb/d113 0 2024-09-16T08:57:17.225 INFO:tasks.workunit.client.1.smithi097.stdout:2/803: rename d3/d4/d8/d51/ff0 to d3/d4/d68/d8b/d7c/d7d/d53/d47/ff7 0 2024-09-16T08:57:17.225 INFO:tasks.workunit.client.1.smithi097.stdout:2/804: readlink d3/d4/d9/l93 0 2024-09-16T08:57:17.225 INFO:tasks.workunit.client.1.smithi097.stdout:2/805: write d3/d4/d8/d51/f63 [2040134,39182] 0 2024-09-16T08:57:17.226 INFO:tasks.workunit.client.0.smithi031.stdout:4/300: symlink d0/d2/d3a/d55/l60 0 2024-09-16T08:57:17.226 INFO:tasks.workunit.client.1.smithi097.stdout:3/796: rename d7/de/d3c/d4f/ca4 to d7/de/d8d/d90/d96/d9f/c114 0 2024-09-16T08:57:17.228 INFO:tasks.workunit.client.1.smithi097.stdout:2/806: getdents d3/d4/d68/d8b/d61/dac/dd9 0 2024-09-16T08:57:17.228 INFO:tasks.workunit.client.1.smithi097.stdout:2/807: stat d3/fae 0 2024-09-16T08:57:17.233 INFO:tasks.workunit.client.0.smithi031.stdout:0/343: getdents d6 0 2024-09-16T08:57:17.235 INFO:tasks.workunit.client.1.smithi097.stdout:3/797: mknod d7/d28/d30/d88/db3/c115 0 2024-09-16T08:57:17.237 INFO:tasks.workunit.client.0.smithi031.stdout:4/301: creat d0/d2/d16/d2b/f61 x:0 0 0 2024-09-16T08:57:17.240 INFO:tasks.workunit.client.0.smithi031.stdout:0/344: creat d6/f6e x:0 0 0 2024-09-16T08:57:17.243 INFO:tasks.workunit.client.0.smithi031.stdout:0/345: creat d6/d12/d13/f6f x:0 0 0 2024-09-16T08:57:17.243 INFO:tasks.workunit.client.0.smithi031.stdout:0/346: truncate d6/f57 5544408 0 2024-09-16T08:57:17.243 INFO:tasks.workunit.client.0.smithi031.stdout:0/347: chown d6/fc 56499667 1 2024-09-16T08:57:17.244 INFO:tasks.workunit.client.0.smithi031.stdout:0/348: link d6/fa d6/d12/d13/d3c/f70 0 2024-09-16T08:57:17.247 INFO:tasks.workunit.client.1.smithi097.stdout:2/808: dwrite d3/d4/d9/d67/f84 [0,4194304] 0 2024-09-16T08:57:17.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/302: write d0/d2/d16/f30 [2183318,54694] 0 2024-09-16T08:57:17.247 INFO:tasks.workunit.client.0.smithi031.stdout:4/303: fdatasync d0/d2/f1a 0 2024-09-16T08:57:17.250 INFO:tasks.workunit.client.1.smithi097.stdout:2/809: mkdir d3/d4/d68/d8b/d7c/d7d/d53/da3/de8/df8 0 2024-09-16T08:57:17.252 INFO:tasks.workunit.client.0.smithi031.stdout:4/304: creat d0/d3c/f62 x:0 0 0 2024-09-16T08:57:17.253 INFO:tasks.workunit.client.1.smithi097.stdout:2/810: rename d3/d4/d9/f36 to d3/d1b/ff9 0 2024-09-16T08:57:17.256 INFO:tasks.workunit.client.1.smithi097.stdout:2/811: symlink d3/d4/d8/lfa 0 2024-09-16T08:57:17.257 INFO:tasks.workunit.client.0.smithi031.stdout:4/305: rename d0/fb to d0/d2/d16/d59/f63 0 2024-09-16T08:57:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.258+0000 7f26956aa640 1 -- 172.21.15.31:0/888266493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690102f40 msgr2=0x7f2690105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:17.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.258+0000 7f26956aa640 1 --2- 172.21.15.31:0/888266493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690102f40 0x7f2690105330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f26840098b0 tx=0x7f268402a4d0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:16 smithi031 ceph-mon[27844]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:17.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:16 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/407268374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:57:17.261 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:16 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3537088789' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:57:17.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.258+0000 7f26956aa640 1 -- 172.21.15.31:0/888266493 shutdown_connections 2024-09-16T08:57:17.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.258+0000 7f26956aa640 1 --2- 172.21.15.31:0/888266493 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690102f40 0x7f2690105330 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.258+0000 7f26956aa640 1 --2- 172.21.15.31:0/888266493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690100610 0x7f2690102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.258+0000 7f26956aa640 1 -- 172.21.15.31:0/888266493 >> 172.21.15.31:0/888266493 conn(0x7f26900fa230 msgr2=0x7f26900fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.259+0000 7f26956aa640 1 -- 172.21.15.31:0/888266493 shutdown_connections 2024-09-16T08:57:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.259+0000 7f26956aa640 1 -- 172.21.15.31:0/888266493 wait complete. 2024-09-16T08:57:17.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.259+0000 7f26956aa640 1 Processor -- start 2024-09-16T08:57:17.262 INFO:tasks.workunit.client.1.smithi097.stdout:2/812: link d3/d1b/f76 d3/d1b/da7/ffb 0 2024-09-16T08:57:17.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:16 smithi097 ceph-mon[35478]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:17.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:16 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/407268374' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:57:17.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:16 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3537088789' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:57:17.263 INFO:tasks.workunit.client.0.smithi031.stdout:0/349: dwrite d6/fa [0,4194304] 0 2024-09-16T08:57:17.264 INFO:tasks.workunit.client.0.smithi031.stdout:0/350: fsync d6/d12/d13/d36/f43 0 2024-09-16T08:57:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.261+0000 7f26956aa640 1 -- start start 2024-09-16T08:57:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.261+0000 7f26956aa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100610 0x7f26901a4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.261+0000 7f26956aa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690102f40 0x7f26901a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.261+0000 7f26956aa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26901a5030 con 0x7f2690100610 2024-09-16T08:57:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.261+0000 7f26956aa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26901a51a0 con 0x7f2690102f40 2024-09-16T08:57:17.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100610 0x7f26901a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100610 0x7f26901a4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34876/0 (socket says 172.21.15.31:34876) 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268effd640 1 -- 172.21.15.31:0/1694206469 learned_addr learned my addr 172.21.15.31:0/1694206469 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268e7fc640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690102f40 0x7f26901a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268effd640 1 -- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690102f40 msgr2=0x7f26901a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268effd640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690102f40 0x7f26901a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268effd640 1 -- 172.21.15.31:0/1694206469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2684009560 con 0x7f2690100610 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.262+0000 7f268effd640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100610 0x7f26901a4520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f268000e9a0 tx=0x7f268000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.263+0000 7f2677fff640 1 -- 172.21.15.31:0/1694206469 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f268000cd50 con 0x7f2690100610 2024-09-16T08:57:17.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.263+0000 7f2677fff640 1 -- 172.21.15.31:0/1694206469 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f268000ceb0 con 0x7f2690100610 2024-09-16T08:57:17.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.263+0000 7f2677fff640 1 -- 172.21.15.31:0/1694206469 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2680010660 con 0x7f2690100610 2024-09-16T08:57:17.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.263+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26901a9b90 con 0x7f2690100610 2024-09-16T08:57:17.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.263+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26901aa0b0 con 0x7f2690100610 2024-09-16T08:57:17.267 INFO:tasks.workunit.client.1.smithi097.stdout:2/813: mkdir d3/d4/d68/d8b/d7c/d7d/d53/da3/de8/df8/dfc 0 2024-09-16T08:57:17.267 INFO:tasks.workunit.client.0.smithi031.stdout:0/351: creat d6/d12/d47/d5b/f71 x:0 0 0 2024-09-16T08:57:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.264+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2690108060 con 0x7f2690100610 2024-09-16T08:57:17.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.265+0000 7f2677fff640 1 -- 172.21.15.31:0/1694206469 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f26800048a0 con 0x7f2690100610 2024-09-16T08:57:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.266+0000 7f2677fff640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2678073d40 0x7f2678076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.266+0000 7f2677fff640 1 -- 172.21.15.31:0/1694206469 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f2680014070 con 0x7f2690100610 2024-09-16T08:57:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.266+0000 7f268e7fc640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2678073d40 0x7f2678076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:17.269 INFO:tasks.workunit.client.0.smithi031.stdout:0/352: rename d6/d12/d13/d36/l58 to d6/d61/l72 0 2024-09-16T08:57:17.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.267+0000 7f268e7fc640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2678073d40 0x7f2678076200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2684006010 tx=0x7f2684005d50 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:17.270 INFO:tasks.workunit.client.0.smithi031.stdout:8/278: rmdir d4/d7/d21/d35 39 2024-09-16T08:57:17.271 INFO:tasks.workunit.client.0.smithi031.stdout:8/279: mkdir d4/d7/d36/d5e 0 2024-09-16T08:57:17.272 INFO:tasks.workunit.client.0.smithi031.stdout:4/306: dwrite d0/d2/d3a/d54/f5e [0,4194304] 0 2024-09-16T08:57:17.273 INFO:tasks.workunit.client.0.smithi031.stdout:8/280: symlink d4/d7/d21/d35/d50/l5f 0 2024-09-16T08:57:17.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.272+0000 7f2677fff640 1 -- 172.21.15.31:0/1694206469 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f2680058be0 con 0x7f2690100610 2024-09-16T08:57:17.276 INFO:tasks.workunit.client.0.smithi031.stdout:4/307: creat d0/d2/d3a/d55/f64 x:0 0 0 2024-09-16T08:57:17.277 INFO:tasks.workunit.client.0.smithi031.stdout:8/281: symlink d4/d7/d21/d24/l60 0 2024-09-16T08:57:17.280 INFO:tasks.workunit.client.0.smithi031.stdout:4/308: symlink d0/d2/d16/d59/l65 0 2024-09-16T08:57:17.283 INFO:tasks.workunit.client.0.smithi031.stdout:4/309: unlink d0/c42 0 2024-09-16T08:57:17.284 INFO:tasks.workunit.client.0.smithi031.stdout:4/310: symlink d0/d2/d16/d2b/l66 0 2024-09-16T08:57:17.285 INFO:tasks.workunit.client.0.smithi031.stdout:4/311: creat d0/d2/d16/d2b/f67 x:0 0 0 2024-09-16T08:57:17.285 INFO:tasks.workunit.client.0.smithi031.stdout:8/282: dread d4/d7/f51 [0,4194304] 0 2024-09-16T08:57:17.288 INFO:tasks.workunit.client.0.smithi031.stdout:4/312: mknod d0/d2/d3a/c68 0 2024-09-16T08:57:17.288 INFO:tasks.workunit.client.0.smithi031.stdout:4/313: write d0/d2/d16/d2b/f67 [772581,100681] 0 2024-09-16T08:57:17.291 INFO:tasks.workunit.client.1.smithi097.stdout:9/723: sync 2024-09-16T08:57:17.291 INFO:tasks.workunit.client.1.smithi097.stdout:9/724: write d2/f4 [768145,2009] 0 2024-09-16T08:57:17.291 INFO:tasks.workunit.client.1.smithi097.stdout:9/725: write d2/d3f/fd4 [984468,18676] 0 2024-09-16T08:57:17.291 INFO:tasks.workunit.client.1.smithi097.stdout:2/814: getdents d3/d4/d68/d8b/d7c/d7d/d53/da3/de8/df8 0 2024-09-16T08:57:17.294 INFO:tasks.workunit.client.0.smithi031.stdout:0/353: link d6/d61/l72 d6/d12/d13/l73 0 2024-09-16T08:57:17.301 INFO:tasks.workunit.client.1.smithi097.stdout:9/726: dread d2/d5/db0/dc8/d74/d8c/d94/f99 [4194304,4194304] 0 2024-09-16T08:57:17.301 INFO:tasks.workunit.client.1.smithi097.stdout:9/727: write d2/d5/fa [4266812,83946] 0 2024-09-16T08:57:17.305 INFO:tasks.workunit.client.0.smithi031.stdout:4/314: write d0/d2/d16/f22 [1561198,15701] 0 2024-09-16T08:57:17.313 INFO:tasks.workunit.client.1.smithi097.stdout:9/728: write d2/d5/f4e [977148,59697] 0 2024-09-16T08:57:17.319 INFO:tasks.workunit.client.0.smithi031.stdout:8/283: dwrite d4/d7/d21/d24/f31 [0,4194304] 0 2024-09-16T08:57:17.319 INFO:tasks.workunit.client.0.smithi031.stdout:8/284: chown d4/d7/d21/d24/c4b 2117 1 2024-09-16T08:57:17.319 INFO:tasks.workunit.client.1.smithi097.stdout:2/815: dwrite d3/d4/d8/d51/f59 [4194304,4194304] 0 2024-09-16T08:57:17.320 INFO:tasks.workunit.client.0.smithi031.stdout:0/354: dwrite d6/f33 [4194304,4194304] 0 2024-09-16T08:57:17.326 INFO:tasks.workunit.client.0.smithi031.stdout:4/315: getdents d0 0 2024-09-16T08:57:17.326 INFO:tasks.workunit.client.0.smithi031.stdout:4/316: write d0/d2/d16/d59/f63 [1005065,21298] 0 2024-09-16T08:57:17.326 INFO:tasks.workunit.client.0.smithi031.stdout:4/317: fdatasync d0/d2/d3a/d54/f5e 0 2024-09-16T08:57:17.335 INFO:tasks.workunit.client.1.smithi097.stdout:2/816: dread d3/d4/d68/d8b/d7c/d7d/d53/f58 [0,4194304] 0 2024-09-16T08:57:17.336 INFO:tasks.workunit.client.1.smithi097.stdout:2/817: creat d3/d4/d9/ffd x:0 0 0 2024-09-16T08:57:17.338 INFO:tasks.workunit.client.1.smithi097.stdout:2/818: unlink d3/d4/l1c 0 2024-09-16T08:57:17.339 INFO:tasks.workunit.client.1.smithi097.stdout:2/819: symlink d3/d4/d68/d8b/d7c/d7d/d53/da3/de8/df8/dfc/lfe 0 2024-09-16T08:57:17.340 INFO:tasks.workunit.client.1.smithi097.stdout:2/820: unlink d3/d1b/c55 0 2024-09-16T08:57:17.342 INFO:tasks.workunit.client.1.smithi097.stdout:9/729: dwrite d2/f4 [0,4194304] 0 2024-09-16T08:57:17.357 INFO:tasks.workunit.client.0.smithi031.stdout:8/285: dwrite d4/f27 [0,4194304] 0 2024-09-16T08:57:17.357 INFO:tasks.workunit.client.0.smithi031.stdout:8/286: mknod d4/d7/d36/d38/c61 0 2024-09-16T08:57:17.361 INFO:tasks.workunit.client.0.smithi031.stdout:0/355: dwrite d6/f34 [0,4194304] 0 2024-09-16T08:57:17.361 INFO:tasks.workunit.client.0.smithi031.stdout:0/356: truncate d6/d12/d47/d5b/f5e 1519387 0 2024-09-16T08:57:17.361 INFO:tasks.workunit.client.0.smithi031.stdout:0/357: creat d6/f74 x:0 0 0 2024-09-16T08:57:17.361 INFO:tasks.workunit.client.0.smithi031.stdout:0/358: fdatasync d6/f33 0 2024-09-16T08:57:17.361 INFO:tasks.workunit.client.0.smithi031.stdout:4/318: dwrite d0/d2/d3a/d55/f64 [0,4194304] 0 2024-09-16T08:57:17.364 INFO:tasks.workunit.client.0.smithi031.stdout:0/359: rename d6/d12/d13/c19 to d6/d61/c75 0 2024-09-16T08:57:17.364 INFO:tasks.workunit.client.0.smithi031.stdout:0/360: chown d6/d12/d13/d3c/f6d 242886 1 2024-09-16T08:57:17.364 INFO:tasks.workunit.client.0.smithi031.stdout:0/361: write d6/d12/d13/f6f [841851,48036] 0 2024-09-16T08:57:17.367 INFO:tasks.workunit.client.1.smithi097.stdout:9/730: dwrite d2/d5/d56/fc4 [0,4194304] 0 2024-09-16T08:57:17.367 INFO:tasks.workunit.client.1.smithi097.stdout:9/731: dread - d2/d5/d3a/f4c zero size 2024-09-16T08:57:17.368 INFO:tasks.workunit.client.1.smithi097.stdout:2/821: dwrite d3/f32 [0,4194304] 0 2024-09-16T08:57:17.368 INFO:tasks.workunit.client.1.smithi097.stdout:2/822: truncate d3/d4/d68/d8b/d7c/d7d/d53/da3/fe3 817303 0 2024-09-16T08:57:17.374 INFO:tasks.workunit.client.1.smithi097.stdout:9/732: creat d2/d5/db0/dc8/d74/d8c/d94/d96/fe7 x:0 0 0 2024-09-16T08:57:17.376 INFO:tasks.workunit.client.0.smithi031.stdout:4/319: creat d0/d5b/f69 x:0 0 0 2024-09-16T08:57:17.376 INFO:tasks.workunit.client.1.smithi097.stdout:2/823: symlink d3/d4/d68/d8b/d7c/d7d/d53/da3/de8/df8/dfc/lff 0 2024-09-16T08:57:17.378 INFO:tasks.workunit.client.0.smithi031.stdout:0/362: dwrite d6/f74 [0,4194304] 0 2024-09-16T08:57:17.378 INFO:tasks.workunit.client.0.smithi031.stdout:0/363: truncate d6/d12/f6c 991422 0 2024-09-16T08:57:17.378 INFO:tasks.workunit.client.0.smithi031.stdout:0/364: write d6/d12/f22 [1121910,20978] 0 2024-09-16T08:57:17.379 INFO:tasks.workunit.client.1.smithi097.stdout:9/733: rmdir d2/d5/db0/dc8/d74/d8c/d94/de4 0 2024-09-16T08:57:17.381 INFO:tasks.workunit.client.0.smithi031.stdout:4/320: dread d0/d2/d16/f30 [0,4194304] 0 2024-09-16T08:57:17.381 INFO:tasks.workunit.client.0.smithi031.stdout:4/321: chown d0/d2/d16/c2c 51964654 1 2024-09-16T08:57:17.381 INFO:tasks.workunit.client.0.smithi031.stdout:4/322: fsync d0/d2/d16/d1e/f34 0 2024-09-16T08:57:17.381 INFO:tasks.workunit.client.0.smithi031.stdout:4/323: chown d0/d2/d3a 333725898 1 2024-09-16T08:57:17.383 INFO:tasks.workunit.client.1.smithi097.stdout:9/734: mknod d2/ce8 0 2024-09-16T08:57:17.383 INFO:tasks.workunit.client.1.smithi097.stdout:9/735: fdatasync f0 0 2024-09-16T08:57:17.383 INFO:tasks.workunit.client.1.smithi097.stdout:9/736: creat d2/d5/d56/fe9 x:0 0 0 2024-09-16T08:57:17.384 INFO:tasks.workunit.client.0.smithi031.stdout:0/365: symlink d6/d12/d13/d3c/l76 0 2024-09-16T08:57:17.385 INFO:tasks.workunit.client.0.smithi031.stdout:6/421: sync 2024-09-16T08:57:17.386 INFO:tasks.workunit.client.1.smithi097.stdout:9/737: creat d2/d5/d72/dc1/de6/fea x:0 0 0 2024-09-16T08:57:17.387 INFO:tasks.workunit.client.1.smithi097.stdout:9/738: chown d2/d5/db0/dc8/d74/d8c/d94 253627 1 2024-09-16T08:57:17.387 INFO:tasks.workunit.client.1.smithi097.stdout:9/739: chown d2/d5/db0/dc8/d74/de0 72227298 1 2024-09-16T08:57:17.388 INFO:tasks.workunit.client.0.smithi031.stdout:0/366: truncate d6/d12/d47/d5b/f5e 1455777 0 2024-09-16T08:57:17.388 INFO:tasks.workunit.client.0.smithi031.stdout:0/367: read - d6/d61/f62 zero size 2024-09-16T08:57:17.388 INFO:tasks.workunit.client.0.smithi031.stdout:0/368: creat d6/d12/f77 x:0 0 0 2024-09-16T08:57:17.389 INFO:tasks.workunit.client.0.smithi031.stdout:4/324: write d0/d2/d3a/d54/f5e [3089848,128260] 0 2024-09-16T08:57:17.389 INFO:tasks.workunit.client.0.smithi031.stdout:4/325: chown d0/fa 0 1 2024-09-16T08:57:17.390 INFO:tasks.workunit.client.0.smithi031.stdout:6/422: unlink d0/dd/d35/f64 0 2024-09-16T08:57:17.390 INFO:tasks.workunit.client.0.smithi031.stdout:6/423: fdatasync d0/dd/d74/d5e/d7c/d3f/f4c 0 2024-09-16T08:57:17.392 INFO:tasks.workunit.client.1.smithi097.stdout:9/740: creat d2/d5/db0/dc8/d74/d8c/feb x:0 0 0 2024-09-16T08:57:17.392 INFO:tasks.workunit.client.1.smithi097.stdout:9/741: readlink d2/d3f/l67 0 2024-09-16T08:57:17.392 INFO:tasks.workunit.client.1.smithi097.stdout:9/742: chown d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/c7e 895363754 1 2024-09-16T08:57:17.394 INFO:tasks.workunit.client.0.smithi031.stdout:0/369: getdents d6/d12/d47/d4d 0 2024-09-16T08:57:17.394 INFO:tasks.workunit.client.0.smithi031.stdout:0/370: creat d6/d61/f78 x:0 0 0 2024-09-16T08:57:17.394 INFO:tasks.workunit.client.0.smithi031.stdout:0/371: chown d6/d61/c75 28639 1 2024-09-16T08:57:17.395 INFO:tasks.workunit.client.1.smithi097.stdout:9/743: dread d2/d5/db0/dc8/d74/d8c/d94/f99 [4194304,4194304] 0 2024-09-16T08:57:17.395 INFO:tasks.workunit.client.1.smithi097.stdout:9/744: write d2/d5/f4e [5057967,35450] 0 2024-09-16T08:57:17.395 INFO:tasks.workunit.client.1.smithi097.stdout:9/745: write d2/d5/d72/f89 [1631397,30926] 0 2024-09-16T08:57:17.396 INFO:tasks.workunit.client.1.smithi097.stdout:9/746: write d2/d5/d3a/f4c [677,17337] 0 2024-09-16T08:57:17.396 INFO:tasks.workunit.client.1.smithi097.stdout:9/747: dread d2/d5/d56/d6b/dcd/dd8/fe2 [0,4194304] 0 2024-09-16T08:57:17.396 INFO:tasks.workunit.client.0.smithi031.stdout:4/326: link d0/c51 d0/d2/d3a/d54/c6a 0 2024-09-16T08:57:17.396 INFO:tasks.workunit.client.0.smithi031.stdout:4/327: fdatasync d0/f38 0 2024-09-16T08:57:17.401 INFO:tasks.workunit.client.1.smithi097.stdout:9/748: link d2/d5/db0/dc8/d74/d8c/d94/d96/l98 d2/d5/db0/dc8/d74/d8c/d94/lec 0 2024-09-16T08:57:17.402 INFO:tasks.workunit.client.0.smithi031.stdout:6/424: symlink d0/dd/d35/d6c/d81/l87 0 2024-09-16T08:57:17.402 INFO:tasks.workunit.client.0.smithi031.stdout:6/425: fdatasync d0/dd/d74/f2d 0 2024-09-16T08:57:17.402 INFO:tasks.workunit.client.0.smithi031.stdout:6/426: write d0/f72 [461284,34799] 0 2024-09-16T08:57:17.405 INFO:tasks.workunit.client.1.smithi097.stdout:9/749: rename d2/d5/d3a/d70/f92 to d2/d5/d72/da2/fed 0 2024-09-16T08:57:17.409 INFO:tasks.workunit.client.0.smithi031.stdout:6/427: link d0/dd/d2b/l39 d0/dd/d35/d6c/d83/l88 0 2024-09-16T08:57:17.411 INFO:tasks.workunit.client.0.smithi031.stdout:4/328: rename d0/d2/d3a/d54 to d0/d3c/d6b 0 2024-09-16T08:57:17.412 INFO:tasks.workunit.client.1.smithi097.stdout:9/750: truncate d2/d5/f29 754240 0 2024-09-16T08:57:17.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/428: unlink d0/dd/d74/d5e/d7c/d3f/f7d 0 2024-09-16T08:57:17.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/429: truncate d0/dd/d74/d67/f7a 51444 0 2024-09-16T08:57:17.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/430: readlink d0/l5c 0 2024-09-16T08:57:17.417 INFO:tasks.workunit.client.0.smithi031.stdout:4/329: rename d0/d2/d16/d2b/l66 to d0/d2/d16/d59/l6c 0 2024-09-16T08:57:17.418 INFO:tasks.workunit.client.0.smithi031.stdout:4/330: mkdir d0/d2/d6d 0 2024-09-16T08:57:17.439 INFO:tasks.workunit.client.1.smithi097.stdout:0/768: sync 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:8/718: sync 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:7/698: sync 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:4/725: sync 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:7/699: chown d4/d9/d2b/f2e 282169 1 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:8/719: chown d8/dd/d15/d2c/d4a/d4d/f5c 462455 1 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:0/769: fsync da/d1a/d41/f71 0 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:7/700: read - d4/d9/d1c/d5f/f9a zero size 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:5/832: sync 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:8/720: creat d8/dd/d15/d2c/d4a/d4e/db2/fde x:0 0 0 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:7/701: write d4/d7/ff [5415733,123361] 0 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:5/833: write db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 [901449,121756] 0 2024-09-16T08:57:17.440 INFO:tasks.workunit.client.1.smithi097.stdout:3/798: sync 2024-09-16T08:57:17.441 INFO:tasks.workunit.client.1.smithi097.stdout:1/878: sync 2024-09-16T08:57:17.441 INFO:tasks.workunit.client.1.smithi097.stdout:6/783: sync 2024-09-16T08:57:17.441 INFO:tasks.workunit.client.1.smithi097.stdout:0/770: creat da/d7e/dc4/dfc/d47/db3/f111 x:0 0 0 2024-09-16T08:57:17.444 INFO:tasks.workunit.client.1.smithi097.stdout:8/721: mknod d8/dd/d15/d16/d7c/da2/cdf 0 2024-09-16T08:57:17.444 INFO:tasks.workunit.client.1.smithi097.stdout:8/722: chown d8/dd/d15/l1d 1035053880 1 2024-09-16T08:57:17.444 INFO:tasks.workunit.client.1.smithi097.stdout:7/702: mkdir d4/d12/de8 0 2024-09-16T08:57:17.444 INFO:tasks.workunit.client.1.smithi097.stdout:5/834: truncate db/d18/d1c/d42/d46/d7e/d64/fb3 2110678 0 2024-09-16T08:57:17.445 INFO:tasks.workunit.client.1.smithi097.stdout:3/799: unlink d7/de/d3c/d3f/d60/dcb/l9a 0 2024-09-16T08:57:17.446 INFO:tasks.workunit.client.1.smithi097.stdout:9/751: getdents d2/d5/db0/dc8/d74/d8c/d94 0 2024-09-16T08:57:17.447 INFO:tasks.workunit.client.1.smithi097.stdout:1/879: mknod d5/dce/c11a 0 2024-09-16T08:57:17.448 INFO:tasks.workunit.client.1.smithi097.stdout:6/784: rename d1/dd3/d9e/dca/d81/lf2 to d1/dd3/d9e/dca/d81/l103 0 2024-09-16T08:57:17.449 INFO:tasks.workunit.client.1.smithi097.stdout:8/723: symlink d8/dd/d15/d2c/d4a/d4d/d3d/le0 0 2024-09-16T08:57:17.450 INFO:tasks.workunit.client.1.smithi097.stdout:7/703: link d4/d7/dab/cb9 d4/d7/d77/ce9 0 2024-09-16T08:57:17.450 INFO:tasks.workunit.client.1.smithi097.stdout:1/880: dread d5/d14/d95/fd8 [0,4194304] 0 2024-09-16T08:57:17.450 INFO:tasks.workunit.client.1.smithi097.stdout:5/835: getdents db/d18 0 2024-09-16T08:57:17.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/785: creat d1/dd3/d9e/dca/d81/dad/f104 x:0 0 0 2024-09-16T08:57:17.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/786: creat d1/dc/f105 x:0 0 0 2024-09-16T08:57:17.460 INFO:tasks.workunit.client.1.smithi097.stdout:6/787: symlink d1/dd3/d9e/db3/de3/l106 0 2024-09-16T08:57:17.461 INFO:tasks.workunit.client.1.smithi097.stdout:9/752: rename d2/d5/d56/d6b to d2/d5/d72/dc1/ddd/dee 0 2024-09-16T08:57:17.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/788: symlink d1/dd8/l107 0 2024-09-16T08:57:17.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/789: chown d1/dd3/d9e/dca/d81/dc4/fc8 5168 1 2024-09-16T08:57:17.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/790: chown d1/dd3/d9e/dca/lbe 974625 1 2024-09-16T08:57:17.462 INFO:tasks.workunit.client.1.smithi097.stdout:6/791: fsync d1/dd3/d9e/dca/dbb/fd1 0 2024-09-16T08:57:17.464 INFO:tasks.workunit.client.1.smithi097.stdout:1/881: rename d5/db/dc/d19/f4d to d5/d17/d54/d6b/da8/dc5/dc7/f11b 0 2024-09-16T08:57:17.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/753: truncate d2/f25 119886 0 2024-09-16T08:57:17.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/754: readlink d2/d3f/ld6 0 2024-09-16T08:57:17.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/755: fdatasync d2/f31 0 2024-09-16T08:57:17.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/756: creat d2/d5/d72/da2/fef x:0 0 0 2024-09-16T08:57:17.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/757: dread - d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/fa7 zero size 2024-09-16T08:57:17.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/758: readlink d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/lb6 0 2024-09-16T08:57:17.466 INFO:tasks.workunit.client.1.smithi097.stdout:9/759: fdatasync d2/d5/d3a/d3c/f6d 0 2024-09-16T08:57:17.467 INFO:tasks.workunit.client.1.smithi097.stdout:6/792: creat d1/dd3/d9e/dca/d81/f108 x:0 0 0 2024-09-16T08:57:17.470 INFO:tasks.workunit.client.1.smithi097.stdout:5/836: rename db/d18/d20/fec to db/d18/d1c/d42/d46/d7e/d64/d7f/d89/dbc/f112 0 2024-09-16T08:57:17.470 INFO:tasks.workunit.client.1.smithi097.stdout:5/837: fdatasync db/d18/d1c/d42/d46/d7e/d76/fc0 0 2024-09-16T08:57:17.471 INFO:tasks.workunit.client.1.smithi097.stdout:1/882: unlink d5/db/f34 0 2024-09-16T08:57:17.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/431: dwrite d0/dd/d74/d5e/d7c/d56/f78 [0,4194304] 0 2024-09-16T08:57:17.474 INFO:tasks.workunit.client.1.smithi097.stdout:9/760: mknod d2/d5/d56/cf0 0 2024-09-16T08:57:17.474 INFO:tasks.workunit.client.1.smithi097.stdout:9/761: dread - d2/d5/d72/dc1/de6/fea zero size 2024-09-16T08:57:17.476 INFO:tasks.workunit.client.1.smithi097.stdout:6/793: rename d1/dd3/db0/ce9 to d1/dd3/d9e/dca/d16/d4f/d9f/c109 0 2024-09-16T08:57:17.478 INFO:tasks.workunit.client.1.smithi097.stdout:5/838: creat db/d18/f113 x:0 0 0 2024-09-16T08:57:17.479 INFO:tasks.workunit.client.1.smithi097.stdout:5/839: dread - db/d1a/faa zero size 2024-09-16T08:57:17.479 INFO:tasks.workunit.client.1.smithi097.stdout:5/840: write db/d22/f96 [1696730,9089] 0 2024-09-16T08:57:17.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/432: creat d0/dd/d35/d6c/d83/f89 x:0 0 0 2024-09-16T08:57:17.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/771: dwrite da/df/f3f [0,4194304] 0 2024-09-16T08:57:17.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/772: fdatasync da/d1a/d41/f71 0 2024-09-16T08:57:17.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/773: readlink da/d7e/dc4/dfc/d94/d101/le1 0 2024-09-16T08:57:17.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/774: dread - da/d7e/dc4/dfc/d47/db3/f111 zero size 2024-09-16T08:57:17.481 INFO:tasks.workunit.client.1.smithi097.stdout:0/775: chown da/d7e/dc4/dfc/d47/db3/d60/d68/cb5 1238437 1 2024-09-16T08:57:17.482 INFO:tasks.workunit.client.1.smithi097.stdout:1/883: dread d5/db/dc/d19/f104 [0,4194304] 0 2024-09-16T08:57:17.482 INFO:tasks.workunit.client.1.smithi097.stdout:1/884: chown d5/d103 30770 1 2024-09-16T08:57:17.482 INFO:tasks.workunit.client.1.smithi097.stdout:1/885: creat d5/d2c/f11c x:0 0 0 2024-09-16T08:57:17.483 INFO:tasks.workunit.client.1.smithi097.stdout:6/794: unlink d1/dc/d3c/cd9 0 2024-09-16T08:57:17.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/433: unlink d0/dd/d2b/f73 0 2024-09-16T08:57:17.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/434: write d0/dd/d74/d19/f85 [5020880,111341] 0 2024-09-16T08:57:17.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/435: fdatasync d0/dd/d74/d5e/d7c/d3f/f70 0 2024-09-16T08:57:17.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/436: stat d0/dd/d74/d19/f2e 0 2024-09-16T08:57:17.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/437: chown d0/f72 267247246 1 2024-09-16T08:57:17.485 INFO:tasks.workunit.client.0.smithi031.stdout:6/438: fsync d0/ff 0 2024-09-16T08:57:17.486 INFO:tasks.workunit.client.1.smithi097.stdout:9/762: dread d2/d5/d56/f8a [4194304,4194304] 0 2024-09-16T08:57:17.487 INFO:tasks.workunit.client.0.smithi031.stdout:0/372: dwrite d6/f1a [0,4194304] 0 2024-09-16T08:57:17.487 INFO:tasks.workunit.client.0.smithi031.stdout:0/373: fdatasync d6/d12/d13/f24 0 2024-09-16T08:57:17.487 INFO:tasks.workunit.client.1.smithi097.stdout:0/776: mkdir da/d24/d4b/d112 0 2024-09-16T08:57:17.489 INFO:tasks.workunit.client.0.smithi031.stdout:4/331: dwrite d0/d2/d16/d59/f5d [0,4194304] 0 2024-09-16T08:57:17.490 INFO:tasks.workunit.client.1.smithi097.stdout:7/704: dwrite d4/f64 [0,4194304] 0 2024-09-16T08:57:17.490 INFO:tasks.workunit.client.0.smithi031.stdout:0/374: mknod d6/d12/d47/d5b/c79 0 2024-09-16T08:57:17.491 INFO:tasks.workunit.client.1.smithi097.stdout:5/841: dread db/d18/d1c/f39 [0,4194304] 0 2024-09-16T08:57:17.491 INFO:tasks.workunit.client.1.smithi097.stdout:8/724: dwrite d8/dd/d15/d16/f3a [0,4194304] 0 2024-09-16T08:57:17.494 INFO:tasks.workunit.client.1.smithi097.stdout:8/725: dread d8/dd/d15/d2c/d4a/d4d/d3f/fa6 [0,4194304] 0 2024-09-16T08:57:17.494 INFO:tasks.workunit.client.1.smithi097.stdout:6/795: dread d1/dc/f7d [0,4194304] 0 2024-09-16T08:57:17.496 INFO:tasks.workunit.client.1.smithi097.stdout:5/842: mknod db/d18/d1c/d42/d46/d7e/d64/d7f/d89/c114 0 2024-09-16T08:57:17.496 INFO:tasks.workunit.client.1.smithi097.stdout:5/843: chown db/d22/f4b 782716784 1 2024-09-16T08:57:17.496 INFO:tasks.workunit.client.1.smithi097.stdout:5/844: chown db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/cac 25 1 2024-09-16T08:57:17.497 INFO:tasks.workunit.client.0.smithi031.stdout:6/439: read d0/dd/d74/d19/f29 [3849716,111863] 0 2024-09-16T08:57:17.501 INFO:tasks.workunit.client.1.smithi097.stdout:8/726: creat d8/dd/d15/d2c/fe1 x:0 0 0 2024-09-16T08:57:17.505 INFO:tasks.workunit.client.1.smithi097.stdout:1/886: rename d5/d14/d95 to d5/db/dc/d11d 0 2024-09-16T08:57:17.506 INFO:tasks.workunit.client.1.smithi097.stdout:1/887: chown d5/d2c/dcb/de2 11 1 2024-09-16T08:57:17.507 INFO:tasks.workunit.client.1.smithi097.stdout:8/727: truncate d8/f1e 4232163 0 2024-09-16T08:57:17.508 INFO:tasks.workunit.client.1.smithi097.stdout:6/796: write d1/dc/f38 [2473367,40949] 0 2024-09-16T08:57:17.509 INFO:tasks.workunit.client.1.smithi097.stdout:6/797: write d1/dc/f38 [3564352,31049] 0 2024-09-16T08:57:17.509 INFO:tasks.workunit.client.1.smithi097.stdout:6/798: read - d1/dd3/d9e/dca/d81/dad/f104 zero size 2024-09-16T08:57:17.509 INFO:tasks.workunit.client.1.smithi097.stdout:6/799: creat d1/dd3/d9e/dca/d81/f10a x:0 0 0 2024-09-16T08:57:17.509 INFO:tasks.workunit.client.1.smithi097.stdout:6/800: write d1/db7/dfd/fa2 [917752,34228] 0 2024-09-16T08:57:17.509 INFO:tasks.workunit.client.1.smithi097.stdout:6/801: dread - d1/dd3/fb4 zero size 2024-09-16T08:57:17.511 INFO:tasks.workunit.client.1.smithi097.stdout:9/763: rename d2/d5/d72/dc1/ddd/dee/fb7 to d2/d5/db0/dc8/ff1 0 2024-09-16T08:57:17.512 INFO:tasks.workunit.client.1.smithi097.stdout:5/845: dread db/d18/d20/f29 [0,4194304] 0 2024-09-16T08:57:17.516 INFO:tasks.workunit.client.1.smithi097.stdout:1/888: symlink d5/d2c/dcb/de2/l11e 0 2024-09-16T08:57:17.519 INFO:tasks.workunit.client.1.smithi097.stdout:0/777: dwrite da/d1a/d41/f71 [0,4194304] 0 2024-09-16T08:57:17.522 INFO:tasks.workunit.client.1.smithi097.stdout:7/705: dwrite d4/d1b/d41/f95 [0,4194304] 0 2024-09-16T08:57:17.522 INFO:tasks.workunit.client.1.smithi097.stdout:7/706: creat d4/d7/d79/fea x:0 0 0 2024-09-16T08:57:17.523 INFO:tasks.workunit.client.1.smithi097.stdout:8/728: symlink d8/dd/d71/le2 0 2024-09-16T08:57:17.528 INFO:tasks.workunit.client.1.smithi097.stdout:5/846: symlink db/d1a/d6d/l115 0 2024-09-16T08:57:17.529 INFO:tasks.workunit.client.0.smithi031.stdout:4/332: creat d0/d2/d16/f6e x:0 0 0 2024-09-16T08:57:17.529 INFO:tasks.workunit.client.0.smithi031.stdout:4/333: readlink d0/d2/d3a/d55/l58 0 2024-09-16T08:57:17.529 INFO:tasks.workunit.client.0.smithi031.stdout:4/334: write d0/f5f [1222056,101847] 0 2024-09-16T08:57:17.529 INFO:tasks.workunit.client.0.smithi031.stdout:4/335: write d0/d2/d16/d1e/f52 [1035719,45498] 0 2024-09-16T08:57:17.530 INFO:tasks.workunit.client.1.smithi097.stdout:1/889: symlink d5/d17/d54/d6b/da8/dc5/l11f 0 2024-09-16T08:57:17.530 INFO:tasks.workunit.client.1.smithi097.stdout:1/890: getdents d5/dce/df9 0 2024-09-16T08:57:17.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/375: truncate d6/f33 144417 0 2024-09-16T08:57:17.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/376: readlink d6/d12/d47/d5b/l5c 0 2024-09-16T08:57:17.531 INFO:tasks.workunit.client.0.smithi031.stdout:6/440: dwrite d0/dd/d74/d5e/d7c/d3f/d65/f68 [0,4194304] 0 2024-09-16T08:57:17.532 INFO:tasks.workunit.client.1.smithi097.stdout:0/778: link da/d1a/f22 da/d24/d4c/d80/d3b/d86/f113 0 2024-09-16T08:57:17.535 INFO:tasks.workunit.client.0.smithi031.stdout:0/377: creat d6/d12/d47/d4d/f7a x:0 0 0 2024-09-16T08:57:17.537 INFO:tasks.workunit.client.1.smithi097.stdout:0/779: dread da/d24/d4c/d80/d3b/f5a [0,4194304] 0 2024-09-16T08:57:17.538 INFO:tasks.workunit.client.1.smithi097.stdout:9/764: dwrite d2/d3f/f4b [0,4194304] 0 2024-09-16T08:57:17.539 INFO:tasks.workunit.client.1.smithi097.stdout:6/802: dwrite d1/dd3/d9e/dca/d16/f79 [4194304,4194304] 0 2024-09-16T08:57:17.539 INFO:tasks.workunit.client.1.smithi097.stdout:9/765: write d2/d5/d72/f89 [964284,24638] 0 2024-09-16T08:57:17.540 INFO:tasks.workunit.client.1.smithi097.stdout:7/707: unlink d4/d7/d77/f94 0 2024-09-16T08:57:17.541 INFO:tasks.workunit.client.1.smithi097.stdout:0/780: dread da/d24/d76/f9a [0,4194304] 0 2024-09-16T08:57:17.545 INFO:tasks.workunit.client.1.smithi097.stdout:8/729: unlink d8/dd/d15/caf 0 2024-09-16T08:57:17.545 INFO:tasks.workunit.client.1.smithi097.stdout:8/730: chown d8/dd/d15/d2c/fe1 18158 1 2024-09-16T08:57:17.545 INFO:tasks.workunit.client.1.smithi097.stdout:8/731: truncate d8/dd/d15/d2c/d4a/d4d/d3d/fcd 890103 0 2024-09-16T08:57:17.545 INFO:tasks.workunit.client.1.smithi097.stdout:8/732: dread - d8/dd/d15/d2c/d4a/d4e/d85/fd6 zero size 2024-09-16T08:57:17.546 INFO:tasks.workunit.client.0.smithi031.stdout:4/336: dread d0/d2/d16/d59/f63 [0,4194304] 0 2024-09-16T08:57:17.546 INFO:tasks.workunit.client.0.smithi031.stdout:4/337: fsync d0/f4c 0 2024-09-16T08:57:17.546 INFO:tasks.workunit.client.0.smithi031.stdout:6/441: mknod d0/c8a 0 2024-09-16T08:57:17.547 INFO:tasks.workunit.client.1.smithi097.stdout:0/781: dread da/d24/d4b/ff4 [0,4194304] 0 2024-09-16T08:57:17.548 INFO:tasks.workunit.client.1.smithi097.stdout:5/847: rename db/d18/d1c/c107 to db/d1a/d6d/c116 0 2024-09-16T08:57:17.549 INFO:tasks.workunit.client.0.smithi031.stdout:6/442: mkdir d0/dd/d74/d8b 0 2024-09-16T08:57:17.549 INFO:tasks.workunit.client.0.smithi031.stdout:6/443: write d0/dd/d74/f7e [856177,55149] 0 2024-09-16T08:57:17.550 INFO:tasks.workunit.client.0.smithi031.stdout:6/444: truncate d0/dd/d74/d5e/d7c/d3f/f48 4542363 0 2024-09-16T08:57:17.551 INFO:tasks.workunit.client.1.smithi097.stdout:1/891: link d5/d17/d54/d6b/lbf d5/d14/l120 0 2024-09-16T08:57:17.551 INFO:tasks.workunit.client.1.smithi097.stdout:1/892: chown d5/d17/d54/d6b/l6e 0 1 2024-09-16T08:57:17.551 INFO:tasks.workunit.client.0.smithi031.stdout:4/338: dread d0/d2/d16/f30 [0,4194304] 0 2024-09-16T08:57:17.552 INFO:tasks.workunit.client.1.smithi097.stdout:1/893: write d5/d2c/d3b/d7f/fdb [125794,97916] 0 2024-09-16T08:57:17.555 INFO:tasks.workunit.client.0.smithi031.stdout:0/378: dwrite d6/fc [0,4194304] 0 2024-09-16T08:57:17.557 INFO:tasks.workunit.client.0.smithi031.stdout:4/339: creat d0/d2/d3a/f6f x:0 0 0 2024-09-16T08:57:17.557 INFO:tasks.workunit.client.0.smithi031.stdout:9/311: sync 2024-09-16T08:57:17.557 INFO:tasks.workunit.client.0.smithi031.stdout:1/296: sync 2024-09-16T08:57:17.557 INFO:tasks.workunit.client.0.smithi031.stdout:7/426: sync 2024-09-16T08:57:17.557 INFO:tasks.workunit.client.0.smithi031.stdout:3/437: sync 2024-09-16T08:57:17.558 INFO:tasks.workunit.client.0.smithi031.stdout:2/273: sync 2024-09-16T08:57:17.559 INFO:tasks.workunit.client.0.smithi031.stdout:6/445: mknod d0/dd/d74/d19/c8c 0 2024-09-16T08:57:17.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/400: sync 2024-09-16T08:57:17.560 INFO:tasks.workunit.client.0.smithi031.stdout:8/287: sync 2024-09-16T08:57:17.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/401: chown d2/dc/l7c 3 1 2024-09-16T08:57:17.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/402: fdatasync d2/f59 0 2024-09-16T08:57:17.562 INFO:tasks.workunit.client.0.smithi031.stdout:4/340: mknod d0/d5b/c70 0 2024-09-16T08:57:17.563 INFO:tasks.workunit.client.1.smithi097.stdout:1/894: dread d5/d17/d54/d6b/d71/d94/f99 [0,4194304] 0 2024-09-16T08:57:17.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/312: truncate d3/f2a 680383 0 2024-09-16T08:57:17.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/313: chown d3/d5/da/l40 2674 1 2024-09-16T08:57:17.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/314: write d3/d5/f6 [1587866,12409] 0 2024-09-16T08:57:17.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/315: stat d3/d5/da/f3e 0 2024-09-16T08:57:17.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/316: chown d3/d5/da/db/c20 1 1 2024-09-16T08:57:17.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/317: rename d3/d48 to d3/d48/d6f 22 2024-09-16T08:57:17.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/318: chown d3/d5/da/d34/d3c/d42/c4c 5116180 1 2024-09-16T08:57:17.567 INFO:tasks.workunit.client.0.smithi031.stdout:2/274: dread f4 [0,4194304] 0 2024-09-16T08:57:17.567 INFO:tasks.workunit.client.1.smithi097.stdout:5/848: dwrite db/d22/f6a [0,4194304] 0 2024-09-16T08:57:17.571 INFO:tasks.workunit.client.0.smithi031.stdout:1/297: link d2/d4/d1b/d5c/f66 d2/d15/d4a/f6a 0 2024-09-16T08:57:17.571 INFO:tasks.workunit.client.0.smithi031.stdout:1/298: write d2/d4/f28 [1025707,84817] 0 2024-09-16T08:57:17.573 INFO:tasks.workunit.client.0.smithi031.stdout:6/446: dread d0/dd/d74/d5e/d7c/d3f/f70 [0,4194304] 0 2024-09-16T08:57:17.574 INFO:tasks.workunit.client.0.smithi031.stdout:7/427: link d2/f56 d2/d8/d11/d5d/d6e/f95 0 2024-09-16T08:57:17.574 INFO:tasks.workunit.client.0.smithi031.stdout:7/428: creat d2/d8/d69/d82/f96 x:0 0 0 2024-09-16T08:57:17.577 INFO:tasks.workunit.client.0.smithi031.stdout:3/438: symlink d3/d14/d37/d1a/d1d/l83 0 2024-09-16T08:57:17.577 INFO:tasks.workunit.client.0.smithi031.stdout:3/439: chown d3/d14/d37/d1a 14247437 1 2024-09-16T08:57:17.582 INFO:tasks.workunit.client.1.smithi097.stdout:6/803: mkdir d1/db7/dfd/d10b 0 2024-09-16T08:57:17.583 INFO:tasks.workunit.client.1.smithi097.stdout:6/804: dread d1/dd8/fe6 [0,4194304] 0 2024-09-16T08:57:17.583 INFO:tasks.workunit.client.1.smithi097.stdout:6/805: write d1/dd3/d9e/db3/fbd [1374554,47590] 0 2024-09-16T08:57:17.583 INFO:tasks.workunit.client.0.smithi031.stdout:9/319: getdents d3/d5/da/db/d21 0 2024-09-16T08:57:17.585 INFO:tasks.workunit.client.0.smithi031.stdout:0/379: dwrite d6/d12/f2d [0,4194304] 0 2024-09-16T08:57:17.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/299: dread d2/f14 [4194304,4194304] 0 2024-09-16T08:57:17.586 INFO:tasks.workunit.client.1.smithi097.stdout:9/766: unlink d2/d5/c1d 0 2024-09-16T08:57:17.587 INFO:tasks.workunit.client.0.smithi031.stdout:2/275: mkdir d6/d2e/d40/d58 0 2024-09-16T08:57:17.587 INFO:tasks.workunit.client.0.smithi031.stdout:4/341: dwrite d0/d2/d16/d1e/f52 [0,4194304] 0 2024-09-16T08:57:17.587 INFO:tasks.workunit.client.0.smithi031.stdout:8/288: dwrite d4/d7/d21/d24/f31 [0,4194304] 0 2024-09-16T08:57:17.588 INFO:tasks.workunit.client.1.smithi097.stdout:7/708: symlink d4/d9/leb 0 2024-09-16T08:57:17.589 INFO:tasks.workunit.client.0.smithi031.stdout:6/447: mknod d0/dd/d74/d5e/d7c/c8d 0 2024-09-16T08:57:17.595 INFO:tasks.workunit.client.0.smithi031.stdout:7/429: symlink d2/d8/d11/d66/l97 0 2024-09-16T08:57:17.596 INFO:tasks.workunit.client.1.smithi097.stdout:8/733: rename d8/dd/d15/d2c/c5d to d8/dd/d15/d2c/d4a/d4d/d3f/ce3 0 2024-09-16T08:57:17.596 INFO:tasks.workunit.client.1.smithi097.stdout:8/734: creat d8/dd/d15/d2c/d4a/d4d/d3d/db9/fe4 x:0 0 0 2024-09-16T08:57:17.598 INFO:tasks.workunit.client.0.smithi031.stdout:3/440: creat d3/d14/d37/f84 x:0 0 0 2024-09-16T08:57:17.601 INFO:tasks.workunit.client.0.smithi031.stdout:5/403: mkdir d2/dc/d25/d21/d81 0 2024-09-16T08:57:17.604 INFO:tasks.workunit.client.1.smithi097.stdout:1/895: mknod d5/d17/d54/d6b/da8/dc5/dc7/df3/c121 0 2024-09-16T08:57:17.605 INFO:tasks.workunit.client.1.smithi097.stdout:7/709: write d4/f64 [3205950,71575] 0 2024-09-16T08:57:17.608 INFO:tasks.workunit.client.1.smithi097.stdout:0/782: rename da/df/fc2 to da/d7e/dc4/dfc/d94/f114 0 2024-09-16T08:57:17.610 INFO:tasks.workunit.client.1.smithi097.stdout:6/806: dwrite d1/dd3/d9e/dca/d16/f4d [4194304,4194304] 0 2024-09-16T08:57:17.610 INFO:tasks.workunit.client.0.smithi031.stdout:8/289: dwrite d4/d2f/f52 [0,4194304] 0 2024-09-16T08:57:17.611 INFO:tasks.workunit.client.0.smithi031.stdout:8/290: truncate d4/f27 4830233 0 2024-09-16T08:57:17.611 INFO:tasks.workunit.client.0.smithi031.stdout:8/291: readlink d4/d7/d36/d38/l3e 0 2024-09-16T08:57:17.611 INFO:tasks.workunit.client.0.smithi031.stdout:1/300: mkdir d2/d4/d1b/d5c/d6b 0 2024-09-16T08:57:17.616 INFO:tasks.workunit.client.0.smithi031.stdout:2/276: symlink d6/l59 0 2024-09-16T08:57:17.619 INFO:tasks.workunit.client.1.smithi097.stdout:8/735: link d8/dd/d15/d16/l88 d8/dd/d15/d2c/d4a/d4d/d3f/d99/le5 0 2024-09-16T08:57:17.620 INFO:tasks.workunit.client.1.smithi097.stdout:8/736: fdatasync d8/dd/d15/d2c/d4a/d4d/d62/f76 0 2024-09-16T08:57:17.620 INFO:tasks.workunit.client.0.smithi031.stdout:4/342: mkdir d0/d71 0 2024-09-16T08:57:17.620 INFO:tasks.workunit.client.1.smithi097.stdout:1/896: link d5/d70/fcc d5/d14/f122 0 2024-09-16T08:57:17.621 INFO:tasks.workunit.client.1.smithi097.stdout:1/897: chown d5/c111 15271298 1 2024-09-16T08:57:17.622 INFO:tasks.workunit.client.0.smithi031.stdout:6/448: mkdir d0/d8e 0 2024-09-16T08:57:17.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/449: write d0/dd/d74/f18 [2255061,102294] 0 2024-09-16T08:57:17.623 INFO:tasks.workunit.client.0.smithi031.stdout:6/450: getdents d0/d8e 0 2024-09-16T08:57:17.624 INFO:tasks.workunit.client.0.smithi031.stdout:7/430: symlink d2/d8/d11/d5d/d7e/l98 0 2024-09-16T08:57:17.627 INFO:tasks.workunit.client.1.smithi097.stdout:9/767: rename d2/d5/d3a/d3c/lae to d2/d5/db0/dc8/d74/lf2 0 2024-09-16T08:57:17.627 INFO:tasks.workunit.client.1.smithi097.stdout:9/768: chown d2/d5/l8f 282612 1 2024-09-16T08:57:17.628 INFO:tasks.workunit.client.1.smithi097.stdout:9/769: dread - d2/d5/d72/da2/fef zero size 2024-09-16T08:57:17.628 INFO:tasks.workunit.client.1.smithi097.stdout:9/770: write d2/d5/db0/dc8/d74/d8c/feb [1033392,31776] 0 2024-09-16T08:57:17.629 INFO:tasks.workunit.client.1.smithi097.stdout:8/737: dread d8/dd/f19 [0,4194304] 0 2024-09-16T08:57:17.632 INFO:tasks.workunit.client.0.smithi031.stdout:5/404: getdents d2/dc/d3d 0 2024-09-16T08:57:17.632 INFO:tasks.workunit.client.0.smithi031.stdout:5/405: chown d2/dc/d1d/f3f 19388 1 2024-09-16T08:57:17.636 INFO:tasks.workunit.client.0.smithi031.stdout:0/380: getdents d6 0 2024-09-16T08:57:17.636 INFO:tasks.workunit.client.0.smithi031.stdout:0/381: truncate d6/d12/d13/f3e 1657302 0 2024-09-16T08:57:17.642 INFO:tasks.workunit.client.1.smithi097.stdout:6/807: rename d1/db7/dfd/fc7 to d1/dd3/d9e/f10c 0 2024-09-16T08:57:17.642 INFO:tasks.workunit.client.1.smithi097.stdout:6/808: truncate d1/dd3/d9e/dca/d16/d86/fff 51117 0 2024-09-16T08:57:17.643 INFO:tasks.workunit.client.0.smithi031.stdout:8/292: creat d4/d5b/f62 x:0 0 0 2024-09-16T08:57:17.644 INFO:tasks.workunit.client.1.smithi097.stdout:9/771: mknod d2/d5/d3a/d70/da6/cf3 0 2024-09-16T08:57:17.646 INFO:tasks.workunit.client.0.smithi031.stdout:1/301: mkdir d2/de/d6c 0 2024-09-16T08:57:17.649 INFO:tasks.workunit.client.1.smithi097.stdout:8/738: mknod d8/dd/ce6 0 2024-09-16T08:57:17.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/320: dwrite d3/f2a [0,4194304] 0 2024-09-16T08:57:17.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/321: write d3/d5/da/db/d21/f41 [920651,60703] 0 2024-09-16T08:57:17.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/322: chown d3/d5/da/d34/d3c 340334993 1 2024-09-16T08:57:17.650 INFO:tasks.workunit.client.0.smithi031.stdout:9/323: chown d3/d5/da/d34/d3c/d5e/d62/d65 41530 1 2024-09-16T08:57:17.651 INFO:tasks.workunit.client.0.smithi031.stdout:1/302: dread d2/d4/d1b/f2b [0,4194304] 0 2024-09-16T08:57:17.652 INFO:tasks.workunit.client.0.smithi031.stdout:2/277: creat d6/d41/f5a x:0 0 0 2024-09-16T08:57:17.652 INFO:tasks.workunit.client.0.smithi031.stdout:2/278: dread - d6/da/ff zero size 2024-09-16T08:57:17.652 INFO:tasks.workunit.client.0.smithi031.stdout:2/279: chown d6/d20/f32 99 1 2024-09-16T08:57:17.653 INFO:tasks.workunit.client.0.smithi031.stdout:4/343: symlink d0/d2/d16/d2b/l72 0 2024-09-16T08:57:17.654 INFO:tasks.workunit.client.1.smithi097.stdout:6/809: read d1/dc/f72 [1334867,13662] 0 2024-09-16T08:57:17.655 INFO:tasks.workunit.client.1.smithi097.stdout:8/739: truncate d8/dd/d15/f29 1881801 0 2024-09-16T08:57:17.656 INFO:tasks.workunit.client.0.smithi031.stdout:6/451: symlink d0/dd/d2b/l8f 0 2024-09-16T08:57:17.659 INFO:tasks.workunit.client.1.smithi097.stdout:6/810: rename d1/db7/dfd/c18 to d1/dd3/d9e/dca/d81/dc4/c10d 0 2024-09-16T08:57:17.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/303: dread d2/d15/d38/f47 [0,4194304] 0 2024-09-16T08:57:17.660 INFO:tasks.workunit.client.0.smithi031.stdout:1/304: dread - d2/d4/d1b/f65 zero size 2024-09-16T08:57:17.660 INFO:tasks.workunit.client.0.smithi031.stdout:7/431: truncate d2/d8/d11/f24 4301694 0 2024-09-16T08:57:17.663 INFO:tasks.workunit.client.1.smithi097.stdout:6/811: mknod d1/dd3/d9e/dca/c10e 0 2024-09-16T08:57:17.663 INFO:tasks.workunit.client.0.smithi031.stdout:3/441: unlink d3/c18 0 2024-09-16T08:57:17.663 INFO:tasks.workunit.client.0.smithi031.stdout:3/442: stat d3/d14/d37/d33/f7a 0 2024-09-16T08:57:17.663 INFO:tasks.workunit.client.0.smithi031.stdout:3/443: chown d3/d14/d37/d33/f3c 1989994 1 2024-09-16T08:57:17.665 INFO:tasks.workunit.client.1.smithi097.stdout:8/740: getdents d8/dd/d15/d2c/d4a/d4d 0 2024-09-16T08:57:17.665 INFO:tasks.workunit.client.0.smithi031.stdout:7/432: dread d2/d8/d11/d5d/d6e/f95 [0,4194304] 0 2024-09-16T08:57:17.666 INFO:tasks.workunit.client.0.smithi031.stdout:9/324: rmdir d3/d5/da/db/d21 39 2024-09-16T08:57:17.667 INFO:tasks.workunit.client.0.smithi031.stdout:2/280: rename d6/da/ff to d6/f5b 0 2024-09-16T08:57:17.667 INFO:tasks.workunit.client.0.smithi031.stdout:2/281: dread - d6/d41/f3b zero size 2024-09-16T08:57:17.668 INFO:tasks.workunit.client.1.smithi097.stdout:0/783: dwrite da/d7e/dc4/dfc/d47/d97/d84/fc5 [0,4194304] 0 2024-09-16T08:57:17.668 INFO:tasks.workunit.client.1.smithi097.stdout:0/784: fsync da/d24/d76/ff3 0 2024-09-16T08:57:17.668 INFO:tasks.workunit.client.1.smithi097.stdout:7/710: dwrite d4/d7/d7b/d43/f44 [0,4194304] 0 2024-09-16T08:57:17.668 INFO:tasks.workunit.client.1.smithi097.stdout:8/741: mknod d8/dd/d15/d2c/d4a/d4d/d3d/ce7 0 2024-09-16T08:57:17.668 INFO:tasks.workunit.client.1.smithi097.stdout:8/742: fsync d8/dd/d15/d2c/d4a/d4e/db2/fb3 0 2024-09-16T08:57:17.671 INFO:tasks.workunit.client.0.smithi031.stdout:2/282: dread d6/da/f3d [0,4194304] 0 2024-09-16T08:57:17.671 INFO:tasks.workunit.client.0.smithi031.stdout:4/344: rename d0/d2/d16/c2c to d0/d2/d16/c73 0 2024-09-16T08:57:17.671 INFO:tasks.workunit.client.0.smithi031.stdout:4/345: creat d0/d2/d16/d2b/f74 x:0 0 0 2024-09-16T08:57:17.674 INFO:tasks.workunit.client.0.smithi031.stdout:6/452: stat d0/dd/d74/d5e/d7c/d56/c71 0 2024-09-16T08:57:17.678 INFO:tasks.workunit.client.0.smithi031.stdout:1/305: symlink d2/d15/l6d 0 2024-09-16T08:57:17.682 INFO:tasks.workunit.client.0.smithi031.stdout:9/325: write d3/d5/da/db/f18 [3197094,93412] 0 2024-09-16T08:57:17.682 INFO:tasks.workunit.client.0.smithi031.stdout:3/444: mkdir d3/d14/d85 0 2024-09-16T08:57:17.682 INFO:tasks.workunit.client.0.smithi031.stdout:3/445: chown d3/d14/d37/f84 15474 1 2024-09-16T08:57:17.685 INFO:tasks.workunit.client.0.smithi031.stdout:0/382: dwrite d6/d12/d13/d3c/f45 [0,4194304] 0 2024-09-16T08:57:17.685 INFO:tasks.workunit.client.1.smithi097.stdout:7/711: dread d4/d9/d2b/f4d [0,4194304] 0 2024-09-16T08:57:17.685 INFO:tasks.workunit.client.1.smithi097.stdout:7/712: truncate d4/d7/d79/fae 855284 0 2024-09-16T08:57:17.685 INFO:tasks.workunit.client.1.smithi097.stdout:0/785: creat da/d7e/dc4/dfc/d94/d101/f115 x:0 0 0 2024-09-16T08:57:17.685 INFO:tasks.workunit.client.1.smithi097.stdout:9/772: dwrite d2/d3f/f43 [0,4194304] 0 2024-09-16T08:57:17.686 INFO:tasks.workunit.client.0.smithi031.stdout:5/406: getdents d2 0 2024-09-16T08:57:17.687 INFO:tasks.workunit.client.1.smithi097.stdout:6/812: dwrite d1/f12 [0,4194304] 0 2024-09-16T08:57:17.687 INFO:tasks.workunit.client.1.smithi097.stdout:1/898: dwrite d5/db/dc/d19/f104 [0,4194304] 0 2024-09-16T08:57:17.688 INFO:tasks.workunit.client.1.smithi097.stdout:8/743: symlink d8/dd/d15/d2c/d4a/d4e/le8 0 2024-09-16T08:57:17.691 INFO:tasks.workunit.client.0.smithi031.stdout:7/433: creat d2/d8/d11/d5d/d6e/f99 x:0 0 0 2024-09-16T08:57:17.692 INFO:tasks.workunit.client.0.smithi031.stdout:6/453: write d0/dd/d74/f1e [3382769,125750] 0 2024-09-16T08:57:17.692 INFO:tasks.workunit.client.0.smithi031.stdout:6/454: chown d0/dd/d74/c2c 80592 1 2024-09-16T08:57:17.695 INFO:tasks.workunit.client.0.smithi031.stdout:2/283: symlink d6/d20/d51/l5c 0 2024-09-16T08:57:17.695 INFO:tasks.workunit.client.0.smithi031.stdout:2/284: chown d6/d41/d2c/c30 4 1 2024-09-16T08:57:17.695 INFO:tasks.workunit.client.0.smithi031.stdout:2/285: creat d6/da/f5d x:0 0 0 2024-09-16T08:57:17.696 INFO:tasks.workunit.client.1.smithi097.stdout:7/713: rename d4/d12/de8 to d4/d12/dec 0 2024-09-16T08:57:17.697 INFO:tasks.workunit.client.1.smithi097.stdout:0/786: link da/d24/d4c/d6d/ldb da/d7e/dc4/dfc/d94/l116 0 2024-09-16T08:57:17.697 INFO:tasks.workunit.client.1.smithi097.stdout:0/787: creat da/d1a/d41/db2/f117 x:0 0 0 2024-09-16T08:57:17.697 INFO:tasks.workunit.client.1.smithi097.stdout:0/788: chown da/d24/d4c/d80/d3b/f4e 29107 1 2024-09-16T08:57:17.699 INFO:tasks.workunit.client.1.smithi097.stdout:9/773: mknod d2/d5/d72/dc1/ddd/dee/dcd/cf4 0 2024-09-16T08:57:17.701 INFO:tasks.workunit.client.0.smithi031.stdout:1/306: mknod d2/d4/d1b/c6e 0 2024-09-16T08:57:17.702 INFO:tasks.workunit.client.0.smithi031.stdout:1/307: creat d2/d4/d52/f6f x:0 0 0 2024-09-16T08:57:17.703 INFO:tasks.workunit.client.1.smithi097.stdout:6/813: rename d1/dc/d3c/f6f to d1/dd3/d9e/dca/d16/d4f/df8/f10f 0 2024-09-16T08:57:17.712 INFO:tasks.workunit.client.0.smithi031.stdout:0/383: creat d6/f7b x:0 0 0 2024-09-16T08:57:17.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/286: write d6/d20/f4f [3358618,19855] 0 2024-09-16T08:57:17.713 INFO:tasks.workunit.client.1.smithi097.stdout:7/714: write d4/d9/dc/fe1 [2894771,82618] 0 2024-09-16T08:57:17.716 INFO:tasks.workunit.client.0.smithi031.stdout:3/446: dwrite d3/d14/d37/d47/f5b [0,4194304] 0 2024-09-16T08:57:17.717 INFO:tasks.workunit.client.0.smithi031.stdout:3/447: readlink d3/d14/d37/l71 0 2024-09-16T08:57:17.717 INFO:tasks.workunit.client.0.smithi031.stdout:3/448: fdatasync d3/f4 0 2024-09-16T08:57:17.717 INFO:tasks.workunit.client.0.smithi031.stdout:4/346: dwrite d0/d2/fe [0,4194304] 0 2024-09-16T08:57:17.717 INFO:tasks.workunit.client.0.smithi031.stdout:9/326: dwrite d3/f5b [0,4194304] 0 2024-09-16T08:57:17.717 INFO:tasks.workunit.client.1.smithi097.stdout:1/899: dwrite d5/db/dc/d19/f8c [0,4194304] 0 2024-09-16T08:57:17.720 INFO:tasks.workunit.client.0.smithi031.stdout:5/407: rename d2/dc/d3d/d4c/c66 to d2/dc/d36/d76/c82 0 2024-09-16T08:57:17.722 INFO:tasks.workunit.client.0.smithi031.stdout:7/434: truncate d2/d8/d11/f31 2776036 0 2024-09-16T08:57:17.728 INFO:tasks.workunit.client.0.smithi031.stdout:6/455: unlink d0/dd/d74/d5e/d7c/c6d 0 2024-09-16T08:57:17.730 INFO:tasks.workunit.client.1.smithi097.stdout:8/744: symlink d8/dd/le9 0 2024-09-16T08:57:17.731 INFO:tasks.workunit.client.1.smithi097.stdout:8/745: chown d8/dd/d15/d16/d7c/l81 1641 1 2024-09-16T08:57:17.731 INFO:tasks.workunit.client.0.smithi031.stdout:5/408: dread d2/dc/f19 [0,4194304] 0 2024-09-16T08:57:17.734 INFO:tasks.workunit.client.0.smithi031.stdout:1/308: mknod d2/c70 0 2024-09-16T08:57:17.736 INFO:tasks.workunit.client.1.smithi097.stdout:0/789: creat da/d7e/dc4/dfc/d47/db3/d60/f118 x:0 0 0 2024-09-16T08:57:17.736 INFO:tasks.workunit.client.0.smithi031.stdout:0/384: mkdir d6/d12/d47/d7c 0 2024-09-16T08:57:17.740 INFO:tasks.workunit.client.1.smithi097.stdout:9/774: rename d2/d5/d72/dc1/ddd/dee/dcd to d2/d5/db0/dc8/d63/da8/dca/df5 0 2024-09-16T08:57:17.740 INFO:tasks.workunit.client.0.smithi031.stdout:2/287: mkdir d6/da/d5e 0 2024-09-16T08:57:17.741 INFO:tasks.workunit.client.0.smithi031.stdout:6/456: write d0/dd/d2b/f31 [2219609,38192] 0 2024-09-16T08:57:17.742 INFO:tasks.workunit.client.1.smithi097.stdout:6/814: mkdir d1/dd8/d110 0 2024-09-16T08:57:17.745 INFO:tasks.workunit.client.1.smithi097.stdout:7/715: creat d4/d12/d1f/fed x:0 0 0 2024-09-16T08:57:17.745 INFO:tasks.workunit.client.1.smithi097.stdout:7/716: fsync d4/f23 0 2024-09-16T08:57:17.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.746+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2690106580 con 0x7f2690100610 2024-09-16T08:57:17.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.746+0000 7f2677fff640 1 -- 172.21.15.31:0/1694206469 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f268005bd00 con 0x7f2690100610 2024-09-16T08:57:17.748 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:57:17.751 INFO:tasks.workunit.client.0.smithi031.stdout:4/347: mkdir d0/d2/d6d/d75 0 2024-09-16T08:57:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2678073d40 msgr2=0x7f2678076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2678073d40 0x7f2678076200 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2684006010 tx=0x7f2684005d50 comp rx=0 tx=0).stop 2024-09-16T08:57:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100610 msgr2=0x7f26901a4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100610 0x7f26901a4520 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f268000e9a0 tx=0x7f268000eda0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 shutdown_connections 2024-09-16T08:57:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f2678073d40 0x7f2678076200 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2690102f40 0x7f26901a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 --2- 172.21.15.31:0/1694206469 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2690100610 0x7f26901a4520 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 >> 172.21.15.31:0/1694206469 conn(0x7f26900fa230 msgr2=0x7f26900fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 shutdown_connections 2024-09-16T08:57:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:17.750+0000 7f26956aa640 1 -- 172.21.15.31:0/1694206469 wait complete. 2024-09-16T08:57:17.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/288: write f1 [1207365,121942] 0 2024-09-16T08:57:17.754 INFO:tasks.workunit.client.1.smithi097.stdout:8/746: dwrite d8/dd/d15/f28 [0,4194304] 0 2024-09-16T08:57:17.754 INFO:tasks.workunit.client.1.smithi097.stdout:8/747: dread d8/dd/d15/d2c/d4a/d4d/d3f/fa6 [0,4194304] 0 2024-09-16T08:57:17.754 INFO:tasks.workunit.client.1.smithi097.stdout:2/824: sync 2024-09-16T08:57:17.754 INFO:tasks.workunit.client.1.smithi097.stdout:5/849: sync 2024-09-16T08:57:17.754 INFO:tasks.workunit.client.1.smithi097.stdout:4/726: sync 2024-09-16T08:57:17.754 INFO:tasks.workunit.client.1.smithi097.stdout:1/900: symlink d5/d2c/d3b/d7f/l123 0 2024-09-16T08:57:17.754 INFO:tasks.workunit.client.1.smithi097.stdout:3/800: sync 2024-09-16T08:57:17.755 INFO:tasks.workunit.client.1.smithi097.stdout:6/815: dread d1/dd3/d9e/dca/d16/d4f/d9f/fa4 [0,4194304] 0 2024-09-16T08:57:17.756 INFO:tasks.workunit.client.1.smithi097.stdout:0/790: mknod da/d7e/dc4/dfc/d94/dc9/dcb/c119 0 2024-09-16T08:57:17.760 INFO:tasks.workunit.client.1.smithi097.stdout:9/775: creat d2/d5/d3a/ff6 x:0 0 0 2024-09-16T08:57:17.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/309: dwrite d2/de/f33 [0,4194304] 0 2024-09-16T08:57:17.766 INFO:tasks.workunit.client.0.smithi031.stdout:9/327: unlink d3/f9 0 2024-09-16T08:57:17.767 INFO:tasks.workunit.client.0.smithi031.stdout:4/348: write d0/d2/d16/d1e/f52 [3479045,102374] 0 2024-09-16T08:57:17.768 INFO:tasks.workunit.client.0.smithi031.stdout:4/349: creat d0/d2/d16/d59/f76 x:0 0 0 2024-09-16T08:57:17.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/717: creat d4/d1b/da4/fee x:0 0 0 2024-09-16T08:57:17.769 INFO:tasks.workunit.client.1.smithi097.stdout:8/748: dread d8/fa [4194304,4194304] 0 2024-09-16T08:57:17.772 INFO:tasks.workunit.client.0.smithi031.stdout:3/449: rename d3/d14/d37/d1a/d1d/d3b/d3f/l5d to d3/d14/d37/d1a/d1d/l86 0 2024-09-16T08:57:17.772 INFO:tasks.workunit.client.0.smithi031.stdout:3/450: write d3/d14/d37/d33/f3c [2577268,60649] 0 2024-09-16T08:57:17.772 INFO:tasks.workunit.client.0.smithi031.stdout:3/451: chown d3/d14/d37/d1a/d1d/d1e/d67 0 1 2024-09-16T08:57:17.772 INFO:tasks.workunit.client.1.smithi097.stdout:5/850: unlink db/d18/d1c/d42/d46/d7e/d76/c82 0 2024-09-16T08:57:17.772 INFO:tasks.workunit.client.1.smithi097.stdout:5/851: chown db/d18/d20/l104 5308532 1 2024-09-16T08:57:17.775 INFO:tasks.workunit.client.1.smithi097.stdout:1/901: link d5/d2c/l5f d5/d17/d54/d6b/d71/d94/l124 0 2024-09-16T08:57:17.777 INFO:tasks.workunit.client.1.smithi097.stdout:6/816: symlink d1/dd3/d9e/dca/d16/d4f/d9f/l111 0 2024-09-16T08:57:17.778 INFO:tasks.workunit.client.0.smithi031.stdout:5/409: symlink d2/dc/d25/d21/d81/l83 0 2024-09-16T08:57:17.779 INFO:tasks.workunit.client.1.smithi097.stdout:0/791: truncate f7 232465 0 2024-09-16T08:57:17.780 INFO:tasks.workunit.client.1.smithi097.stdout:0/792: fsync da/d24/d4c/d80/d3b/d86/faa 0 2024-09-16T08:57:17.781 INFO:tasks.workunit.client.1.smithi097.stdout:9/776: mknod d2/d5/d3a/d70/da6/cf7 0 2024-09-16T08:57:17.784 INFO:tasks.workunit.client.0.smithi031.stdout:3/452: write d3/d14/d37/d1a/d1d/d3b/f4f [1387785,73288] 0 2024-09-16T08:57:17.784 INFO:tasks.workunit.client.0.smithi031.stdout:6/457: creat d0/dd/d74/d8b/f90 x:0 0 0 2024-09-16T08:57:17.785 INFO:tasks.workunit.client.0.smithi031.stdout:2/289: creat d6/da/d4b/d4c/f5f x:0 0 0 2024-09-16T08:57:17.786 INFO:tasks.workunit.client.1.smithi097.stdout:1/902: read d5/d70/d9d/fd7 [945598,34446] 0 2024-09-16T08:57:17.786 INFO:tasks.workunit.client.1.smithi097.stdout:1/903: truncate d5/db/dc/d19/f88 1374063 0 2024-09-16T08:57:17.789 INFO:tasks.workunit.client.1.smithi097.stdout:1/904: dread d5/d2c/dcb/f45 [0,4194304] 0 2024-09-16T08:57:17.789 INFO:tasks.workunit.client.1.smithi097.stdout:8/749: write d8/f7f [1585643,125743] 0 2024-09-16T08:57:17.791 INFO:tasks.workunit.client.1.smithi097.stdout:7/718: creat d4/d7/d7b/fef x:0 0 0 2024-09-16T08:57:17.791 INFO:tasks.workunit.client.1.smithi097.stdout:7/719: chown d4/d7/d79/fae 440635411 1 2024-09-16T08:57:17.793 INFO:tasks.workunit.client.1.smithi097.stdout:6/817: creat d1/dd3/d9e/db3/f112 x:0 0 0 2024-09-16T08:57:17.793 INFO:tasks.workunit.client.1.smithi097.stdout:6/818: write d1/dd3/d9e/dca/d81/dad/f9a [1604275,49776] 0 2024-09-16T08:57:17.793 INFO:tasks.workunit.client.0.smithi031.stdout:1/310: mknod d2/d4/d52/c71 0 2024-09-16T08:57:17.794 INFO:tasks.workunit.client.1.smithi097.stdout:3/801: dwrite d7/de/d3c/f99 [0,4194304] 0 2024-09-16T08:57:17.794 INFO:tasks.workunit.client.0.smithi031.stdout:2/290: dread d6/d20/f3f [0,4194304] 0 2024-09-16T08:57:17.795 INFO:tasks.workunit.client.0.smithi031.stdout:9/328: mknod d3/d5/da/db/d21/d3b/c70 0 2024-09-16T08:57:17.795 INFO:tasks.workunit.client.0.smithi031.stdout:9/329: truncate d3/d5/da/d34/f5c 741487 0 2024-09-16T08:57:17.795 INFO:tasks.workunit.client.0.smithi031.stdout:9/330: chown d3/d5/c1d 15757 1 2024-09-16T08:57:17.801 INFO:tasks.workunit.client.1.smithi097.stdout:0/793: rename da/d24/d76/ff3 to da/f11a 0 2024-09-16T08:57:17.801 INFO:tasks.workunit.client.1.smithi097.stdout:0/794: chown da/d24/d4c/d80/d3b/d86/ce8 31065 1 2024-09-16T08:57:17.802 INFO:tasks.workunit.client.0.smithi031.stdout:6/458: dread d0/dd/d74/d5e/d7c/d3f/f48 [0,4194304] 0 2024-09-16T08:57:17.802 INFO:tasks.workunit.client.0.smithi031.stdout:6/459: creat d0/f91 x:0 0 0 2024-09-16T08:57:17.804 INFO:tasks.workunit.client.0.smithi031.stdout:3/453: read d3/d14/d37/d47/f5b [3388351,22391] 0 2024-09-16T08:57:17.804 INFO:tasks.workunit.client.0.smithi031.stdout:3/454: fsync d3/d14/d37/d1a/d1d/f75 0 2024-09-16T08:57:17.804 INFO:tasks.workunit.client.1.smithi097.stdout:5/852: dwrite db/f56 [0,4194304] 0 2024-09-16T08:57:17.805 INFO:tasks.workunit.client.1.smithi097.stdout:2/825: dwrite d3/d4/d68/d7f/fbd [0,4194304] 0 2024-09-16T08:57:17.805 INFO:tasks.workunit.client.1.smithi097.stdout:3/802: write d7/de/d3c/f99 [3793292,58766] 0 2024-09-16T08:57:17.805 INFO:tasks.workunit.client.1.smithi097.stdout:5/853: stat db/d18/d1c/d42/d46/d7e/c10e 0 2024-09-16T08:57:17.806 INFO:tasks.workunit.client.0.smithi031.stdout:1/311: write d2/d4/f5 [6831441,82177] 0 2024-09-16T08:57:17.811 INFO:tasks.workunit.client.0.smithi031.stdout:4/350: rename d0/f5f to d0/d2/d6d/f77 0 2024-09-16T08:57:17.811 INFO:tasks.workunit.client.0.smithi031.stdout:4/351: chown d0/f2d 912029 1 2024-09-16T08:57:17.812 INFO:tasks.workunit.client.1.smithi097.stdout:0/795: dread da/d7e/dc4/dfc/d47/db3/d60/d73/d85/f9f [0,4194304] 0 2024-09-16T08:57:17.813 INFO:tasks.workunit.client.0.smithi031.stdout:5/410: symlink d2/l84 0 2024-09-16T08:57:17.818 INFO:tasks.workunit.client.0.smithi031.stdout:9/331: creat d3/d5/da/db/d21/d67/f71 x:0 0 0 2024-09-16T08:57:17.818 INFO:tasks.workunit.client.0.smithi031.stdout:9/332: write d3/d48/f49 [280809,52934] 0 2024-09-16T08:57:17.819 INFO:tasks.workunit.client.0.smithi031.stdout:3/455: dread d3/f59 [0,4194304] 0 2024-09-16T08:57:17.821 INFO:tasks.workunit.client.1.smithi097.stdout:6/819: mknod d1/dd3/d9e/dca/c113 0 2024-09-16T08:57:17.826 INFO:tasks.workunit.client.0.smithi031.stdout:6/460: mkdir d0/dd/d74/d5e/d7c/d3f/d92 0 2024-09-16T08:57:17.830 INFO:tasks.workunit.client.0.smithi031.stdout:1/312: link d2/d4/f31 d2/d4/d67/f72 0 2024-09-16T08:57:17.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/291: rename d6/d41/d28 to d6/d41/d2c/d53/d60 0 2024-09-16T08:57:17.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/292: fdatasync d6/da/fd 0 2024-09-16T08:57:17.834 INFO:tasks.workunit.client.1.smithi097.stdout:8/750: dwrite d8/dd/d15/d2c/d4a/d4d/fbe [0,4194304] 0 2024-09-16T08:57:17.834 INFO:tasks.workunit.client.1.smithi097.stdout:9/777: dwrite d2/d5/db0/dc8/d63/da8/dca/fd3 [0,4194304] 0 2024-09-16T08:57:17.835 INFO:tasks.workunit.client.0.smithi031.stdout:4/352: mkdir d0/d2/d3a/d78 0 2024-09-16T08:57:17.838 INFO:tasks.workunit.client.0.smithi031.stdout:5/411: unlink d2/f5 0 2024-09-16T08:57:17.839 INFO:tasks.workunit.client.1.smithi097.stdout:2/826: link d3/d4/d68/d8b/d7c/d7d/fc6 d3/d4/d8/f100 0 2024-09-16T08:57:17.845 INFO:tasks.workunit.client.1.smithi097.stdout:7/720: dwrite d4/d7/d30/f5b [0,4194304] 0 2024-09-16T08:57:17.845 INFO:tasks.workunit.client.0.smithi031.stdout:6/461: getdents d0/dd/d74/d5e/d7c/d3f/d65 0 2024-09-16T08:57:17.846 INFO:tasks.workunit.client.1.smithi097.stdout:5/854: creat db/d18/d20/d2b/f117 x:0 0 0 2024-09-16T08:57:17.846 INFO:tasks.workunit.client.1.smithi097.stdout:5/855: creat db/f118 x:0 0 0 2024-09-16T08:57:17.846 INFO:tasks.workunit.client.1.smithi097.stdout:5/856: fsync db/d18/d1c/d42/d46/d7e/fe8 0 2024-09-16T08:57:17.846 INFO:tasks.workunit.client.1.smithi097.stdout:1/905: getdents d5/db/dc/d19 0 2024-09-16T08:57:17.846 INFO:tasks.workunit.client.1.smithi097.stdout:1/906: fsync d5/d2c/d3b/f80 0 2024-09-16T08:57:17.846 INFO:tasks.workunit.client.1.smithi097.stdout:1/907: write d5/db/dc/fad [31346,4146] 0 2024-09-16T08:57:17.847 INFO:tasks.workunit.client.1.smithi097.stdout:1/908: fsync d5/d2c/f5e 0 2024-09-16T08:57:17.847 INFO:tasks.workunit.client.0.smithi031.stdout:1/313: rmdir d2/de 39 2024-09-16T08:57:17.849 INFO:tasks.workunit.client.1.smithi097.stdout:3/803: dwrite d7/de/d3c/d3f/d60/dcb/f10a [0,4194304] 0 2024-09-16T08:57:17.849 INFO:tasks.workunit.client.0.smithi031.stdout:9/333: rename d3/d5/da/d34/d4d/c61 to d3/d5/da/db/c72 0 2024-09-16T08:57:17.849 INFO:tasks.workunit.client.0.smithi031.stdout:1/314: dread d2/d4/d67/f72 [0,4194304] 0 2024-09-16T08:57:17.849 INFO:tasks.workunit.client.0.smithi031.stdout:1/315: fsync d2/d15/d1f/f62 0 2024-09-16T08:57:17.850 INFO:tasks.workunit.client.1.smithi097.stdout:6/820: dwrite d1/dd3/d9e/dca/f4b [0,4194304] 0 2024-09-16T08:57:17.850 INFO:tasks.workunit.client.1.smithi097.stdout:6/821: fdatasync d1/dd3/d9e/dca/f11 0 2024-09-16T08:57:17.851 INFO:tasks.workunit.client.0.smithi031.stdout:5/412: creat d2/dc/d25/d21/f85 x:0 0 0 2024-09-16T08:57:17.852 INFO:tasks.workunit.client.0.smithi031.stdout:5/413: truncate d2/dc/d25/d21/d2e/f72 4350636 0 2024-09-16T08:57:17.853 INFO:tasks.workunit.client.1.smithi097.stdout:8/751: creat d8/dd/d71/fea x:0 0 0 2024-09-16T08:57:17.853 INFO:tasks.workunit.client.1.smithi097.stdout:9/778: mkdir d2/df8 0 2024-09-16T08:57:17.853 INFO:tasks.workunit.client.1.smithi097.stdout:9/779: readlink d2/d5/l39 0 2024-09-16T08:57:17.854 INFO:tasks.workunit.client.1.smithi097.stdout:2/827: mknod d3/d4/d68/d8b/d7c/d7d/d53/c101 0 2024-09-16T08:57:17.854 INFO:tasks.workunit.client.1.smithi097.stdout:2/828: fdatasync d3/f9b 0 2024-09-16T08:57:17.855 INFO:tasks.workunit.client.0.smithi031.stdout:9/334: mkdir d3/d5/d73 0 2024-09-16T08:57:17.856 INFO:tasks.workunit.client.0.smithi031.stdout:9/335: fsync d3/d5/da/d34/f5c 0 2024-09-16T08:57:17.856 INFO:tasks.workunit.client.1.smithi097.stdout:7/721: rename d4/d1b/d41/fb1 to d4/d9/d1c/ff0 0 2024-09-16T08:57:17.858 INFO:tasks.workunit.client.0.smithi031.stdout:2/293: rename d6/d41/c25 to d6/d41/d2c/c61 0 2024-09-16T08:57:17.860 INFO:tasks.workunit.client.1.smithi097.stdout:5/857: symlink db/d6e/d79/l119 0 2024-09-16T08:57:17.864 INFO:tasks.workunit.client.0.smithi031.stdout:5/414: dread d2/dc/f3a [0,4194304] 0 2024-09-16T08:57:17.865 INFO:tasks.workunit.client.1.smithi097.stdout:1/909: mkdir d5/db/dc/d19/d125 0 2024-09-16T08:57:17.869 INFO:tasks.workunit.client.1.smithi097.stdout:6/822: dread d1/dd3/d9e/dca/f11 [0,4194304] 0 2024-09-16T08:57:17.869 INFO:tasks.workunit.client.1.smithi097.stdout:6/823: chown d1/lec 16470871 1 2024-09-16T08:57:17.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/316: symlink d2/d15/d5d/l73 0 2024-09-16T08:57:17.877 INFO:tasks.workunit.client.0.smithi031.stdout:2/294: creat d6/d20/d51/f62 x:0 0 0 2024-09-16T08:57:17.879 INFO:tasks.workunit.client.1.smithi097.stdout:9/780: creat d2/d5/db0/dc8/d63/da8/dca/df5/dd8/ff9 x:0 0 0 2024-09-16T08:57:17.879 INFO:tasks.workunit.client.1.smithi097.stdout:9/781: write d2/d5/d3a/d70/da6/fd9 [1368418,75622] 0 2024-09-16T08:57:17.880 INFO:tasks.workunit.client.0.smithi031.stdout:5/415: link d2/dc/d25/d68/f6b d2/dc/d3d/f86 0 2024-09-16T08:57:17.881 INFO:tasks.workunit.client.1.smithi097.stdout:9/782: read d2/d5/f4e [4819382,127245] 0 2024-09-16T08:57:17.881 INFO:tasks.workunit.client.1.smithi097.stdout:9/783: chown d2/f31 24653761 1 2024-09-16T08:57:17.881 INFO:tasks.workunit.client.1.smithi097.stdout:9/784: write d2/f28 [3217004,86042] 0 2024-09-16T08:57:17.882 INFO:tasks.workunit.client.0.smithi031.stdout:4/353: dwrite d0/f1 [0,4194304] 0 2024-09-16T08:57:17.882 INFO:tasks.workunit.client.0.smithi031.stdout:4/354: chown d0/d2/d16/c20 19287953 1 2024-09-16T08:57:17.882 INFO:tasks.workunit.client.0.smithi031.stdout:4/355: write d0/d2/d16/f4f [964681,32804] 0 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:3/804: dwrite d7/de/d3c/f99 [0,4194304] 0 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:2/829: mkdir d3/d4/d68/d8b/d7c/d7d/d53/da3/d102 0 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:2/830: chown d3/d4/d68/d8b/d7c/lb0 637 1 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:2/831: readlink d3/d4/d9/l8a 0 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:9/785: dread d2/d5/f29 [0,4194304] 0 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:9/786: chown d2/f4 63770661 1 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:9/787: chown d2/d5/db0/dc8/d74/de0 112000309 1 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.1.smithi097.stdout:5/858: dwrite db/d18/d20/fc5 [0,4194304] 0 2024-09-16T08:57:17.883 INFO:tasks.workunit.client.0.smithi031.stdout:6/462: dwrite d0/d10/f3b [0,4194304] 0 2024-09-16T08:57:17.885 INFO:tasks.workunit.client.0.smithi031.stdout:1/317: rename d2/d4/f46 to d2/d4/d52/f74 0 2024-09-16T08:57:17.885 INFO:tasks.workunit.client.0.smithi031.stdout:1/318: write d2/d15/d4a/f6a [802657,30836] 0 2024-09-16T08:57:17.886 INFO:tasks.workunit.client.0.smithi031.stdout:2/295: dread d6/da/fe [0,4194304] 0 2024-09-16T08:57:17.886 INFO:tasks.workunit.client.0.smithi031.stdout:2/296: write d6/f36 [528784,65392] 0 2024-09-16T08:57:17.886 INFO:tasks.workunit.client.0.smithi031.stdout:2/297: read - d6/f5b zero size 2024-09-16T08:57:17.886 INFO:tasks.workunit.client.0.smithi031.stdout:2/298: truncate d6/d20/f26 1409251 0 2024-09-16T08:57:17.887 INFO:tasks.workunit.client.0.smithi031.stdout:2/299: stat d6/d2e 0 2024-09-16T08:57:17.887 INFO:tasks.workunit.client.1.smithi097.stdout:7/722: rename d4/d7/d7b/f46 to d4/d12/d1f/d22/ff1 0 2024-09-16T08:57:17.887 INFO:tasks.workunit.client.1.smithi097.stdout:7/723: chown d4/d9/d1c 757 1 2024-09-16T08:57:17.887 INFO:tasks.workunit.client.1.smithi097.stdout:7/724: creat d4/d1b/d41/ff2 x:0 0 0 2024-09-16T08:57:17.889 INFO:tasks.workunit.client.0.smithi031.stdout:5/416: creat d2/f87 x:0 0 0 2024-09-16T08:57:17.891 INFO:tasks.workunit.client.1.smithi097.stdout:1/910: link d5/d2c/d3b/f61 d5/d70/d85/f126 0 2024-09-16T08:57:17.895 INFO:tasks.workunit.client.0.smithi031.stdout:4/356: dread d0/d2/f33 [0,4194304] 0 2024-09-16T08:57:17.895 INFO:tasks.workunit.client.1.smithi097.stdout:1/911: write d5/d70/d85/f126 [511072,74960] 0 2024-09-16T08:57:17.896 INFO:tasks.workunit.client.0.smithi031.stdout:6/463: stat d0/d10/l44 0 2024-09-16T08:57:17.896 INFO:tasks.workunit.client.0.smithi031.stdout:6/464: readlink d0/dd/d74/d5e/l5f 0 2024-09-16T08:57:17.896 INFO:tasks.workunit.client.0.smithi031.stdout:6/465: creat d0/dd/d35/d6c/d83/f93 x:0 0 0 2024-09-16T08:57:17.903 INFO:tasks.workunit.client.1.smithi097.stdout:3/805: link d7/d28/d30/d69/le5 d7/de/d6c/dcf/l116 0 2024-09-16T08:57:17.903 INFO:tasks.workunit.client.1.smithi097.stdout:3/806: chown d7/de/d3c/f99 82276207 1 2024-09-16T08:57:17.903 INFO:tasks.workunit.client.1.smithi097.stdout:3/807: creat d7/de/d9c/daf/f117 x:0 0 0 2024-09-16T08:57:17.904 INFO:tasks.workunit.client.1.smithi097.stdout:5/859: dread db/d22/d58/f10b [0,4194304] 0 2024-09-16T08:57:17.904 INFO:tasks.workunit.client.0.smithi031.stdout:4/357: write d0/f2d [1631899,21965] 0 2024-09-16T08:57:17.904 INFO:tasks.workunit.client.1.smithi097.stdout:8/752: write d8/dd/d15/d2c/d4a/d4d/fbe [4562934,3151] 0 2024-09-16T08:57:17.905 INFO:tasks.workunit.client.1.smithi097.stdout:2/832: rmdir d3/d4/d68/d8b/d61 39 2024-09-16T08:57:17.907 INFO:tasks.workunit.client.1.smithi097.stdout:9/788: getdents d2/d5/db0 0 2024-09-16T08:57:17.908 INFO:tasks.workunit.client.0.smithi031.stdout:4/358: dread d0/d2/d16/d2b/f67 [0,4194304] 0 2024-09-16T08:57:17.908 INFO:tasks.workunit.client.0.smithi031.stdout:4/359: chown d0/d2/d16/d1e/f32 53 1 2024-09-16T08:57:17.912 INFO:tasks.workunit.client.0.smithi031.stdout:6/466: dread d0/d10/f5a [0,4194304] 0 2024-09-16T08:57:17.913 INFO:tasks.workunit.client.1.smithi097.stdout:9/789: dread d2/d5/d56/f88 [0,4194304] 0 2024-09-16T08:57:17.913 INFO:tasks.workunit.client.0.smithi031.stdout:5/417: dwrite d2/dc/d25/d68/f6e [0,4194304] 0 2024-09-16T08:57:17.913 INFO:tasks.workunit.client.0.smithi031.stdout:5/418: write d2/dc/d3d/f7e [978075,96765] 0 2024-09-16T08:57:17.914 INFO:tasks.workunit.client.0.smithi031.stdout:5/419: creat d2/dc/f88 x:0 0 0 2024-09-16T08:57:17.914 INFO:tasks.workunit.client.0.smithi031.stdout:4/360: dread d0/d2/d16/d1e/f3f [0,4194304] 0 2024-09-16T08:57:17.919 INFO:tasks.workunit.client.0.smithi031.stdout:6/467: symlink d0/dd/d74/d8b/l94 0 2024-09-16T08:57:17.919 INFO:tasks.workunit.client.0.smithi031.stdout:6/468: fdatasync d0/f84 0 2024-09-16T08:57:17.919 INFO:tasks.workunit.client.0.smithi031.stdout:6/469: fsync d0/dd/f23 0 2024-09-16T08:57:17.919 INFO:tasks.workunit.client.1.smithi097.stdout:7/725: dwrite d4/d12/d1f/d22/ff1 [0,4194304] 0 2024-09-16T08:57:17.920 INFO:tasks.workunit.client.1.smithi097.stdout:7/726: fsync d4/d9/d1c/f38 0 2024-09-16T08:57:17.922 INFO:tasks.workunit.client.0.smithi031.stdout:4/361: mknod d0/d71/c79 0 2024-09-16T08:57:17.924 INFO:tasks.workunit.client.0.smithi031.stdout:6/470: symlink d0/dd/d2b/l95 0 2024-09-16T08:57:17.926 INFO:tasks.workunit.client.0.smithi031.stdout:6/471: rename d0/dd/d74/d5e/d7c/d3f/f4c to d0/dd/d74/f96 0 2024-09-16T08:57:17.927 INFO:tasks.workunit.client.1.smithi097.stdout:6/824: getdents d1/dd3/d9e/dca/d16 0 2024-09-16T08:57:17.929 INFO:tasks.workunit.client.0.smithi031.stdout:6/472: truncate d0/f4 7954573 0 2024-09-16T08:57:17.929 INFO:tasks.workunit.client.0.smithi031.stdout:6/473: chown d0/dd/d35/d6c/d83 484145511 1 2024-09-16T08:57:17.929 INFO:tasks.workunit.client.0.smithi031.stdout:6/474: getdents d0/d8e 0 2024-09-16T08:57:17.929 INFO:tasks.workunit.client.0.smithi031.stdout:6/475: truncate d0/dd/d74/d8b/f90 694010 0 2024-09-16T08:57:17.929 INFO:tasks.workunit.client.0.smithi031.stdout:6/476: chown d0/l5c 3247 1 2024-09-16T08:57:17.930 INFO:tasks.workunit.client.1.smithi097.stdout:1/912: unlink d5/d14/cb6 0 2024-09-16T08:57:17.930 INFO:tasks.workunit.client.1.smithi097.stdout:1/913: write d5/d70/fcc [227158,67078] 0 2024-09-16T08:57:17.930 INFO:tasks.workunit.client.1.smithi097.stdout:1/914: chown d5/d70/d85/db5 8238938 1 2024-09-16T08:57:17.931 INFO:tasks.workunit.client.1.smithi097.stdout:1/915: creat d5/d17/d54/f127 x:0 0 0 2024-09-16T08:57:17.933 INFO:tasks.workunit.client.0.smithi031.stdout:6/477: symlink d0/l97 0 2024-09-16T08:57:17.934 INFO:tasks.workunit.client.1.smithi097.stdout:5/860: symlink db/d1a/l11a 0 2024-09-16T08:57:17.935 INFO:tasks.workunit.client.1.smithi097.stdout:8/753: creat d8/dd/d15/d16/d7c/feb x:0 0 0 2024-09-16T08:57:17.937 INFO:tasks.workunit.client.0.smithi031.stdout:6/478: creat d0/dd/d74/d5e/f98 x:0 0 0 2024-09-16T08:57:17.938 INFO:tasks.workunit.client.1.smithi097.stdout:2/833: mkdir d3/d4/d8/d103 0 2024-09-16T08:57:17.938 INFO:tasks.workunit.client.1.smithi097.stdout:2/834: chown d3/d4/d68/d7f/lca 5041355 1 2024-09-16T08:57:17.939 INFO:tasks.workunit.client.1.smithi097.stdout:9/790: symlink d2/d5/db0/dc8/d63/da8/dca/df5/lfa 0 2024-09-16T08:57:17.940 INFO:tasks.workunit.client.1.smithi097.stdout:3/808: dwrite d7/d28/d63/d70/fdc [0,4194304] 0 2024-09-16T08:57:17.942 INFO:tasks.workunit.client.0.smithi031.stdout:4/362: dread d0/d2/f10 [0,4194304] 0 2024-09-16T08:57:17.942 INFO:tasks.workunit.client.1.smithi097.stdout:7/727: truncate d4/d7/d30/f5c 1167528 0 2024-09-16T08:57:17.943 INFO:tasks.workunit.client.1.smithi097.stdout:2/835: dread d3/d4/d68/d8b/d7c/d7d/d53/f58 [0,4194304] 0 2024-09-16T08:57:17.946 INFO:tasks.workunit.client.1.smithi097.stdout:6/825: mknod d1/dd3/c114 0 2024-09-16T08:57:17.946 INFO:tasks.workunit.client.1.smithi097.stdout:6/826: chown d1/dd3/d9e/dca/d16/d4f/l58 1923 1 2024-09-16T08:57:17.948 INFO:tasks.workunit.client.0.smithi031.stdout:1/319: getdents d2/d4 0 2024-09-16T08:57:17.953 INFO:tasks.workunit.client.1.smithi097.stdout:9/791: dread d2/d3f/f43 [0,4194304] 0 2024-09-16T08:57:17.958 INFO:tasks.workunit.client.1.smithi097.stdout:3/809: truncate d7/de/d3c/d3f/d60/dcb/d35/d3d/f112 676138 0 2024-09-16T08:57:17.958 INFO:tasks.workunit.client.1.smithi097.stdout:3/810: write d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/f78 [1580514,21890] 0 2024-09-16T08:57:17.958 INFO:tasks.workunit.client.1.smithi097.stdout:9/792: dread d2/d5/d3a/d3c/f85 [0,4194304] 0 2024-09-16T08:57:17.962 INFO:tasks.workunit.client.1.smithi097.stdout:7/728: rename d4/d9/d1c/l45 to d4/d7/d77/lf3 0 2024-09-16T08:57:17.962 INFO:tasks.workunit.client.1.smithi097.stdout:7/729: chown d4/d7/d77 491734062 1 2024-09-16T08:57:17.962 INFO:tasks.workunit.client.1.smithi097.stdout:7/730: write d4/d9/dc/f49 [2231895,125697] 0 2024-09-16T08:57:17.963 INFO:tasks.workunit.client.0.smithi031.stdout:6/479: rmdir d0/dd/d35/d6c/d83 39 2024-09-16T08:57:17.963 INFO:tasks.workunit.client.0.smithi031.stdout:1/320: mkdir d2/d15/d1e/d75 0 2024-09-16T08:57:17.963 INFO:tasks.workunit.client.0.smithi031.stdout:1/321: fdatasync d2/d4/f1c 0 2024-09-16T08:57:17.965 INFO:tasks.workunit.client.0.smithi031.stdout:6/480: mknod d0/dd/d35/d6c/c99 0 2024-09-16T08:57:17.966 INFO:tasks.workunit.client.1.smithi097.stdout:2/836: mkdir d3/d4/d68/d104 0 2024-09-16T08:57:17.966 INFO:tasks.workunit.client.1.smithi097.stdout:2/837: chown d3/d4/d68/d8b/d7c/d7d/f8d 550987 1 2024-09-16T08:57:17.967 INFO:tasks.workunit.client.0.smithi031.stdout:1/322: creat d2/d15/d38/f76 x:0 0 0 2024-09-16T08:57:17.968 INFO:tasks.workunit.client.1.smithi097.stdout:6/827: symlink d1/dd3/d9e/dca/dbb/l115 0 2024-09-16T08:57:17.968 INFO:tasks.workunit.client.1.smithi097.stdout:6/828: chown d1/dd3/d9e/db3/fbd 237079 1 2024-09-16T08:57:17.969 INFO:tasks.workunit.client.1.smithi097.stdout:9/793: mkdir d2/d5/d3a/dfb 0 2024-09-16T08:57:17.969 INFO:tasks.workunit.client.1.smithi097.stdout:9/794: readlink d2/d5/l20 0 2024-09-16T08:57:17.970 INFO:tasks.workunit.client.1.smithi097.stdout:3/811: dwrite d7/de/d9c/daf/f117 [0,4194304] 0 2024-09-16T08:57:17.970 INFO:tasks.workunit.client.1.smithi097.stdout:3/812: readlink d7/de/d3c/d3f/l4c 0 2024-09-16T08:57:17.970 INFO:tasks.workunit.client.1.smithi097.stdout:3/813: chown d7/d28/d30/d88/fc1 2 1 2024-09-16T08:57:17.971 INFO:tasks.workunit.client.0.smithi031.stdout:1/323: truncate d2/d4/f5 5827161 0 2024-09-16T08:57:17.971 INFO:tasks.workunit.client.1.smithi097.stdout:2/838: dread d3/d4/d8/f7a [0,4194304] 0 2024-09-16T08:57:17.973 INFO:tasks.workunit.client.0.smithi031.stdout:6/481: mkdir d0/dd/d74/d19/d9a 0 2024-09-16T08:57:17.974 INFO:tasks.workunit.client.1.smithi097.stdout:7/731: truncate d4/d9/f24 1680250 0 2024-09-16T08:57:17.975 INFO:tasks.workunit.client.1.smithi097.stdout:6/829: creat d1/dd8/f116 x:0 0 0 2024-09-16T08:57:17.975 INFO:tasks.workunit.client.1.smithi097.stdout:6/830: write d1/dd3/d9e/dca/d16/d86/fb2 [5240112,4234] 0 2024-09-16T08:57:17.975 INFO:tasks.workunit.client.1.smithi097.stdout:6/831: fsync d1/dd3/d9e/dca/f2c 0 2024-09-16T08:57:17.976 INFO:tasks.workunit.client.0.smithi031.stdout:1/324: mknod d2/d15/d5d/c77 0 2024-09-16T08:57:17.976 INFO:tasks.workunit.client.0.smithi031.stdout:1/325: fsync f0 0 2024-09-16T08:57:17.976 INFO:tasks.workunit.client.0.smithi031.stdout:1/326: creat d2/d4/d1b/f78 x:0 0 0 2024-09-16T08:57:17.976 INFO:tasks.workunit.client.0.smithi031.stdout:6/482: write d0/dd/d74/f2d [1135142,16377] 0 2024-09-16T08:57:17.983 INFO:tasks.workunit.client.1.smithi097.stdout:3/814: truncate d7/de/d3c/d3f/d60/dcb/d35/f67 21288 0 2024-09-16T08:57:17.983 INFO:tasks.workunit.client.1.smithi097.stdout:3/815: write d7/de/d3c/d3f/d60/dcb/fe0 [511395,31444] 0 2024-09-16T08:57:17.983 INFO:tasks.workunit.client.1.smithi097.stdout:3/816: write d7/de/d3c/f45 [3209769,67347] 0 2024-09-16T08:57:17.984 INFO:tasks.workunit.client.0.smithi031.stdout:1/327: rename d2/de/f33 to d2/de/f79 0 2024-09-16T08:57:17.990 INFO:tasks.workunit.client.0.smithi031.stdout:6/483: rename d0/dd/d74/d5e/d7c/d3f/f70 to d0/dd/d35/d6c/f9b 0 2024-09-16T08:57:17.991 INFO:tasks.workunit.client.1.smithi097.stdout:7/732: link d4/d9/d1c/d5f/c61 d4/d9/d51/db7/cf4 0 2024-09-16T08:57:17.993 INFO:tasks.workunit.client.0.smithi031.stdout:6/484: creat d0/dd/d74/d19/d54/f9c x:0 0 0 2024-09-16T08:57:17.994 INFO:tasks.workunit.client.0.smithi031.stdout:6/485: unlink d0/dd/l86 0 2024-09-16T08:57:17.996 INFO:tasks.workunit.client.0.smithi031.stdout:6/486: rename d0/dd/d35/c4d to d0/c9d 0 2024-09-16T08:57:17.996 INFO:tasks.workunit.client.0.smithi031.stdout:6/487: fsync d0/dd/d74/f55 0 2024-09-16T08:57:17.997 INFO:tasks.workunit.client.1.smithi097.stdout:6/832: dwrite d1/dd3/d9e/dca/dbb/fd1 [0,4194304] 0 2024-09-16T08:57:18.001 INFO:tasks.workunit.client.0.smithi031.stdout:1/328: dwrite d2/f29 [0,4194304] 0 2024-09-16T08:57:18.001 INFO:tasks.workunit.client.0.smithi031.stdout:1/329: readlink d2/d15/l6d 0 2024-09-16T08:57:18.002 INFO:tasks.workunit.client.1.smithi097.stdout:6/833: dread d1/dd3/d9e/dca/d16/f21 [0,4194304] 0 2024-09-16T08:57:18.003 INFO:tasks.workunit.client.0.smithi031.stdout:1/330: rename d2/d4/c69 to d2/d15/d5d/c7a 0 2024-09-16T08:57:18.004 INFO:tasks.workunit.client.1.smithi097.stdout:6/834: rename d1/dc/l77 to d1/dd3/d9e/dca/db6/l117 0 2024-09-16T08:57:18.004 INFO:tasks.workunit.client.0.smithi031.stdout:1/331: creat d2/d4/d1b/d5c/d6b/f7b x:0 0 0 2024-09-16T08:57:18.005 INFO:tasks.workunit.client.1.smithi097.stdout:6/835: link d1/dd3/d9e/dca/c31 d1/dd3/d9e/dca/d16/c118 0 2024-09-16T08:57:18.006 INFO:tasks.workunit.client.1.smithi097.stdout:6/836: symlink d1/dc/l119 0 2024-09-16T08:57:18.009 INFO:tasks.workunit.client.1.smithi097.stdout:6/837: dread d1/f5b [0,4194304] 0 2024-09-16T08:57:18.010 INFO:tasks.workunit.client.1.smithi097.stdout:6/838: link d1/dc/c67 d1/dd3/d9e/db3/c11a 0 2024-09-16T08:57:18.011 INFO:tasks.workunit.client.1.smithi097.stdout:6/839: symlink d1/dd3/d9e/dca/d81/dc4/l11b 0 2024-09-16T08:57:18.011 INFO:tasks.workunit.client.1.smithi097.stdout:6/840: chown d1/dd3/d9e/dca/d81/dad/l43 28282 1 2024-09-16T08:57:18.012 INFO:tasks.workunit.client.0.smithi031.stdout:1/332: write d2/d15/d1f/d43/f4c [3044702,26452] 0 2024-09-16T08:57:18.012 INFO:tasks.workunit.client.0.smithi031.stdout:1/333: chown d2/d4/d1b/f2b 20192 1 2024-09-16T08:57:18.012 INFO:tasks.workunit.client.0.smithi031.stdout:1/334: fdatasync d2/f17 0 2024-09-16T08:57:18.012 INFO:tasks.workunit.client.0.smithi031.stdout:1/335: chown d2/f3c 2529 1 2024-09-16T08:57:18.013 INFO:tasks.workunit.client.1.smithi097.stdout:6/841: rename d1/dd3/d9e/dca/d16/d4f/d9f/cae to d1/db7/c11c 0 2024-09-16T08:57:18.014 INFO:tasks.workunit.client.1.smithi097.stdout:0/796: sync 2024-09-16T08:57:18.014 INFO:tasks.workunit.client.1.smithi097.stdout:0/797: dread - da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/fc7 zero size 2024-09-16T08:57:18.016 INFO:tasks.workunit.client.1.smithi097.stdout:5/861: dwrite db/d1a/d6d/dce/ff3 [0,4194304] 0 2024-09-16T08:57:18.016 INFO:tasks.workunit.client.1.smithi097.stdout:4/727: sync 2024-09-16T08:57:18.017 INFO:tasks.workunit.client.1.smithi097.stdout:4/728: creat d0/d35/da4/dd1/f105 x:0 0 0 2024-09-16T08:57:18.019 INFO:tasks.workunit.client.1.smithi097.stdout:0/798: symlink da/d7e/dc4/dfc/d94/l11b 0 2024-09-16T08:57:18.019 INFO:tasks.workunit.client.1.smithi097.stdout:0/799: creat da/d7e/f11c x:0 0 0 2024-09-16T08:57:18.019 INFO:tasks.workunit.client.1.smithi097.stdout:0/800: stat da/d7e/dc4/dfc/d47/db3/d60/d68/cb5 0 2024-09-16T08:57:18.019 INFO:tasks.workunit.client.1.smithi097.stdout:0/801: fsync da/d1a/d41/db2/fd6 0 2024-09-16T08:57:18.021 INFO:tasks.workunit.client.1.smithi097.stdout:5/862: mkdir db/d18/d1c/d42/d46/d7e/d8f/d102/d11b 0 2024-09-16T08:57:18.026 INFO:tasks.workunit.client.1.smithi097.stdout:4/729: stat d0/d7/l4d 0 2024-09-16T08:57:18.026 INFO:tasks.workunit.client.1.smithi097.stdout:4/730: truncate d0/d7/d52/f8d 4211462 0 2024-09-16T08:57:18.026 INFO:tasks.workunit.client.1.smithi097.stdout:4/731: fdatasync d0/d7/d52/d26/d8c/fb2 0 2024-09-16T08:57:18.026 INFO:tasks.workunit.client.1.smithi097.stdout:4/732: write d0/d36/d4c/fac [1916134,51962] 0 2024-09-16T08:57:18.026 INFO:tasks.workunit.client.1.smithi097.stdout:4/733: write d0/d7/d52/d26/d8c/db0/fca [1020000,107837] 0 2024-09-16T08:57:18.031 INFO:tasks.workunit.client.1.smithi097.stdout:5/863: mkdir db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d11c 0 2024-09-16T08:57:18.032 INFO:tasks.workunit.client.1.smithi097.stdout:8/754: dwrite d8/dd/d15/d2c/d4a/d4e/f70 [0,4194304] 0 2024-09-16T08:57:18.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/795: dwrite d2/d5/d72/fab [4194304,4194304] 0 2024-09-16T08:57:18.032 INFO:tasks.workunit.client.1.smithi097.stdout:9/796: dread - d2/d5/db0/dc8/ff1 zero size 2024-09-16T08:57:18.032 INFO:tasks.workunit.client.0.smithi031.stdout:6/488: dwrite d0/f91 [0,4194304] 0 2024-09-16T08:57:18.032 INFO:tasks.workunit.client.0.smithi031.stdout:6/489: getdents d0/dd/d74/d19/d9a 0 2024-09-16T08:57:18.035 INFO:tasks.workunit.client.1.smithi097.stdout:5/864: creat db/d18/d1c/d42/d46/d7e/d64/d8d/f11d x:0 0 0 2024-09-16T08:57:18.035 INFO:tasks.workunit.client.1.smithi097.stdout:5/865: chown db/f40 159 1 2024-09-16T08:57:18.035 INFO:tasks.workunit.client.1.smithi097.stdout:5/866: write db/d18/d1c/d42/d46/d7e/f97 [434286,38515] 0 2024-09-16T08:57:18.038 INFO:tasks.workunit.client.1.smithi097.stdout:4/734: write d0/d7/df/f13 [1096969,84748] 0 2024-09-16T08:57:18.042 INFO:tasks.workunit.client.1.smithi097.stdout:8/755: truncate d8/dd/f7e 991693 0 2024-09-16T08:57:18.044 INFO:tasks.workunit.client.1.smithi097.stdout:2/839: dwrite d3/d4/d68/d8b/d7c/d7d/f6d [4194304,4194304] 0 2024-09-16T08:57:18.048 INFO:tasks.workunit.client.1.smithi097.stdout:0/802: dwrite da/d24/d4c/d80/d42/fef [0,4194304] 0 2024-09-16T08:57:18.048 INFO:tasks.workunit.client.1.smithi097.stdout:0/803: chown da/df/d13/c7d 156637315 1 2024-09-16T08:57:18.048 INFO:tasks.workunit.client.0.smithi031.stdout:6/490: dwrite d0/dd/d74/d5e/d7c/d56/f78 [0,4194304] 0 2024-09-16T08:57:18.053 INFO:tasks.workunit.client.0.smithi031.stdout:6/491: truncate d0/dd/d74/d5e/d7c/d3f/d65/f68 293477 0 2024-09-16T08:57:18.053 INFO:tasks.workunit.client.1.smithi097.stdout:5/867: truncate db/d22/d58/f10b 3367982 0 2024-09-16T08:57:18.054 INFO:tasks.workunit.client.1.smithi097.stdout:5/868: creat db/d6e/d95/f11e x:0 0 0 2024-09-16T08:57:18.054 INFO:tasks.workunit.client.1.smithi097.stdout:5/869: chown db/d18/d1c/d42/d46/d7e/fb5 41249962 1 2024-09-16T08:57:18.054 INFO:tasks.workunit.client.1.smithi097.stdout:4/735: symlink d0/d23/d5f/l106 0 2024-09-16T08:57:18.057 INFO:tasks.workunit.client.1.smithi097.stdout:8/756: link d8/dd/lce d8/dd/d15/d2c/d4a/lec 0 2024-09-16T08:57:18.059 INFO:tasks.workunit.client.1.smithi097.stdout:2/840: mknod d3/d1b/da7/c105 0 2024-09-16T08:57:18.060 INFO:tasks.workunit.client.0.smithi031.stdout:6/492: write d0/d10/f60 [427532,105190] 0 2024-09-16T08:57:18.061 INFO:tasks.workunit.client.1.smithi097.stdout:7/733: rmdir d4/d9/d51 39 2024-09-16T08:57:18.065 INFO:tasks.workunit.client.1.smithi097.stdout:0/804: stat da/d7e/dc4/dfc/d94/d101/c70 0 2024-09-16T08:57:18.067 INFO:tasks.workunit.client.1.smithi097.stdout:2/841: write d3/d4/d8/d51/f59 [3790761,73462] 0 2024-09-16T08:57:18.067 INFO:tasks.workunit.client.1.smithi097.stdout:2/842: creat d3/d4/d68/d8b/d61/dac/f106 x:0 0 0 2024-09-16T08:57:18.069 INFO:tasks.workunit.client.1.smithi097.stdout:5/870: creat db/d18/d1c/d42/d46/d7e/d76/dca/f11f x:0 0 0 2024-09-16T08:57:18.069 INFO:tasks.workunit.client.1.smithi097.stdout:0/805: dread da/d7e/f96 [0,4194304] 0 2024-09-16T08:57:18.069 INFO:tasks.workunit.client.1.smithi097.stdout:0/806: creat da/d1a/d41/db2/f11d x:0 0 0 2024-09-16T08:57:18.070 INFO:tasks.workunit.client.1.smithi097.stdout:0/807: fsync da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dad/fec 0 2024-09-16T08:57:18.070 INFO:tasks.workunit.client.1.smithi097.stdout:0/808: fsync da/df/f3f 0 2024-09-16T08:57:18.075 INFO:tasks.workunit.client.1.smithi097.stdout:5/871: mknod db/d18/d1c/d42/c120 0 2024-09-16T08:57:18.075 INFO:tasks.workunit.client.1.smithi097.stdout:5/872: creat db/d6e/d79/dba/f121 x:0 0 0 2024-09-16T08:57:18.076 INFO:tasks.workunit.client.1.smithi097.stdout:4/736: getdents d0/d23/ddc/de5 0 2024-09-16T08:57:18.077 INFO:tasks.workunit.client.1.smithi097.stdout:8/757: getdents d8/dd/d71 0 2024-09-16T08:57:18.080 INFO:tasks.workunit.client.1.smithi097.stdout:5/873: mknod db/d18/d1c/d42/c122 0 2024-09-16T08:57:18.082 INFO:tasks.workunit.client.1.smithi097.stdout:4/737: rename d0/d7/df/d83/df8/f85 to d0/d35/f107 0 2024-09-16T08:57:18.083 INFO:tasks.workunit.client.0.smithi031.stdout:1/336: truncate d2/d15/d1f/d43/f4c 3207184 0 2024-09-16T08:57:18.084 INFO:tasks.workunit.client.1.smithi097.stdout:5/874: mknod db/d1a/d6d/dc6/c123 0 2024-09-16T08:57:18.086 INFO:tasks.workunit.client.1.smithi097.stdout:8/758: rename d8/dd/d15/d2c/d4a/d4d/f35 to d8/dd/d15/d2c/d4a/d4d/d3d/db9/fed 0 2024-09-16T08:57:18.086 INFO:tasks.workunit.client.1.smithi097.stdout:7/734: dwrite d4/d12/d1f/fb6 [0,4194304] 0 2024-09-16T08:57:18.087 INFO:tasks.workunit.client.1.smithi097.stdout:7/735: read - d4/d9/d1c/fbf zero size 2024-09-16T08:57:18.087 INFO:tasks.workunit.client.1.smithi097.stdout:2/843: dwrite d3/d4/d9/fa1 [4194304,4194304] 0 2024-09-16T08:57:18.088 INFO:tasks.workunit.client.1.smithi097.stdout:4/738: fdatasync d0/d7/df/f13 0 2024-09-16T08:57:18.089 INFO:tasks.workunit.client.1.smithi097.stdout:5/875: dread db/d18/d1c/d42/d46/d7e/d64/d7f/d89/f98 [0,4194304] 0 2024-09-16T08:57:18.092 INFO:tasks.workunit.client.0.smithi031.stdout:1/337: dread d2/d4/d1b/d5c/f66 [0,4194304] 0 2024-09-16T08:57:18.092 INFO:tasks.workunit.client.0.smithi031.stdout:1/338: write d2/d4/f54 [472606,55037] 0 2024-09-16T08:57:18.109 INFO:tasks.workunit.client.1.smithi097.stdout:4/739: dwrite d0/d7/df/f37 [4194304,4194304] 0 2024-09-16T08:57:18.233 INFO:tasks.workunit.client.1.smithi097.stdout:7/736: link d4/d1b/f69 d4/d7/d7b/ff5 0 2024-09-16T08:57:18.233 INFO:tasks.workunit.client.1.smithi097.stdout:7/737: creat d4/ff6 x:0 0 0 2024-09-16T08:57:18.234 INFO:tasks.workunit.client.0.smithi031.stdout:1/339: symlink d2/d15/d1f/l7c 0 2024-09-16T08:57:18.237 INFO:tasks.workunit.client.1.smithi097.stdout:2/844: truncate d3/d4/d68/d8b/d7c/d7d/f6b 1456826 0 2024-09-16T08:57:18.237 INFO:tasks.workunit.client.1.smithi097.stdout:2/845: fsync d3/d4/d9/fb4 0 2024-09-16T08:57:18.239 INFO:tasks.workunit.client.1.smithi097.stdout:8/759: rmdir d8/dd/d15/d2c/d4a/d4e/db2 39 2024-09-16T08:57:18.240 INFO:tasks.workunit.client.1.smithi097.stdout:2/846: dread d3/d4/d68/f72 [0,4194304] 0 2024-09-16T08:57:18.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/340: symlink d2/d15/l7d 0 2024-09-16T08:57:18.246 INFO:tasks.workunit.client.1.smithi097.stdout:9/797: write d2/d5/f69 [888356,54384] 0 2024-09-16T08:57:18.247 INFO:tasks.workunit.client.0.smithi031.stdout:1/341: creat d2/d4/d36/f7e x:0 0 0 2024-09-16T08:57:18.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/342: rmdir d2/d15/d32 39 2024-09-16T08:57:18.253 INFO:tasks.workunit.client.0.smithi031.stdout:1/343: rename d2/d15/d1e/d75 to d2/d4/d36/d7f 0 2024-09-16T08:57:18.257 INFO:tasks.workunit.client.1.smithi097.stdout:7/738: rename d4/d7/d30/f68 to d4/d9/dc/ff7 0 2024-09-16T08:57:18.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:17 smithi097 ceph-mon[35478]: pgmap v10: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 201 MiB/s rd, 265 MiB/s wr, 444 op/s 2024-09-16T08:57:18.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:17 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1694206469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:57:18.263 INFO:tasks.workunit.client.1.smithi097.stdout:2/847: dwrite d3/d4/d9/d78/f9e [0,4194304] 0 2024-09-16T08:57:18.263 INFO:tasks.workunit.client.1.smithi097.stdout:8/760: dwrite d8/dd/d15/d2c/d4a/d4d/f49 [0,4194304] 0 2024-09-16T08:57:18.264 INFO:tasks.workunit.client.1.smithi097.stdout:9/798: rmdir d2/d5/db0/dc8 39 2024-09-16T08:57:18.267 INFO:tasks.workunit.client.1.smithi097.stdout:2/848: mknod d3/d4/d68/d8b/c107 0 2024-09-16T08:57:18.267 INFO:tasks.workunit.client.1.smithi097.stdout:2/849: truncate d3/d4/d68/d8b/fda 887207 0 2024-09-16T08:57:18.269 INFO:tasks.workunit.client.1.smithi097.stdout:9/799: rename d2/l22 to d2/d5/d72/dc1/lfc 0 2024-09-16T08:57:18.271 INFO:tasks.workunit.client.1.smithi097.stdout:2/850: rmdir d3/d4/d8/d103 0 2024-09-16T08:57:18.272 INFO:tasks.workunit.client.1.smithi097.stdout:2/851: getdents d3/d4/d77 0 2024-09-16T08:57:18.276 INFO:tasks.workunit.client.1.smithi097.stdout:7/739: dwrite d4/f5 [0,4194304] 0 2024-09-16T08:57:18.278 INFO:tasks.workunit.client.0.smithi031.stdout:7/435: sync 2024-09-16T08:57:18.278 INFO:tasks.workunit.client.0.smithi031.stdout:0/385: sync 2024-09-16T08:57:18.278 INFO:tasks.workunit.client.0.smithi031.stdout:8/293: sync 2024-09-16T08:57:18.280 INFO:tasks.workunit.client.1.smithi097.stdout:9/800: read d2/d5/fb [7759741,32315] 0 2024-09-16T08:57:18.283 INFO:tasks.workunit.client.1.smithi097.stdout:9/801: mkdir d2/d5/db0/dc8/d74/d8c/d94/d96/dac/dfd 0 2024-09-16T08:57:18.285 INFO:tasks.workunit.client.1.smithi097.stdout:9/802: creat d2/d5/db0/dc8/d74/d8c/dad/ffe x:0 0 0 2024-09-16T08:57:18.285 INFO:tasks.workunit.client.1.smithi097.stdout:9/803: chown d2/d5/db0/dc8/f5a 2356 1 2024-09-16T08:57:18.285 INFO:tasks.workunit.client.1.smithi097.stdout:9/804: write d2/d5/d72/dc1/de6/fea [524967,118482] 0 2024-09-16T08:57:18.285 INFO:tasks.workunit.client.1.smithi097.stdout:9/805: chown d2/d5/d72/fab 34398 1 2024-09-16T08:57:18.286 INFO:tasks.workunit.client.1.smithi097.stdout:4/740: sync 2024-09-16T08:57:18.286 INFO:tasks.workunit.client.1.smithi097.stdout:5/876: sync 2024-09-16T08:57:18.286 INFO:tasks.workunit.client.1.smithi097.stdout:0/809: sync 2024-09-16T08:57:18.287 INFO:tasks.workunit.client.1.smithi097.stdout:3/817: sync 2024-09-16T08:57:18.287 INFO:tasks.workunit.client.1.smithi097.stdout:0/810: write da/df/d13/fa7 [721259,22172] 0 2024-09-16T08:57:18.287 INFO:tasks.workunit.client.1.smithi097.stdout:3/818: dread - d7/d28/d30/d88/df7/f10e zero size 2024-09-16T08:57:18.287 INFO:tasks.workunit.client.1.smithi097.stdout:6/842: sync 2024-09-16T08:57:18.287 INFO:tasks.workunit.client.1.smithi097.stdout:1/916: sync 2024-09-16T08:57:18.287 INFO:tasks.workunit.client.1.smithi097.stdout:1/917: fsync d5/d2c/d3b/d7f/fdb 0 2024-09-16T08:57:18.288 INFO:tasks.workunit.client.1.smithi097.stdout:9/806: creat d2/d5/d8d/db1/fff x:0 0 0 2024-09-16T08:57:18.288 INFO:tasks.workunit.client.1.smithi097.stdout:9/807: chown d2/d5/db0/dc8/lbc 13736801 1 2024-09-16T08:57:18.288 INFO:tasks.workunit.client.1.smithi097.stdout:9/808: fsync d2/d5/d3a/ff6 0 2024-09-16T08:57:18.288 INFO:tasks.workunit.client.1.smithi097.stdout:8/761: dwrite d8/dd/d71/f7d [0,4194304] 0 2024-09-16T08:57:18.289 INFO:tasks.workunit.client.1.smithi097.stdout:4/741: mkdir d0/d7/df/d83/df8/dbc/d108 0 2024-09-16T08:57:18.291 INFO:tasks.workunit.client.0.smithi031.stdout:1/344: rmdir d2/d4/d36/d7f 0 2024-09-16T08:57:18.292 INFO:tasks.workunit.client.1.smithi097.stdout:5/877: mkdir db/d18/d1c/d42/d124 0 2024-09-16T08:57:18.293 INFO:tasks.workunit.client.0.smithi031.stdout:0/386: write d6/d12/d13/f3e [792745,79575] 0 2024-09-16T08:57:18.295 INFO:tasks.workunit.client.0.smithi031.stdout:1/345: symlink d2/d4/l80 0 2024-09-16T08:57:18.297 INFO:tasks.workunit.client.1.smithi097.stdout:3/819: rename d7/de/d3c/d3f/d60/d6b/db8/cc7 to d7/de/d3c/d3f/d60/d6b/c118 0 2024-09-16T08:57:18.298 INFO:tasks.workunit.client.1.smithi097.stdout:3/820: read d7/de/d3c/d3f/d60/dcb/fa7 [105196,67991] 0 2024-09-16T08:57:18.298 INFO:tasks.workunit.client.1.smithi097.stdout:3/821: write d7/de/d9c/ff2 [714146,41230] 0 2024-09-16T08:57:18.303 INFO:tasks.workunit.client.1.smithi097.stdout:1/918: mknod d5/c128 0 2024-09-16T08:57:18.304 INFO:tasks.workunit.client.1.smithi097.stdout:9/809: truncate d2/d5/fb 5985937 0 2024-09-16T08:57:18.305 INFO:tasks.workunit.client.0.smithi031.stdout:8/294: dread d4/d7/d36/d38/f3c [0,4194304] 0 2024-09-16T08:57:18.307 INFO:tasks.workunit.client.1.smithi097.stdout:4/742: symlink d0/d7/df/d15/d1f/l109 0 2024-09-16T08:57:18.307 INFO:tasks.workunit.client.1.smithi097.stdout:4/743: dread - d0/d7/df/d83/ff5 zero size 2024-09-16T08:57:18.308 INFO:tasks.workunit.client.0.smithi031.stdout:8/295: symlink d4/d7/d36/d5e/l63 0 2024-09-16T08:57:18.309 INFO:tasks.workunit.client.1.smithi097.stdout:5/878: symlink db/d18/d1c/d42/d46/d7e/d76/dca/l125 0 2024-09-16T08:57:18.310 INFO:tasks.workunit.client.1.smithi097.stdout:5/879: fdatasync db/d18/d1c/d42/d46/d7e/d8f/fd8 0 2024-09-16T08:57:18.310 INFO:tasks.workunit.client.1.smithi097.stdout:5/880: write db/d22/f4b [699173,111594] 0 2024-09-16T08:57:18.310 INFO:tasks.workunit.client.1.smithi097.stdout:0/811: rmdir da/d7e/dc4/dfc/d94/d101 39 2024-09-16T08:57:18.310 INFO:tasks.workunit.client.0.smithi031.stdout:8/296: symlink d4/d7/d21/l64 0 2024-09-16T08:57:18.315 INFO:tasks.workunit.client.0.smithi031.stdout:8/297: symlink d4/d7/d21/d35/l65 0 2024-09-16T08:57:18.315 INFO:tasks.workunit.client.0.smithi031.stdout:8/298: write d4/f6 [3852455,37218] 0 2024-09-16T08:57:18.315 INFO:tasks.workunit.client.0.smithi031.stdout:7/436: dread d2/d8/f50 [0,4194304] 0 2024-09-16T08:57:18.315 INFO:tasks.workunit.client.0.smithi031.stdout:7/437: readlink d2/d8/d11/d5d/l8e 0 2024-09-16T08:57:18.315 INFO:tasks.workunit.client.0.smithi031.stdout:7/438: readlink d2/d2d/l38 0 2024-09-16T08:57:18.315 INFO:tasks.workunit.client.0.smithi031.stdout:7/439: dread - d2/f62 zero size 2024-09-16T08:57:18.315 INFO:tasks.workunit.client.0.smithi031.stdout:7/440: truncate d2/f5b 845981 0 2024-09-16T08:57:18.316 INFO:tasks.workunit.client.0.smithi031.stdout:7/441: stat d2/d8/d11/c4c 0 2024-09-16T08:57:18.316 INFO:tasks.workunit.client.1.smithi097.stdout:1/919: rename d5/d17/d54/d6b/d71/laa to d5/d17/d54/l129 0 2024-09-16T08:57:18.316 INFO:tasks.workunit.client.1.smithi097.stdout:9/810: mkdir d2/d5/db0/d100 0 2024-09-16T08:57:18.317 INFO:tasks.workunit.client.0.smithi031.stdout:7/442: dread d2/d2d/f6f [0,4194304] 0 2024-09-16T08:57:18.317 INFO:tasks.workunit.client.1.smithi097.stdout:4/744: unlink d0/f4 0 2024-09-16T08:57:18.317 INFO:tasks.workunit.client.1.smithi097.stdout:4/745: chown d0/d35/da4/dd1/fe1 0 1 2024-09-16T08:57:18.318 INFO:tasks.workunit.client.1.smithi097.stdout:0/812: rmdir da/d24/d4b/d112 0 2024-09-16T08:57:18.318 INFO:tasks.workunit.client.1.smithi097.stdout:0/813: truncate da/df/d8a/fa9 409914 0 2024-09-16T08:57:18.319 INFO:tasks.workunit.client.1.smithi097.stdout:3/822: write d7/d15/f17 [3852025,120213] 0 2024-09-16T08:57:18.320 INFO:tasks.workunit.client.0.smithi031.stdout:7/443: getdents d2/d2d/d6d 0 2024-09-16T08:57:18.321 INFO:tasks.workunit.client.1.smithi097.stdout:1/920: symlink d5/d70/l12a 0 2024-09-16T08:57:18.321 INFO:tasks.workunit.client.1.smithi097.stdout:0/814: rmdir da/d24/d4c/d6d 39 2024-09-16T08:57:18.322 INFO:tasks.workunit.client.0.smithi031.stdout:7/444: mkdir d2/d8/d11/d66/d9a 0 2024-09-16T08:57:18.322 INFO:tasks.workunit.client.1.smithi097.stdout:2/852: dwrite d3/d4/d68/d8b/fda [0,4194304] 0 2024-09-16T08:57:18.322 INFO:tasks.workunit.client.1.smithi097.stdout:2/853: chown d3/d4/d8/d51/fc7 5991919 1 2024-09-16T08:57:18.323 INFO:tasks.workunit.client.1.smithi097.stdout:3/823: mknod d7/d28/d30/d69/c119 0 2024-09-16T08:57:18.323 INFO:tasks.workunit.client.0.smithi031.stdout:7/445: read d2/f5b [296926,94650] 0 2024-09-16T08:57:18.327 INFO:tasks.workunit.client.0.smithi031.stdout:7/446: getdents d2/d8/d86 0 2024-09-16T08:57:18.327 INFO:tasks.workunit.client.0.smithi031.stdout:7/447: write d2/d2d/f58 [327670,98989] 0 2024-09-16T08:57:18.327 INFO:tasks.workunit.client.1.smithi097.stdout:9/811: write d2/d5/d72/fab [5523843,54864] 0 2024-09-16T08:57:18.329 INFO:tasks.workunit.client.1.smithi097.stdout:4/746: dread d0/d7/d52/dbf/f100 [0,4194304] 0 2024-09-16T08:57:18.329 INFO:tasks.workunit.client.1.smithi097.stdout:4/747: read d0/d39/fa3 [248072,58027] 0 2024-09-16T08:57:18.329 INFO:tasks.workunit.client.1.smithi097.stdout:4/748: chown d0/d7/df/f27 19769 1 2024-09-16T08:57:18.329 INFO:tasks.workunit.client.1.smithi097.stdout:4/749: read - d0/d35/da4/dc5/de7/fec zero size 2024-09-16T08:57:18.330 INFO:tasks.workunit.client.0.smithi031.stdout:7/448: rename d2/d8/d69/d90/f7f to d2/d8/d69/d82/f9b 0 2024-09-16T08:57:18.332 INFO:tasks.workunit.client.0.smithi031.stdout:7/449: write d2/d2d/f89 [209029,6955] 0 2024-09-16T08:57:18.332 INFO:tasks.workunit.client.0.smithi031.stdout:7/450: read - d2/d8/d69/d82/f96 zero size 2024-09-16T08:57:18.333 INFO:tasks.workunit.client.1.smithi097.stdout:7/740: dwrite d4/d7/fbd [0,4194304] 0 2024-09-16T08:57:18.333 INFO:tasks.workunit.client.1.smithi097.stdout:8/762: dwrite d8/dd/d15/d2c/d4a/d4d/d3d/db9/fd4 [0,4194304] 0 2024-09-16T08:57:18.334 INFO:tasks.workunit.client.0.smithi031.stdout:1/346: dwrite d2/d4/d36/f7e [0,4194304] 0 2024-09-16T08:57:18.334 INFO:tasks.workunit.client.0.smithi031.stdout:1/347: fsync d2/d4/d36/f7e 0 2024-09-16T08:57:18.334 INFO:tasks.workunit.client.0.smithi031.stdout:1/348: stat d2/f53 0 2024-09-16T08:57:18.334 INFO:tasks.workunit.client.1.smithi097.stdout:5/881: dwrite db/d1a/fe2 [0,4194304] 0 2024-09-16T08:57:18.334 INFO:tasks.workunit.client.1.smithi097.stdout:5/882: read - db/d18/d1c/d42/d46/d7e/d64/fbd zero size 2024-09-16T08:57:18.337 INFO:tasks.workunit.client.0.smithi031.stdout:0/387: dwrite d6/f7b [0,4194304] 0 2024-09-16T08:57:18.338 INFO:tasks.workunit.client.1.smithi097.stdout:2/854: creat d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4/f108 x:0 0 0 2024-09-16T08:57:18.339 INFO:tasks.workunit.client.0.smithi031.stdout:7/451: unlink d2/d8/d11/d5d/d6e/f99 0 2024-09-16T08:57:18.340 INFO:tasks.workunit.client.1.smithi097.stdout:3/824: creat d7/d15/d7d/f11a x:0 0 0 2024-09-16T08:57:18.341 INFO:tasks.workunit.client.1.smithi097.stdout:1/921: write d5/db/dc/d19/f8c [3689797,59870] 0 2024-09-16T08:57:18.342 INFO:tasks.workunit.client.1.smithi097.stdout:4/750: write d0/d7/f75 [151135,80450] 0 2024-09-16T08:57:18.343 INFO:tasks.workunit.client.1.smithi097.stdout:0/815: truncate da/d24/d4c/d80/d3b/f91 1195535 0 2024-09-16T08:57:18.344 INFO:tasks.workunit.client.1.smithi097.stdout:0/816: stat da/df/d8a 0 2024-09-16T08:57:18.344 INFO:tasks.workunit.client.1.smithi097.stdout:0/817: stat da/d7e/dc4/dfc/d94/d101/f44 0 2024-09-16T08:57:18.348 INFO:tasks.workunit.client.0.smithi031.stdout:1/349: truncate d2/f29 2277637 0 2024-09-16T08:57:18.350 INFO:tasks.workunit.client.0.smithi031.stdout:8/299: getdents d4/d7/d36/d5e 0 2024-09-16T08:57:18.350 INFO:tasks.workunit.client.0.smithi031.stdout:8/300: fsync d4/f29 0 2024-09-16T08:57:18.350 INFO:tasks.workunit.client.1.smithi097.stdout:2/855: write d3/d4/d68/d8b/d61/dac/fd8 [2658778,30116] 0 2024-09-16T08:57:18.351 INFO:tasks.workunit.client.1.smithi097.stdout:3/825: dread d7/de/d3c/dde/ff0 [0,4194304] 0 2024-09-16T08:57:18.353 INFO:tasks.workunit.client.1.smithi097.stdout:7/741: rmdir d4/d9 39 2024-09-16T08:57:18.354 INFO:tasks.workunit.client.1.smithi097.stdout:9/812: dwrite d2/d5/db0/dc8/d74/d8c/d94/fa9 [0,4194304] 0 2024-09-16T08:57:18.354 INFO:tasks.workunit.client.1.smithi097.stdout:9/813: truncate d2/d5/f21 4989765 0 2024-09-16T08:57:18.354 INFO:tasks.workunit.client.1.smithi097.stdout:8/763: dwrite d8/f24 [4194304,4194304] 0 2024-09-16T08:57:18.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/388: unlink d6/fc 0 2024-09-16T08:57:18.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/389: fsync d6/d12/f16 0 2024-09-16T08:57:18.358 INFO:tasks.workunit.client.1.smithi097.stdout:1/922: getdents d5/d17/d54/d6b/da8/dc5/dc7/df3 0 2024-09-16T08:57:18.364 INFO:tasks.workunit.client.0.smithi031.stdout:8/301: rename d4/d7/d21/d35/d40/d4f to d4/d7/d66 0 2024-09-16T08:57:18.368 INFO:tasks.workunit.client.1.smithi097.stdout:0/818: truncate da/d7e/dc4/dfc/d94/d101/f44 3142382 0 2024-09-16T08:57:18.368 INFO:tasks.workunit.client.1.smithi097.stdout:0/819: chown da/df/d13/c17 2 1 2024-09-16T08:57:18.369 INFO:tasks.workunit.client.0.smithi031.stdout:7/452: fsync d2/d8/d69/d82/f9b 0 2024-09-16T08:57:18.370 INFO:tasks.workunit.client.1.smithi097.stdout:1/923: dread d5/f12 [0,4194304] 0 2024-09-16T08:57:18.370 INFO:tasks.workunit.client.1.smithi097.stdout:3/826: rename d7/de/d3c/dcc/ff3 to d7/d15/dee/f11b 0 2024-09-16T08:57:18.371 INFO:tasks.workunit.client.0.smithi031.stdout:0/390: getdents d6/d61 0 2024-09-16T08:57:18.372 INFO:tasks.workunit.client.0.smithi031.stdout:0/391: chown d6/d61 43 1 2024-09-16T08:57:18.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:17 smithi031 ceph-mon[27844]: pgmap v10: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 201 MiB/s rd, 265 MiB/s wr, 444 op/s 2024-09-16T08:57:18.372 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:17 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1694206469' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:57:18.373 INFO:tasks.workunit.client.1.smithi097.stdout:7/742: unlink d4/d7/d77/lf3 0 2024-09-16T08:57:18.373 INFO:tasks.workunit.client.0.smithi031.stdout:7/453: mkdir d2/d8/d11/d66/d9a/d9c 0 2024-09-16T08:57:18.374 INFO:tasks.workunit.client.1.smithi097.stdout:4/751: dwrite d0/d36/f51 [0,4194304] 0 2024-09-16T08:57:18.374 INFO:tasks.workunit.client.0.smithi031.stdout:8/302: getdents d4/d5b 0 2024-09-16T08:57:18.375 INFO:tasks.workunit.client.1.smithi097.stdout:5/883: dwrite db/d6e/d95/fa3 [0,4194304] 0 2024-09-16T08:57:18.375 INFO:tasks.workunit.client.1.smithi097.stdout:5/884: chown db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/fb7 11747586 1 2024-09-16T08:57:18.375 INFO:tasks.workunit.client.1.smithi097.stdout:2/856: dwrite d3/d4/d68/d8b/d7c/d7d/f6b [0,4194304] 0 2024-09-16T08:57:18.376 INFO:tasks.workunit.client.1.smithi097.stdout:9/814: creat d2/f101 x:0 0 0 2024-09-16T08:57:18.379 INFO:tasks.workunit.client.1.smithi097.stdout:8/764: symlink d8/dd/d15/d2c/d4a/d4d/d3f/lee 0 2024-09-16T08:57:18.381 INFO:tasks.workunit.client.0.smithi031.stdout:0/392: creat d6/d12/d47/f7d x:0 0 0 2024-09-16T08:57:18.381 INFO:tasks.workunit.client.0.smithi031.stdout:0/393: creat d6/d12/d47/f7e x:0 0 0 2024-09-16T08:57:18.381 INFO:tasks.workunit.client.0.smithi031.stdout:0/394: chown d6/d12/d47/f7d 6142 1 2024-09-16T08:57:18.386 INFO:tasks.workunit.client.1.smithi097.stdout:9/815: write d2/d5/db0/dc8/d74/d8c/dad/fb2 [516124,18751] 0 2024-09-16T08:57:18.386 INFO:tasks.workunit.client.1.smithi097.stdout:7/743: write d4/d9/fda [481623,90927] 0 2024-09-16T08:57:18.388 INFO:tasks.workunit.client.1.smithi097.stdout:0/820: link da/d7e/dc4/dfc/d47/db3/fd4 da/d7e/f11e 0 2024-09-16T08:57:18.388 INFO:tasks.workunit.client.1.smithi097.stdout:0/821: truncate da/d1a/d41/db2/fd6 4531245 0 2024-09-16T08:57:18.393 INFO:tasks.workunit.client.0.smithi031.stdout:0/395: dread d6/d12/d13/d3c/f70 [0,4194304] 0 2024-09-16T08:57:18.393 INFO:tasks.workunit.client.0.smithi031.stdout:7/454: getdents d2/d8/d11 0 2024-09-16T08:57:18.394 INFO:tasks.workunit.client.0.smithi031.stdout:8/303: dwrite d4/d7/d21/d35/d40/f58 [0,4194304] 0 2024-09-16T08:57:18.396 INFO:tasks.workunit.client.1.smithi097.stdout:3/827: unlink d7/de/d8d/d90/ff9 0 2024-09-16T08:57:18.396 INFO:tasks.workunit.client.1.smithi097.stdout:3/828: fsync d7/de/d3c/d3f/f50 0 2024-09-16T08:57:18.403 INFO:tasks.workunit.client.0.smithi031.stdout:7/455: mknod d2/d8/d11/c9d 0 2024-09-16T08:57:18.403 INFO:tasks.workunit.client.0.smithi031.stdout:7/456: write d2/d8/f63 [802275,122862] 0 2024-09-16T08:57:18.404 INFO:tasks.workunit.client.1.smithi097.stdout:4/752: symlink d0/d35/da4/dc5/de7/l10a 0 2024-09-16T08:57:18.407 INFO:tasks.workunit.client.1.smithi097.stdout:5/885: creat db/d18/d1c/d42/d46/d7e/d8f/d102/f126 x:0 0 0 2024-09-16T08:57:18.408 INFO:tasks.workunit.client.0.smithi031.stdout:0/396: rename d6/d12/d13/d36/f43 to d6/d12/d13/f7f 0 2024-09-16T08:57:18.410 INFO:tasks.workunit.client.0.smithi031.stdout:8/304: dwrite d4/d7/d36/d38/f3c [0,4194304] 0 2024-09-16T08:57:18.410 INFO:tasks.workunit.client.0.smithi031.stdout:8/305: chown c1 14 1 2024-09-16T08:57:18.410 INFO:tasks.workunit.client.0.smithi031.stdout:8/306: chown d4/d5b 19857 1 2024-09-16T08:57:18.412 INFO:tasks.workunit.client.1.smithi097.stdout:2/857: getdents d3/d4/d68/d8b/d7c/d7d 0 2024-09-16T08:57:18.412 INFO:tasks.workunit.client.1.smithi097.stdout:2/858: creat d3/d4/d9/d78/dcc/f109 x:0 0 0 2024-09-16T08:57:18.413 INFO:tasks.workunit.client.0.smithi031.stdout:7/457: dread d2/d8/d11/d5d/d6e/f95 [0,4194304] 0 2024-09-16T08:57:18.413 INFO:tasks.workunit.client.0.smithi031.stdout:7/458: write d2/f5b [1251735,1993] 0 2024-09-16T08:57:18.413 INFO:tasks.workunit.client.0.smithi031.stdout:7/459: write d2/f1a [4347958,31285] 0 2024-09-16T08:57:18.414 INFO:tasks.workunit.client.0.smithi031.stdout:8/307: dread d4/d7/f28 [0,4194304] 0 2024-09-16T08:57:18.415 INFO:tasks.workunit.client.1.smithi097.stdout:8/765: rename d8/dd/d15/l79 to d8/dd/d15/d2c/d4a/d4e/d97/lef 0 2024-09-16T08:57:18.415 INFO:tasks.workunit.client.1.smithi097.stdout:8/766: chown d8/dd/cd7 30887134 1 2024-09-16T08:57:18.415 INFO:tasks.workunit.client.1.smithi097.stdout:8/767: creat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/ff0 x:0 0 0 2024-09-16T08:57:18.422 INFO:tasks.workunit.client.1.smithi097.stdout:9/816: unlink d2/c1f 0 2024-09-16T08:57:18.423 INFO:tasks.workunit.client.1.smithi097.stdout:7/744: symlink d4/d7/d30/lf8 0 2024-09-16T08:57:18.424 INFO:tasks.workunit.client.1.smithi097.stdout:7/745: write d4/d9/d1c/f34 [1481167,88653] 0 2024-09-16T08:57:18.425 INFO:tasks.workunit.client.1.smithi097.stdout:0/822: creat da/f11f x:0 0 0 2024-09-16T08:57:18.426 INFO:tasks.workunit.client.0.smithi031.stdout:0/397: rmdir d6/d12/d13 39 2024-09-16T08:57:18.427 INFO:tasks.workunit.client.0.smithi031.stdout:7/460: rename d2/d8/d11/f31 to d2/d8/d86/f9e 0 2024-09-16T08:57:18.428 INFO:tasks.workunit.client.0.smithi031.stdout:8/308: dwrite d4/d7/d21/f3b [0,4194304] 0 2024-09-16T08:57:18.434 INFO:tasks.workunit.client.0.smithi031.stdout:8/309: truncate d4/d7/d21/d35/d40/f4c 3172707 0 2024-09-16T08:57:18.434 INFO:tasks.workunit.client.0.smithi031.stdout:8/310: creat d4/d7/f67 x:0 0 0 2024-09-16T08:57:18.436 INFO:tasks.workunit.client.0.smithi031.stdout:0/398: rename d6/ce to d6/d12/d13/c80 0 2024-09-16T08:57:18.438 INFO:tasks.workunit.client.0.smithi031.stdout:7/461: creat d2/d8/d11/d81/f9f x:0 0 0 2024-09-16T08:57:18.442 INFO:tasks.workunit.client.0.smithi031.stdout:8/311: mknod d4/d7/d21/d35/d50/c68 0 2024-09-16T08:57:18.442 INFO:tasks.workunit.client.0.smithi031.stdout:8/312: write d4/f29 [370930,6889] 0 2024-09-16T08:57:18.444 INFO:tasks.workunit.client.0.smithi031.stdout:0/399: mknod d6/d12/d47/c81 0 2024-09-16T08:57:18.444 INFO:tasks.workunit.client.0.smithi031.stdout:0/400: creat d6/f82 x:0 0 0 2024-09-16T08:57:18.445 INFO:tasks.workunit.client.0.smithi031.stdout:7/462: unlink d2/d8/d69/d82/f8f 0 2024-09-16T08:57:18.446 INFO:tasks.workunit.client.0.smithi031.stdout:8/313: symlink d4/d7/d21/d24/l69 0 2024-09-16T08:57:18.450 INFO:tasks.workunit.client.0.smithi031.stdout:7/463: rmdir d2/d8/d86 39 2024-09-16T08:57:18.450 INFO:tasks.workunit.client.0.smithi031.stdout:7/464: write d2/d2d/f43 [477676,39898] 0 2024-09-16T08:57:18.452 INFO:tasks.workunit.client.0.smithi031.stdout:8/314: rename d4/d7/d21/d37/d46/f56 to d4/d34/f6a 0 2024-09-16T08:57:18.453 INFO:tasks.workunit.client.0.smithi031.stdout:0/401: getdents d6/d12/d47/d4d 0 2024-09-16T08:57:18.458 INFO:tasks.workunit.client.1.smithi097.stdout:1/924: rmdir d5 39 2024-09-16T08:57:18.458 INFO:tasks.workunit.client.1.smithi097.stdout:3/829: truncate d7/de/d3c/d3f/f7b 2363943 0 2024-09-16T08:57:18.458 INFO:tasks.workunit.client.1.smithi097.stdout:3/830: stat d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa/fc4 0 2024-09-16T08:57:18.464 INFO:tasks.workunit.client.1.smithi097.stdout:4/753: creat d0/d23/f10b x:0 0 0 2024-09-16T08:57:18.465 INFO:tasks.workunit.client.1.smithi097.stdout:5/886: unlink db/d22/f6a 0 2024-09-16T08:57:18.467 INFO:tasks.workunit.client.1.smithi097.stdout:2/859: link d3/d4/d9/d78/dcc/ff6 d3/d4/d8/f10a 0 2024-09-16T08:57:18.471 INFO:tasks.workunit.client.1.smithi097.stdout:9/817: creat d2/d5/db0/dc8/d63/da8/dca/df5/f102 x:0 0 0 2024-09-16T08:57:18.471 INFO:tasks.workunit.client.1.smithi097.stdout:7/746: mkdir d4/d9/d51/db7/df9 0 2024-09-16T08:57:18.471 INFO:tasks.workunit.client.1.smithi097.stdout:0/823: link da/d24/d4b/ca5 da/d7e/dc4/dfc/c120 0 2024-09-16T08:57:18.473 INFO:tasks.workunit.client.1.smithi097.stdout:5/887: symlink db/d18/d20/l127 0 2024-09-16T08:57:18.474 INFO:tasks.workunit.client.1.smithi097.stdout:2/860: symlink d3/d4/d9/d67/l10b 0 2024-09-16T08:57:18.474 INFO:tasks.workunit.client.1.smithi097.stdout:8/768: rmdir d8/dd/d15/d16/d7c/da2 39 2024-09-16T08:57:18.477 INFO:tasks.workunit.client.1.smithi097.stdout:4/754: dread d0/d35/f5c [0,4194304] 0 2024-09-16T08:57:18.477 INFO:tasks.workunit.client.1.smithi097.stdout:7/747: mkdir d4/dfa 0 2024-09-16T08:57:18.477 INFO:tasks.workunit.client.1.smithi097.stdout:0/824: symlink da/df/d8a/dac/l121 0 2024-09-16T08:57:18.477 INFO:tasks.workunit.client.1.smithi097.stdout:0/825: readlink da/df/leb 0 2024-09-16T08:57:18.479 INFO:tasks.workunit.client.1.smithi097.stdout:7/748: mknod d4/d9/dc/dc6/cfb 0 2024-09-16T08:57:18.482 INFO:tasks.workunit.client.1.smithi097.stdout:3/831: rename d7/d28/d63/d70 to d7/de/d3c/d3f/d11c 0 2024-09-16T08:57:18.482 INFO:tasks.workunit.client.1.smithi097.stdout:3/832: fdatasync d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/f8b 0 2024-09-16T08:57:18.482 INFO:tasks.workunit.client.1.smithi097.stdout:3/833: truncate d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa/fcd 935903 0 2024-09-16T08:57:18.482 INFO:tasks.workunit.client.1.smithi097.stdout:3/834: chown d7/de/d9c/daf/f117 256790 1 2024-09-16T08:57:18.483 INFO:tasks.workunit.client.1.smithi097.stdout:3/835: getdents d7/de/d3c/d3f/d60/d6b/db8/deb 0 2024-09-16T08:57:18.483 INFO:tasks.workunit.client.0.smithi031.stdout:3/456: sync 2024-09-16T08:57:18.483 INFO:tasks.workunit.client.1.smithi097.stdout:9/818: dread d2/f28 [0,4194304] 0 2024-09-16T08:57:18.483 INFO:tasks.workunit.client.0.smithi031.stdout:3/457: readlink d3/l5e 0 2024-09-16T08:57:18.487 INFO:tasks.workunit.client.0.smithi031.stdout:3/458: mkdir d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87 0 2024-09-16T08:57:18.488 INFO:tasks.workunit.client.0.smithi031.stdout:7/465: dwrite d2/d2d/f77 [0,4194304] 0 2024-09-16T08:57:18.488 INFO:tasks.workunit.client.0.smithi031.stdout:8/315: dwrite d4/d7/d21/f3b [0,4194304] 0 2024-09-16T08:57:18.489 INFO:tasks.workunit.client.0.smithi031.stdout:3/459: truncate d3/d14/d37/d1a/d1d/f34 711223 0 2024-09-16T08:57:18.491 INFO:tasks.workunit.client.1.smithi097.stdout:7/749: read d4/d7/d7b/f8f [1157548,71414] 0 2024-09-16T08:57:18.494 INFO:tasks.workunit.client.0.smithi031.stdout:7/466: rmdir d2/d8/d86 39 2024-09-16T08:57:18.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/316: mkdir d4/d7/d66/d6b 0 2024-09-16T08:57:18.496 INFO:tasks.workunit.client.0.smithi031.stdout:8/317: truncate d4/d7/f1c 4313021 0 2024-09-16T08:57:18.497 INFO:tasks.workunit.client.1.smithi097.stdout:0/826: rmdir da/d24/d4c 39 2024-09-16T08:57:18.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/460: rmdir d3/d14/d37/d1a/d1d/d3b 39 2024-09-16T08:57:18.498 INFO:tasks.workunit.client.0.smithi031.stdout:3/461: chown d3/d14/d37/d1a/d24 0 1 2024-09-16T08:57:18.501 INFO:tasks.workunit.client.1.smithi097.stdout:5/888: rename db/d6e/d95/fa3 to db/d18/d1c/d42/d46/d7e/f128 0 2024-09-16T08:57:18.504 INFO:tasks.workunit.client.1.smithi097.stdout:3/836: unlink d7/de/d9c/ff2 0 2024-09-16T08:57:18.505 INFO:tasks.workunit.client.1.smithi097.stdout:8/769: rmdir d8/dd/d15/d2c/d4a 39 2024-09-16T08:57:18.506 INFO:tasks.workunit.client.1.smithi097.stdout:9/819: creat d2/d5/d6e/f103 x:0 0 0 2024-09-16T08:57:18.507 INFO:tasks.workunit.client.1.smithi097.stdout:0/827: creat da/d24/d4b/dfb/f122 x:0 0 0 2024-09-16T08:57:18.509 INFO:tasks.workunit.client.1.smithi097.stdout:7/750: dwrite d4/d7/d7b/ff5 [0,4194304] 0 2024-09-16T08:57:18.509 INFO:tasks.workunit.client.1.smithi097.stdout:7/751: readlink d4/d7/d7b/l89 0 2024-09-16T08:57:18.510 INFO:tasks.workunit.client.1.smithi097.stdout:3/837: rename d7/de/d3c/d3f/d60/dcb/d35/d3d/f59 to d7/de/d8d/f11d 0 2024-09-16T08:57:18.510 INFO:tasks.workunit.client.1.smithi097.stdout:0/828: truncate da/d24/d4c/d80/d3b/f7f 4637674 0 2024-09-16T08:57:18.514 INFO:tasks.workunit.client.1.smithi097.stdout:7/752: unlink d4/d7/d30/ddf/l8a 0 2024-09-16T08:57:18.516 INFO:tasks.workunit.client.1.smithi097.stdout:0/829: rmdir da/d24/df5 0 2024-09-16T08:57:18.517 INFO:tasks.workunit.client.0.smithi031.stdout:7/467: dread d2/d2d/f77 [0,4194304] 0 2024-09-16T08:57:18.519 INFO:tasks.workunit.client.1.smithi097.stdout:7/753: creat d4/d1b/da4/de0/ffc x:0 0 0 2024-09-16T08:57:18.519 INFO:tasks.workunit.client.1.smithi097.stdout:7/754: creat d4/d7/d7b/d43/daf/ffd x:0 0 0 2024-09-16T08:57:18.524 INFO:tasks.workunit.client.1.smithi097.stdout:0/830: link da/d24/d4c/d80/d3b/d86/ce8 da/d7e/dc4/c123 0 2024-09-16T08:57:18.524 INFO:tasks.workunit.client.1.smithi097.stdout:0/831: readlink da/d24/d4c/d80/lb8 0 2024-09-16T08:57:18.529 INFO:tasks.workunit.client.0.smithi031.stdout:8/318: dwrite d4/d34/f6a [0,4194304] 0 2024-09-16T08:57:18.531 INFO:tasks.workunit.client.1.smithi097.stdout:6/843: sync 2024-09-16T08:57:18.531 INFO:tasks.workunit.client.1.smithi097.stdout:6/844: dread - d1/dc/f80 zero size 2024-09-16T08:57:18.531 INFO:tasks.workunit.client.1.smithi097.stdout:7/755: rename d4/d7/d7b/d43/daf/lb8 to d4/d7/d7b/d43/lfe 0 2024-09-16T08:57:18.532 INFO:tasks.workunit.client.1.smithi097.stdout:0/832: mknod da/d7e/dc4/dfc/d47/db3/d60/d68/d10e/c124 0 2024-09-16T08:57:18.532 INFO:tasks.workunit.client.1.smithi097.stdout:3/838: write d7/de/d3c/d3f/f6d [3885768,50476] 0 2024-09-16T08:57:18.532 INFO:tasks.workunit.client.1.smithi097.stdout:3/839: chown d7/de/d3c/d3f/d60 27122845 1 2024-09-16T08:57:18.533 INFO:tasks.workunit.client.1.smithi097.stdout:6/845: truncate d1/dc/d3c/f6d 2772692 0 2024-09-16T08:57:18.533 INFO:tasks.workunit.client.1.smithi097.stdout:6/846: dread - d1/dd3/d9e/dca/d81/f10a zero size 2024-09-16T08:57:18.534 INFO:tasks.workunit.client.1.smithi097.stdout:7/756: mkdir d4/d7/d77/da8/de2/dff 0 2024-09-16T08:57:18.534 INFO:tasks.workunit.client.1.smithi097.stdout:7/757: fsync d4/d7/d7b/d43/fa7 0 2024-09-16T08:57:18.534 INFO:tasks.workunit.client.1.smithi097.stdout:7/758: chown d4/d1b/f69 106674 1 2024-09-16T08:57:18.535 INFO:tasks.workunit.client.0.smithi031.stdout:8/319: mknod d4/d7/d36/d38/c6c 0 2024-09-16T08:57:18.535 INFO:tasks.workunit.client.1.smithi097.stdout:3/840: unlink d7/l11 0 2024-09-16T08:57:18.536 INFO:tasks.workunit.client.1.smithi097.stdout:3/841: chown d7/de/d8d/d90/d96/d9f/db5/d110 3962 1 2024-09-16T08:57:18.536 INFO:tasks.workunit.client.0.smithi031.stdout:8/320: symlink d4/d7/d36/l6d 0 2024-09-16T08:57:18.537 INFO:tasks.workunit.client.1.smithi097.stdout:0/833: dread da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dad/fb7 [0,4194304] 0 2024-09-16T08:57:18.537 INFO:tasks.workunit.client.0.smithi031.stdout:8/321: mkdir d4/d2d/d6e 0 2024-09-16T08:57:18.538 INFO:tasks.workunit.client.1.smithi097.stdout:6/847: rmdir d1/dd3/d9e/dca/d81/da9 0 2024-09-16T08:57:18.538 INFO:tasks.workunit.client.1.smithi097.stdout:6/848: write d1/dd3/d9e/dca/d16/d4f/df8/f10f [741168,12055] 0 2024-09-16T08:57:18.539 INFO:tasks.workunit.client.1.smithi097.stdout:0/834: symlink da/d7e/dc4/dfc/d47/d97/l125 0 2024-09-16T08:57:18.539 INFO:tasks.workunit.client.1.smithi097.stdout:1/925: dwrite d5/db/dc/fad [0,4194304] 0 2024-09-16T08:57:18.540 INFO:tasks.workunit.client.0.smithi031.stdout:8/322: rename d4/d7/d21/d24/l69 to d4/d7/l6f 0 2024-09-16T08:57:18.542 INFO:tasks.workunit.client.1.smithi097.stdout:6/849: rename d1/dd3/d9e/dca/d16/d4f/cd4 to d1/dd3/d9e/dca/d81/dad/c11d 0 2024-09-16T08:57:18.543 INFO:tasks.workunit.client.0.smithi031.stdout:8/323: rename d4/d2f/f52 to d4/d7/d66/f70 0 2024-09-16T08:57:18.543 INFO:tasks.workunit.client.1.smithi097.stdout:6/850: dread d1/dd3/d9e/dca/fa7 [0,4194304] 0 2024-09-16T08:57:18.543 INFO:tasks.workunit.client.1.smithi097.stdout:6/851: chown d1/dd3/d9e/dca/d16/ff1 11899 1 2024-09-16T08:57:18.544 INFO:tasks.workunit.client.1.smithi097.stdout:6/852: write d1/dd3/d9e/dca/d16/d4f/f94 [795739,93049] 0 2024-09-16T08:57:18.544 INFO:tasks.workunit.client.1.smithi097.stdout:4/755: dwrite d0/d36/d4c/fac [4194304,4194304] 0 2024-09-16T08:57:18.544 INFO:tasks.workunit.client.1.smithi097.stdout:0/835: stat da/d24/d4c/d6d/c79 0 2024-09-16T08:57:18.544 INFO:tasks.workunit.client.1.smithi097.stdout:0/836: fdatasync da/d24/d4b/fa1 0 2024-09-16T08:57:18.546 INFO:tasks.workunit.client.0.smithi031.stdout:8/324: rmdir d4/d7/d66/d6b 0 2024-09-16T08:57:18.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/462: dwrite d3/d14/d37/d1a/d1d/d1e/f80 [0,4194304] 0 2024-09-16T08:57:18.548 INFO:tasks.workunit.client.1.smithi097.stdout:6/853: creat d1/dd3/d9e/db3/de3/f11e x:0 0 0 2024-09-16T08:57:18.549 INFO:tasks.workunit.client.1.smithi097.stdout:7/759: dread d4/d9/dc/f16 [0,4194304] 0 2024-09-16T08:57:18.549 INFO:tasks.workunit.client.1.smithi097.stdout:7/760: write d4/d12/d1f/d22/f3d [2899480,5211] 0 2024-09-16T08:57:18.549 INFO:tasks.workunit.client.1.smithi097.stdout:7/761: creat d4/d7/d19/f100 x:0 0 0 2024-09-16T08:57:18.549 INFO:tasks.workunit.client.0.smithi031.stdout:8/325: mknod d4/d34/c71 0 2024-09-16T08:57:18.556 INFO:tasks.workunit.client.1.smithi097.stdout:0/837: link da/d24/d4c/d80/d3b/d86/f104 da/d24/d4b/f126 0 2024-09-16T08:57:18.556 INFO:tasks.workunit.client.1.smithi097.stdout:1/926: read d5/d17/d54/d6b/da8/dc5/dc7/f11b [201837,102201] 0 2024-09-16T08:57:18.556 INFO:tasks.workunit.client.1.smithi097.stdout:5/889: dwrite db/d18/d20/f29 [0,4194304] 0 2024-09-16T08:57:18.556 INFO:tasks.workunit.client.1.smithi097.stdout:3/842: dwrite d7/de/d3c/d4f/f80 [0,4194304] 0 2024-09-16T08:57:18.556 INFO:tasks.workunit.client.1.smithi097.stdout:9/820: dwrite d2/d5/db0/dc8/d74/d8c/d94/d96/dac/f7f [0,4194304] 0 2024-09-16T08:57:18.556 INFO:tasks.workunit.client.1.smithi097.stdout:9/821: getdents d2/d5/db0/dc8/d74/de0 0 2024-09-16T08:57:18.557 INFO:tasks.workunit.client.1.smithi097.stdout:9/822: chown d2/d5/d56 1348134 1 2024-09-16T08:57:18.558 INFO:tasks.workunit.client.0.smithi031.stdout:8/326: link d4/d7/d21/d35/d40/f4c d4/d2f/f72 0 2024-09-16T08:57:18.558 INFO:tasks.workunit.client.0.smithi031.stdout:8/327: write d4/d7/f1c [4413696,99481] 0 2024-09-16T08:57:18.560 INFO:tasks.workunit.client.1.smithi097.stdout:6/854: creat d1/dd3/d9e/dca/d81/dc4/dc6/f11f x:0 0 0 2024-09-16T08:57:18.563 INFO:tasks.workunit.client.1.smithi097.stdout:7/762: rename d4/d1b/d41/l62 to d4/d1b/da4/de0/l101 0 2024-09-16T08:57:18.564 INFO:tasks.workunit.client.1.smithi097.stdout:7/763: fdatasync d4/d9/f24 0 2024-09-16T08:57:18.564 INFO:tasks.workunit.client.1.smithi097.stdout:7/764: chown d4/d9/dc/d8e 0 1 2024-09-16T08:57:18.566 INFO:tasks.workunit.client.1.smithi097.stdout:4/756: rmdir d0/d39/d101 0 2024-09-16T08:57:18.566 INFO:tasks.workunit.client.1.smithi097.stdout:0/838: truncate da/d7e/dc4/dfc/d47/db3/fd4 133294 0 2024-09-16T08:57:18.568 INFO:tasks.workunit.client.1.smithi097.stdout:1/927: chown d5/db/dc/d19/c1c 6544 1 2024-09-16T08:57:18.569 INFO:tasks.workunit.client.1.smithi097.stdout:5/890: dread db/d1a/fe2 [0,4194304] 0 2024-09-16T08:57:18.571 INFO:tasks.workunit.client.0.smithi031.stdout:3/463: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/f52 [0,4194304] 0 2024-09-16T08:57:18.572 INFO:tasks.workunit.client.1.smithi097.stdout:6/855: read d1/dd3/d9e/f100 [75722,6777] 0 2024-09-16T08:57:18.580 INFO:tasks.workunit.client.1.smithi097.stdout:1/928: dread d5/d70/f72 [0,4194304] 0 2024-09-16T08:57:18.580 INFO:tasks.workunit.client.1.smithi097.stdout:1/929: dread - d5/d2c/fdc zero size 2024-09-16T08:57:18.580 INFO:tasks.workunit.client.0.smithi031.stdout:3/464: mknod d3/d14/d85/c88 0 2024-09-16T08:57:18.582 INFO:tasks.workunit.client.1.smithi097.stdout:5/891: write db/d1a/f57 [3915827,38072] 0 2024-09-16T08:57:18.582 INFO:tasks.workunit.client.1.smithi097.stdout:7/765: link d4/d9/ca3 d4/d9/d51/db7/df9/c102 0 2024-09-16T08:57:18.583 INFO:tasks.workunit.client.1.smithi097.stdout:5/892: read db/d18/d20/d2b/f6f [1097378,107746] 0 2024-09-16T08:57:18.583 INFO:tasks.workunit.client.1.smithi097.stdout:7/766: chown d4/d9/dc/d8e/f92 33385981 1 2024-09-16T08:57:18.583 INFO:tasks.workunit.client.1.smithi097.stdout:5/893: write db/d22/f10c [759128,73696] 0 2024-09-16T08:57:18.583 INFO:tasks.workunit.client.1.smithi097.stdout:6/856: read d1/dd3/d9e/f10c [883011,60149] 0 2024-09-16T08:57:18.584 INFO:tasks.workunit.client.1.smithi097.stdout:4/757: link d0/d7/d52/l92 d0/d36/d4c/dea/l10c 0 2024-09-16T08:57:18.586 INFO:tasks.workunit.client.1.smithi097.stdout:0/839: creat da/d7e/dc4/dfc/d47/db3/d60/d73/f127 x:0 0 0 2024-09-16T08:57:18.587 INFO:tasks.workunit.client.0.smithi031.stdout:3/465: getdents d3 0 2024-09-16T08:57:18.587 INFO:tasks.workunit.client.0.smithi031.stdout:3/466: stat d3/d14/d37/d1a/f65 0 2024-09-16T08:57:18.588 INFO:tasks.workunit.client.1.smithi097.stdout:6/857: dread d1/dd3/d9e/dca/d81/dad/f62 [0,4194304] 0 2024-09-16T08:57:18.588 INFO:tasks.workunit.client.1.smithi097.stdout:3/843: rmdir d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa 39 2024-09-16T08:57:18.589 INFO:tasks.workunit.client.0.smithi031.stdout:3/467: symlink d3/d14/d6f/d7d/l89 0 2024-09-16T08:57:18.590 INFO:tasks.workunit.client.1.smithi097.stdout:9/823: getdents d2/d5/d72 0 2024-09-16T08:57:18.591 INFO:tasks.workunit.client.1.smithi097.stdout:1/930: mknod d5/ddd/c12b 0 2024-09-16T08:57:18.593 INFO:tasks.workunit.client.1.smithi097.stdout:7/767: rename d4/d1b/l26 to d4/d7/d19/ddd/l103 0 2024-09-16T08:57:18.594 INFO:tasks.workunit.client.1.smithi097.stdout:5/894: link db/d18/d1c/d42/c120 db/d6e/d79/c129 0 2024-09-16T08:57:18.594 INFO:tasks.workunit.client.1.smithi097.stdout:5/895: creat db/d18/d1c/d42/d46/d7e/d64/d8d/f12a x:0 0 0 2024-09-16T08:57:18.594 INFO:tasks.workunit.client.1.smithi097.stdout:5/896: fsync db/d1a/f38 0 2024-09-16T08:57:18.594 INFO:tasks.workunit.client.1.smithi097.stdout:5/897: dread - db/d18/d1c/d42/d46/d7e/d8f/d102/f126 zero size 2024-09-16T08:57:18.598 INFO:tasks.workunit.client.1.smithi097.stdout:0/840: truncate da/d1a/d41/f71 2653505 0 2024-09-16T08:57:18.598 INFO:tasks.workunit.client.1.smithi097.stdout:0/841: chown da/d7e/dc4/dfc/d94/l11b 1033004 1 2024-09-16T08:57:18.598 INFO:tasks.workunit.client.0.smithi031.stdout:3/468: write d3/d14/d37/d47/f66 [3723056,93749] 0 2024-09-16T08:57:18.600 INFO:tasks.workunit.client.1.smithi097.stdout:3/844: mkdir d7/de/d3c/d3f/d60/d11e 0 2024-09-16T08:57:18.601 INFO:tasks.workunit.client.1.smithi097.stdout:1/931: mkdir d5/d70/d9d/d12c 0 2024-09-16T08:57:18.606 INFO:tasks.workunit.client.1.smithi097.stdout:7/768: unlink d4/d1b/f78 0 2024-09-16T08:57:18.609 INFO:tasks.workunit.client.1.smithi097.stdout:6/858: dwrite d1/dd3/d9e/db3/de3/f11e [0,4194304] 0 2024-09-16T08:57:18.609 INFO:tasks.workunit.client.1.smithi097.stdout:4/758: rmdir d0/d35/da4/dc5/de7 39 2024-09-16T08:57:18.613 INFO:tasks.workunit.client.1.smithi097.stdout:1/932: truncate d5/d17/d54/d6b/d71/d94/f99 1153008 0 2024-09-16T08:57:18.615 INFO:tasks.workunit.client.1.smithi097.stdout:9/824: dwrite d2/d3f/f41 [0,4194304] 0 2024-09-16T08:57:18.615 INFO:tasks.workunit.client.1.smithi097.stdout:7/769: truncate d4/d7/d7b/f7c 349042 0 2024-09-16T08:57:18.615 INFO:tasks.workunit.client.1.smithi097.stdout:7/770: truncate d4/d9/dc/fcb 870533 0 2024-09-16T08:57:18.618 INFO:tasks.workunit.client.1.smithi097.stdout:5/898: getdents db/d6e/d95 0 2024-09-16T08:57:18.618 INFO:tasks.workunit.client.1.smithi097.stdout:5/899: chown db/d18/d1c/d42/d46/d7e/f53 33908161 1 2024-09-16T08:57:18.618 INFO:tasks.workunit.client.0.smithi031.stdout:3/469: dwrite d3/d14/d37/d47/f5b [0,4194304] 0 2024-09-16T08:57:18.618 INFO:tasks.workunit.client.0.smithi031.stdout:3/470: chown d3/d14/d37/d33/f7a 2011710 1 2024-09-16T08:57:18.621 INFO:tasks.workunit.client.1.smithi097.stdout:6/859: mknod d1/dd3/d9e/dca/d16/d86/c120 0 2024-09-16T08:57:18.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/471: write d3/d14/d37/d1a/d1d/f34 [345573,54134] 0 2024-09-16T08:57:18.627 INFO:tasks.workunit.client.0.smithi031.stdout:3/472: mkdir d3/d14/d37/d1a/d1d/d3b/d8a 0 2024-09-16T08:57:18.628 INFO:tasks.workunit.client.1.smithi097.stdout:0/842: rmdir da/d7e 39 2024-09-16T08:57:18.630 INFO:tasks.workunit.client.0.smithi031.stdout:3/473: creat d3/d14/d37/d1a/d1d/d3b/f8b x:0 0 0 2024-09-16T08:57:18.630 INFO:tasks.workunit.client.0.smithi031.stdout:3/474: chown d3/d14/d37/d1a/d1d/d3b/d3f/d79 794301852 1 2024-09-16T08:57:18.631 INFO:tasks.workunit.client.1.smithi097.stdout:4/759: dwrite d0/d7/df/d83/df8/dbc/dcc/ff7 [0,4194304] 0 2024-09-16T08:57:18.632 INFO:tasks.workunit.client.1.smithi097.stdout:3/845: rename d7/d28 to d7/d15/dee/d11f 0 2024-09-16T08:57:18.634 INFO:tasks.workunit.client.0.smithi031.stdout:3/475: link d3/d14/d37/f10 d3/d14/f8c 0 2024-09-16T08:57:18.635 INFO:tasks.workunit.client.1.smithi097.stdout:1/933: link d5/dce/ce1 d5/d2c/c12d 0 2024-09-16T08:57:18.636 INFO:tasks.workunit.client.0.smithi031.stdout:3/476: rmdir d3/d14/d37/d1a/d1d/d1e/d67 39 2024-09-16T08:57:18.640 INFO:tasks.workunit.client.1.smithi097.stdout:7/771: symlink d4/d7/d30/ddf/l104 0 2024-09-16T08:57:18.641 INFO:tasks.workunit.client.1.smithi097.stdout:5/900: mknod db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/c12b 0 2024-09-16T08:57:18.641 INFO:tasks.workunit.client.1.smithi097.stdout:5/901: dread - db/d18/d1c/d42/d46/d7e/fb8 zero size 2024-09-16T08:57:18.642 INFO:tasks.workunit.client.1.smithi097.stdout:5/902: chown db/d18/f68 96120895 1 2024-09-16T08:57:18.643 INFO:tasks.workunit.client.1.smithi097.stdout:4/760: dread d0/d7/df/d15/f11 [0,4194304] 0 2024-09-16T08:57:18.645 INFO:tasks.workunit.client.1.smithi097.stdout:0/843: link da/d24/d4c/d80/d42/fef da/d24/f128 0 2024-09-16T08:57:18.645 INFO:tasks.workunit.client.0.smithi031.stdout:3/477: dread d3/d14/d37/d1a/d1d/d3b/d3f/f74 [0,4194304] 0 2024-09-16T08:57:18.648 INFO:tasks.workunit.client.0.smithi031.stdout:3/478: dread d3/d14/d37/f10 [0,4194304] 0 2024-09-16T08:57:18.648 INFO:tasks.workunit.client.1.smithi097.stdout:9/825: rename d2/d5/db0/dc8/d74/d8c/lc7 to d2/d5/d6e/l104 0 2024-09-16T08:57:18.649 INFO:tasks.workunit.client.1.smithi097.stdout:9/826: chown d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/f91 80748752 1 2024-09-16T08:57:18.649 INFO:tasks.workunit.client.1.smithi097.stdout:9/827: write d2/fe [3184420,9186] 0 2024-09-16T08:57:18.651 INFO:tasks.workunit.client.0.smithi031.stdout:3/479: mknod d3/d14/d37/d47/c8d 0 2024-09-16T08:57:18.651 INFO:tasks.workunit.client.0.smithi031.stdout:3/480: stat d3/d14/d37/d33/f39 0 2024-09-16T08:57:18.654 INFO:tasks.workunit.client.1.smithi097.stdout:1/934: dwrite d5/db/f74 [0,4194304] 0 2024-09-16T08:57:18.655 INFO:tasks.workunit.client.0.smithi031.stdout:3/481: symlink d3/d14/d37/d1a/d1d/d1e/d67/l8e 0 2024-09-16T08:57:18.656 INFO:tasks.workunit.client.0.smithi031.stdout:9/336: sync 2024-09-16T08:57:18.656 INFO:tasks.workunit.client.0.smithi031.stdout:9/337: chown d3/f2a 13222 1 2024-09-16T08:57:18.657 INFO:tasks.workunit.client.1.smithi097.stdout:7/772: link d4/d9/d1c/d5f/c61 d4/d7/d19/ddd/c105 0 2024-09-16T08:57:18.657 INFO:tasks.workunit.client.1.smithi097.stdout:7/773: read - d4/ff6 zero size 2024-09-16T08:57:18.657 INFO:tasks.workunit.client.1.smithi097.stdout:6/860: getdents d1/dd3/d9e/dca/d16/d4f 0 2024-09-16T08:57:18.658 INFO:tasks.workunit.client.1.smithi097.stdout:6/861: read - d1/dd8/f116 zero size 2024-09-16T08:57:18.659 INFO:tasks.workunit.client.0.smithi031.stdout:9/338: mkdir d3/d5/d6a/d74 0 2024-09-16T08:57:18.661 INFO:tasks.workunit.client.1.smithi097.stdout:4/761: unlink d0/d36/d4c/le3 0 2024-09-16T08:57:18.661 INFO:tasks.workunit.client.1.smithi097.stdout:4/762: creat d0/d7/f10d x:0 0 0 2024-09-16T08:57:18.661 INFO:tasks.workunit.client.1.smithi097.stdout:4/763: chown d0/d7/d52/d26/f6b 9503 1 2024-09-16T08:57:18.664 INFO:tasks.workunit.client.1.smithi097.stdout:0/844: rename da/d7e/dc4/dfc/d47/db3/d60/d68/f5d to da/d1a/d9c/f129 0 2024-09-16T08:57:18.664 INFO:tasks.workunit.client.1.smithi097.stdout:1/935: rename d5/d17/d54/d6b to d5/d17/d54/d6b/da8/d12e 22 2024-09-16T08:57:18.664 INFO:tasks.workunit.client.1.smithi097.stdout:1/936: chown d5/d17/d54/l96 5867893 1 2024-09-16T08:57:18.664 INFO:tasks.workunit.client.1.smithi097.stdout:1/937: truncate d5/db/dc/fe0 595797 0 2024-09-16T08:57:18.665 INFO:tasks.workunit.client.1.smithi097.stdout:5/903: dwrite db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fda [0,4194304] 0 2024-09-16T08:57:18.665 INFO:tasks.workunit.client.1.smithi097.stdout:5/904: chown db/d18/f62 6768346 1 2024-09-16T08:57:18.665 INFO:tasks.workunit.client.1.smithi097.stdout:9/828: truncate d2/d5/db0/dc8/d74/d8c/d94/d96/dac/f81 911289 0 2024-09-16T08:57:18.666 INFO:tasks.workunit.client.1.smithi097.stdout:9/829: fdatasync d2/d5/db0/dc8/d74/d8c/d94/d96/dac/f81 0 2024-09-16T08:57:18.668 INFO:tasks.workunit.client.0.smithi031.stdout:9/339: read d3/f5b [1559511,57859] 0 2024-09-16T08:57:18.668 INFO:tasks.workunit.client.0.smithi031.stdout:9/340: chown d3/d5/da/db/d21/d3b/f58 15 1 2024-09-16T08:57:18.669 INFO:tasks.workunit.client.1.smithi097.stdout:7/774: creat d4/d9/d51/f106 x:0 0 0 2024-09-16T08:57:18.669 INFO:tasks.workunit.client.1.smithi097.stdout:7/775: fdatasync d4/d7/d7b/f7c 0 2024-09-16T08:57:18.669 INFO:tasks.workunit.client.1.smithi097.stdout:6/862: creat d1/dd3/d9e/dca/d16/d4f/d9f/f121 x:0 0 0 2024-09-16T08:57:18.670 INFO:tasks.workunit.client.0.smithi031.stdout:3/482: dread d3/d14/d37/d47/f5b [0,4194304] 0 2024-09-16T08:57:18.670 INFO:tasks.workunit.client.0.smithi031.stdout:3/483: dread - d3/d14/d37/f84 zero size 2024-09-16T08:57:18.671 INFO:tasks.workunit.client.1.smithi097.stdout:0/845: stat da/d24/d4c/d80/d3b/c82 0 2024-09-16T08:57:18.673 INFO:tasks.workunit.client.1.smithi097.stdout:3/846: write d7/de/d3c/d3f/d60/dcb/d35/f67 [829684,47970] 0 2024-09-16T08:57:18.673 INFO:tasks.workunit.client.1.smithi097.stdout:3/847: stat d7/de/d3c/d3f/d60/fe1 0 2024-09-16T08:57:18.678 INFO:tasks.workunit.client.0.smithi031.stdout:9/341: dread d3/d5/f6 [0,4194304] 0 2024-09-16T08:57:18.683 INFO:tasks.workunit.client.1.smithi097.stdout:3/848: dread d7/de/d6c/f37 [0,4194304] 0 2024-09-16T08:57:18.683 INFO:tasks.workunit.client.1.smithi097.stdout:5/905: creat db/d6e/d79/dd5/f12c x:0 0 0 2024-09-16T08:57:18.684 INFO:tasks.workunit.client.1.smithi097.stdout:5/906: chown db/d18/d1c/d42/d46/d7e/d76/dca/f11f 177070 1 2024-09-16T08:57:18.685 INFO:tasks.workunit.client.1.smithi097.stdout:9/830: rename d2/d3f/f41 to d2/d5/dd5/f105 0 2024-09-16T08:57:18.687 INFO:tasks.workunit.client.1.smithi097.stdout:6/863: creat d1/f122 x:0 0 0 2024-09-16T08:57:18.687 INFO:tasks.workunit.client.1.smithi097.stdout:6/864: write d1/dd3/d9e/dca/d16/d4f/f92 [4672665,129894] 0 2024-09-16T08:57:18.687 INFO:tasks.workunit.client.1.smithi097.stdout:6/865: fdatasync d1/dd3/d9e/dca/d16/d4f/d9f/f121 0 2024-09-16T08:57:18.688 INFO:tasks.workunit.client.1.smithi097.stdout:0/846: mkdir da/d7e/dc4/dfc/d94/d101/d12a 0 2024-09-16T08:57:18.689 INFO:tasks.workunit.client.1.smithi097.stdout:3/849: creat d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/f120 x:0 0 0 2024-09-16T08:57:18.689 INFO:tasks.workunit.client.1.smithi097.stdout:3/850: fdatasync d7/de/d3c/d3f/d60/d6b/f89 0 2024-09-16T08:57:18.689 INFO:tasks.workunit.client.1.smithi097.stdout:3/851: stat d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/fb6 0 2024-09-16T08:57:18.690 INFO:tasks.workunit.client.1.smithi097.stdout:6/866: creat d1/dc/d3c/f123 x:0 0 0 2024-09-16T08:57:18.691 INFO:tasks.workunit.client.1.smithi097.stdout:7/776: rename d4/d9/d1c/d5f/fbc to d4/d7/f107 0 2024-09-16T08:57:18.693 INFO:tasks.workunit.client.1.smithi097.stdout:6/867: creat d1/dd3/db0/f124 x:0 0 0 2024-09-16T08:57:18.693 INFO:tasks.workunit.client.1.smithi097.stdout:7/777: getdents d4/d12 0 2024-09-16T08:57:18.694 INFO:tasks.workunit.client.1.smithi097.stdout:9/831: rename d2/d5/db0/dc8/d63/da8/dca/df5/cf4 to d2/d5/c106 0 2024-09-16T08:57:18.696 INFO:tasks.workunit.client.1.smithi097.stdout:6/868: creat d1/dd3/d9e/dca/dbb/f125 x:0 0 0 2024-09-16T08:57:18.697 INFO:tasks.workunit.client.1.smithi097.stdout:6/869: truncate d1/dd3/d9e/dca/fa7 1642653 0 2024-09-16T08:57:18.698 INFO:tasks.workunit.client.1.smithi097.stdout:0/847: rename da/d7e/dc4/dfc/d47/d97 to da/d7e/dc4/dfc/d47/db3/dbf/d12b 0 2024-09-16T08:57:18.698 INFO:tasks.workunit.client.1.smithi097.stdout:0/848: fsync da/d7e/dc4/dfc/f59 0 2024-09-16T08:57:18.698 INFO:tasks.workunit.client.1.smithi097.stdout:0/849: write da/d1a/f8e [545381,13303] 0 2024-09-16T08:57:18.699 INFO:tasks.workunit.client.1.smithi097.stdout:2/861: sync 2024-09-16T08:57:18.699 INFO:tasks.workunit.client.1.smithi097.stdout:8/770: sync 2024-09-16T08:57:18.699 INFO:tasks.workunit.client.1.smithi097.stdout:8/771: write d8/dd/d15/d16/d7c/da2/fd8 [781377,114468] 0 2024-09-16T08:57:18.702 INFO:tasks.workunit.client.1.smithi097.stdout:8/772: creat d8/dd/d71/d9c/dac/ff1 x:0 0 0 2024-09-16T08:57:18.703 INFO:tasks.workunit.client.1.smithi097.stdout:8/773: mknod d8/dd/d71/cf2 0 2024-09-16T08:57:18.705 INFO:tasks.workunit.client.1.smithi097.stdout:8/774: mkdir d8/dd/d15/d2c/d4a/df3 0 2024-09-16T08:57:18.708 INFO:tasks.workunit.client.1.smithi097.stdout:0/850: write da/d7e/dc4/dfc/d94/d101/f66 [524787,105535] 0 2024-09-16T08:57:18.717 INFO:tasks.workunit.client.1.smithi097.stdout:5/907: dwrite db/d18/d1c/d42/d46/d7e/d8f/fc2 [0,4194304] 0 2024-09-16T08:57:18.721 INFO:tasks.workunit.client.1.smithi097.stdout:0/851: read da/d7e/dc4/dfc/d94/d101/f44 [1191404,49365] 0 2024-09-16T08:57:18.721 INFO:tasks.workunit.client.1.smithi097.stdout:0/852: write da/d24/d4c/d80/d3b/d86/faa [51272,65561] 0 2024-09-16T08:57:18.776 INFO:tasks.workunit.client.0.smithi031.stdout:3/484: dwrite d3/d14/d37/d1a/f76 [0,4194304] 0 2024-09-16T08:57:18.776 INFO:tasks.workunit.client.0.smithi031.stdout:3/485: creat d3/d14/d37/d1a/f8f x:0 0 0 2024-09-16T08:57:18.778 INFO:tasks.workunit.client.0.smithi031.stdout:5/420: sync 2024-09-16T08:57:18.778 INFO:tasks.workunit.client.0.smithi031.stdout:2/300: sync 2024-09-16T08:57:18.778 INFO:tasks.workunit.client.0.smithi031.stdout:4/363: sync 2024-09-16T08:57:18.778 INFO:tasks.workunit.client.0.smithi031.stdout:6/493: sync 2024-09-16T08:57:18.778 INFO:tasks.workunit.client.0.smithi031.stdout:9/342: write d3/d5/da/f2b [1234779,104056] 0 2024-09-16T08:57:18.778 INFO:tasks.workunit.client.0.smithi031.stdout:9/343: dread - d3/d5/da/f6b zero size 2024-09-16T08:57:18.778 INFO:tasks.workunit.client.0.smithi031.stdout:6/494: write d0/dd/d74/d5e/f77 [170353,75690] 0 2024-09-16T08:57:18.779 INFO:tasks.workunit.client.0.smithi031.stdout:8/328: sync 2024-09-16T08:57:18.779 INFO:tasks.workunit.client.0.smithi031.stdout:1/350: sync 2024-09-16T08:57:18.779 INFO:tasks.workunit.client.0.smithi031.stdout:0/402: sync 2024-09-16T08:57:18.779 INFO:tasks.workunit.client.0.smithi031.stdout:8/329: stat d4/d7/d21/d37 0 2024-09-16T08:57:18.779 INFO:tasks.workunit.client.0.smithi031.stdout:0/403: chown d6/d12/l52 734728469 1 2024-09-16T08:57:18.780 INFO:tasks.workunit.client.1.smithi097.stdout:5/908: dread db/d18/d1c/d42/d46/d7e/d76/f8e [0,4194304] 0 2024-09-16T08:57:18.782 INFO:tasks.workunit.client.1.smithi097.stdout:2/862: write d3/d4/d8/f100 [3247606,115847] 0 2024-09-16T08:57:18.782 INFO:tasks.workunit.client.1.smithi097.stdout:2/863: readlink d3/d4/l90 0 2024-09-16T08:57:18.802 INFO:tasks.workunit.client.1.smithi097.stdout:2/864: write d3/d4/d68/d8b/d7c/d7d/f6d [7107955,93337] 0 2024-09-16T08:57:18.802 INFO:tasks.workunit.client.1.smithi097.stdout:3/852: dwrite d7/d15/dee/d11f/d30/d88/fc1 [0,4194304] 0 2024-09-16T08:57:18.817 INFO:tasks.workunit.client.1.smithi097.stdout:5/909: getdents db/d1a 0 2024-09-16T08:57:18.822 INFO:tasks.workunit.client.0.smithi031.stdout:3/486: creat d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 x:0 0 0 2024-09-16T08:57:18.823 INFO:tasks.workunit.client.1.smithi097.stdout:0/853: dwrite da/d24/d4b/f103 [0,4194304] 0 2024-09-16T08:57:18.823 INFO:tasks.workunit.client.1.smithi097.stdout:6/870: dwrite d1/dc/d3c/f76 [4194304,4194304] 0 2024-09-16T08:57:18.823 INFO:tasks.workunit.client.1.smithi097.stdout:6/871: write d1/dd3/d9e/dca/d16/d4f/d9f/fdb [3564244,100015] 0 2024-09-16T08:57:18.823 INFO:tasks.workunit.client.1.smithi097.stdout:0/854: dread da/d1a/d9c/fb9 [0,4194304] 0 2024-09-16T08:57:18.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/301: creat d6/da/d37/f63 x:0 0 0 2024-09-16T08:57:18.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/302: chown d6/da/f3d 5 1 2024-09-16T08:57:18.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/303: chown d6/da/f5d 4624296 1 2024-09-16T08:57:18.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/304: write d6/d20/f26 [515694,45329] 0 2024-09-16T08:57:18.832 INFO:tasks.workunit.client.0.smithi031.stdout:4/364: unlink d0/f43 0 2024-09-16T08:57:18.832 INFO:tasks.workunit.client.0.smithi031.stdout:4/365: read - d0/d2/d16/f4e zero size 2024-09-16T08:57:18.832 INFO:tasks.workunit.client.0.smithi031.stdout:4/366: write d0/f38 [5198016,111561] 0 2024-09-16T08:57:18.832 INFO:tasks.workunit.client.0.smithi031.stdout:4/367: fsync d0/d2/d3a/f6f 0 2024-09-16T08:57:18.832 INFO:tasks.workunit.client.0.smithi031.stdout:4/368: fsync d0/d2/d16/d2b/f67 0 2024-09-16T08:57:18.836 INFO:tasks.workunit.client.1.smithi097.stdout:8/775: dwrite d8/dd/d15/d2c/d4a/f87 [0,4194304] 0 2024-09-16T08:57:18.836 INFO:tasks.workunit.client.1.smithi097.stdout:8/776: chown d8/dd/d71/d9c/fae 1057 1 2024-09-16T08:57:18.836 INFO:tasks.workunit.client.0.smithi031.stdout:0/404: dwrite d6/d12/f77 [0,4194304] 0 2024-09-16T08:57:18.836 INFO:tasks.workunit.client.0.smithi031.stdout:0/405: creat d6/d12/d47/f83 x:0 0 0 2024-09-16T08:57:18.837 INFO:tasks.workunit.client.0.smithi031.stdout:9/344: dwrite d3/d5/da/f33 [0,4194304] 0 2024-09-16T08:57:18.838 INFO:tasks.workunit.client.0.smithi031.stdout:6/495: dwrite d0/dd/f3e [0,4194304] 0 2024-09-16T08:57:18.839 INFO:tasks.workunit.client.0.smithi031.stdout:8/330: dwrite d4/d7/d66/f70 [0,4194304] 0 2024-09-16T08:57:18.848 INFO:tasks.workunit.client.0.smithi031.stdout:8/331: dread d4/d7/d66/f70 [0,4194304] 0 2024-09-16T08:57:18.862 INFO:tasks.workunit.client.0.smithi031.stdout:8/332: dread d4/d34/f6a [0,4194304] 0 2024-09-16T08:57:18.919 INFO:tasks.workunit.client.0.smithi031.stdout:2/305: creat d6/d41/d2c/f64 x:0 0 0 2024-09-16T08:57:18.921 INFO:tasks.workunit.client.0.smithi031.stdout:4/369: symlink d0/d3c/d6b/l7a 0 2024-09-16T08:57:18.922 INFO:tasks.workunit.client.1.smithi097.stdout:2/865: mknod d3/d4/d77/c10c 0 2024-09-16T08:57:18.923 INFO:tasks.workunit.client.0.smithi031.stdout:1/351: getdents d2/d15/d1f 0 2024-09-16T08:57:18.924 INFO:tasks.workunit.client.1.smithi097.stdout:3/853: mknod d7/de/d3c/dde/c121 0 2024-09-16T08:57:18.932 INFO:tasks.workunit.client.1.smithi097.stdout:2/866: dread d3/d1b/ff9 [0,4194304] 0 2024-09-16T08:57:18.933 INFO:tasks.workunit.client.0.smithi031.stdout:9/345: rename c0 to d3/d5/da/d34/d3c/c75 0 2024-09-16T08:57:18.948 INFO:tasks.workunit.client.0.smithi031.stdout:2/306: link d6/da/l14 d6/d41/d2c/l65 0 2024-09-16T08:57:18.949 INFO:tasks.workunit.client.1.smithi097.stdout:5/910: link db/d18/d1c/d42/c120 db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d11c/c12d 0 2024-09-16T08:57:18.950 INFO:tasks.workunit.client.1.smithi097.stdout:5/911: stat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fb6 0 2024-09-16T08:57:18.950 INFO:tasks.workunit.client.1.smithi097.stdout:2/867: dwrite d3/d4/d9/f5a [0,4194304] 0 2024-09-16T08:57:18.951 INFO:tasks.workunit.client.0.smithi031.stdout:9/346: creat d3/d5/d6a/f76 x:0 0 0 2024-09-16T08:57:18.952 INFO:tasks.workunit.client.1.smithi097.stdout:6/872: mknod d1/dd3/d9e/dca/d81/dc4/dc6/c126 0 2024-09-16T08:57:18.953 INFO:tasks.workunit.client.1.smithi097.stdout:6/873: fsync d1/dd3/d9e/dca/d16/d86/f8b 0 2024-09-16T08:57:18.959 INFO:tasks.workunit.client.1.smithi097.stdout:8/777: creat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/ff4 x:0 0 0 2024-09-16T08:57:18.959 INFO:tasks.workunit.client.1.smithi097.stdout:5/912: dread db/d1a/f57 [0,4194304] 0 2024-09-16T08:57:18.961 INFO:tasks.workunit.client.0.smithi031.stdout:9/347: dread d3/d48/f68 [0,4194304] 0 2024-09-16T08:57:18.961 INFO:tasks.workunit.client.0.smithi031.stdout:9/348: creat d3/d5/da/d34/f77 x:0 0 0 2024-09-16T08:57:18.966 INFO:tasks.workunit.client.0.smithi031.stdout:9/349: symlink d3/d5/da/db/d21/d3b/l78 0 2024-09-16T08:57:18.966 INFO:tasks.workunit.client.0.smithi031.stdout:9/350: truncate d3/d5/da/d34/f77 905377 0 2024-09-16T08:57:18.969 INFO:tasks.workunit.client.0.smithi031.stdout:9/351: link d3/d48/f68 d3/d5/da/db/d21/d3b/f79 0 2024-09-16T08:57:18.970 INFO:tasks.workunit.client.0.smithi031.stdout:7/468: sync 2024-09-16T08:57:18.972 INFO:tasks.workunit.client.0.smithi031.stdout:9/352: mknod d3/d5/da/db/d21/c7a 0 2024-09-16T08:57:18.972 INFO:tasks.workunit.client.0.smithi031.stdout:9/353: dread - d3/d5/d6a/f76 zero size 2024-09-16T08:57:18.972 INFO:tasks.workunit.client.0.smithi031.stdout:9/354: chown d3/d5/da/c69 165393988 1 2024-09-16T08:57:18.979 INFO:tasks.workunit.client.1.smithi097.stdout:6/874: unlink d1/dd3/d9e/dca/d16/c52 0 2024-09-16T08:57:18.980 INFO:tasks.workunit.client.1.smithi097.stdout:0/855: rmdir da/d24/d4b/dfb 39 2024-09-16T08:57:18.980 INFO:tasks.workunit.client.1.smithi097.stdout:0/856: write da/d7e/dc4/dfc/d47/db3/d60/d73/d85/fc8 [523886,43000] 0 2024-09-16T08:57:18.980 INFO:tasks.workunit.client.1.smithi097.stdout:0/857: chown da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dad 13412842 1 2024-09-16T08:57:18.981 INFO:tasks.workunit.client.1.smithi097.stdout:8/778: mkdir d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/df5 0 2024-09-16T08:57:18.982 INFO:tasks.workunit.client.1.smithi097.stdout:3/854: rename d7/d15/dee/d11f/d30/l72 to d7/de/d6c/dcf/l122 0 2024-09-16T08:57:18.983 INFO:tasks.workunit.client.1.smithi097.stdout:5/913: creat db/d6e/d81/f12e x:0 0 0 2024-09-16T08:57:18.985 INFO:tasks.workunit.client.1.smithi097.stdout:0/858: link da/df/d13/lb4 da/df/d8a/dac/l12c 0 2024-09-16T08:57:18.985 INFO:tasks.workunit.client.1.smithi097.stdout:0/859: dread - da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/fc7 zero size 2024-09-16T08:57:18.987 INFO:tasks.workunit.client.1.smithi097.stdout:8/779: link d8/dd/f2f d8/dd/d15/d16/ff6 0 2024-09-16T08:57:18.988 INFO:tasks.workunit.client.1.smithi097.stdout:5/914: rmdir db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d11c 39 2024-09-16T08:57:18.991 INFO:tasks.workunit.client.1.smithi097.stdout:8/780: creat d8/dd/d15/d2c/d4a/d4d/ddd/ff7 x:0 0 0 2024-09-16T08:57:18.991 INFO:tasks.workunit.client.1.smithi097.stdout:8/781: read d8/fad [1722271,75787] 0 2024-09-16T08:57:18.992 INFO:tasks.workunit.client.1.smithi097.stdout:8/782: getdents d8/dd/d15/d16/d7c 0 2024-09-16T08:57:18.992 INFO:tasks.workunit.client.1.smithi097.stdout:8/783: truncate d8/dd/d15/d2c/d4a/d4e/d85/fd6 724776 0 2024-09-16T08:57:18.994 INFO:tasks.workunit.client.1.smithi097.stdout:8/784: truncate d8/dd/d15/d2c/d4a/d4d/f49 215064 0 2024-09-16T08:57:18.994 INFO:tasks.workunit.client.1.smithi097.stdout:8/785: write d8/dd/d71/d9c/fae [2655129,1348] 0 2024-09-16T08:57:18.995 INFO:tasks.workunit.client.1.smithi097.stdout:8/786: link d8/dd/d15/d2c/d4a/d4e/le8 d8/dd/d71/lf8 0 2024-09-16T08:57:18.997 INFO:tasks.workunit.client.1.smithi097.stdout:8/787: rmdir d8/dd/d15/d2c/d4a/d4d/d3d 39 2024-09-16T08:57:18.997 INFO:tasks.workunit.client.1.smithi097.stdout:8/788: creat d8/dd/d15/d2c/ff9 x:0 0 0 2024-09-16T08:57:18.997 INFO:tasks.workunit.client.1.smithi097.stdout:8/789: dread - d8/dd/d71/d9c/dac/ff1 zero size 2024-09-16T08:57:18.997 INFO:tasks.workunit.client.1.smithi097.stdout:8/790: chown d8/dd/d15/d2c/d4a/d4d/l32 243 1 2024-09-16T08:57:18.997 INFO:tasks.workunit.client.1.smithi097.stdout:5/915: dread db/d1a/f92 [0,4194304] 0 2024-09-16T08:57:19.004 INFO:tasks.workunit.client.1.smithi097.stdout:3/855: dwrite d7/d15/dee/d11f/d30/f94 [0,4194304] 0 2024-09-16T08:57:19.004 INFO:tasks.workunit.client.1.smithi097.stdout:3/856: read - d7/de/d3c/d3f/d60/d6b/fa3 zero size 2024-09-16T08:57:19.008 INFO:tasks.workunit.client.1.smithi097.stdout:3/857: mkdir d7/d15/dee/d11f/d123 0 2024-09-16T08:57:19.014 INFO:tasks.workunit.client.1.smithi097.stdout:5/916: dread db/d18/d1c/d42/d46/d7e/f53 [0,4194304] 0 2024-09-16T08:57:19.014 INFO:tasks.workunit.client.1.smithi097.stdout:6/875: rename d1/dd3/d9e/dca/d81/ccd to d1/dd3/d9e/dca/d16/d4f/df8/c127 0 2024-09-16T08:57:19.014 INFO:tasks.workunit.client.1.smithi097.stdout:6/876: creat d1/dd3/db0/f128 x:0 0 0 2024-09-16T08:57:19.014 INFO:tasks.workunit.client.1.smithi097.stdout:6/877: chown d1/dd3/d9e/db3 0 1 2024-09-16T08:57:19.014 INFO:tasks.workunit.client.1.smithi097.stdout:6/878: write d1/dd3/f84 [783887,97446] 0 2024-09-16T08:57:19.015 INFO:tasks.workunit.client.1.smithi097.stdout:5/917: rename db/l4f to db/l12f 0 2024-09-16T08:57:19.018 INFO:tasks.workunit.client.1.smithi097.stdout:0/860: rename da/d24/d4b/dfb/l10f to da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dce/l12d 0 2024-09-16T08:57:19.021 INFO:tasks.workunit.client.1.smithi097.stdout:0/861: mknod da/c12e 0 2024-09-16T08:57:19.021 INFO:tasks.workunit.client.1.smithi097.stdout:0/862: fsync da/df/d8a/fda 0 2024-09-16T08:57:19.023 INFO:tasks.workunit.client.1.smithi097.stdout:8/791: dwrite d8/dd/d15/d16/f3c [0,4194304] 0 2024-09-16T08:57:19.026 INFO:tasks.workunit.client.1.smithi097.stdout:8/792: rename d8/dd/d15/d2c/d4a/d4e/db2/fde to d8/dd/d15/d16/d7c/da2/ffa 0 2024-09-16T08:57:19.026 INFO:tasks.workunit.client.1.smithi097.stdout:5/918: dread db/d18/d1c/d42/d46/f47 [0,4194304] 0 2024-09-16T08:57:19.027 INFO:tasks.workunit.client.0.smithi031.stdout:9/355: unlink d3/f3a 0 2024-09-16T08:57:19.030 INFO:tasks.workunit.client.1.smithi097.stdout:5/919: link db/d18/d1c/d42/d46/d7e/d76/dca/lf2 db/d1a/d6d/dce/l130 0 2024-09-16T08:57:19.033 INFO:tasks.workunit.client.1.smithi097.stdout:0/863: dread da/d24/f128 [0,4194304] 0 2024-09-16T08:57:19.033 INFO:tasks.workunit.client.1.smithi097.stdout:0/864: write da/d7e/dc4/dfc/d47/db3/fd4 [257863,42233] 0 2024-09-16T08:57:19.034 INFO:tasks.workunit.client.0.smithi031.stdout:5/421: link d2/fa d2/dc/f89 0 2024-09-16T08:57:19.035 INFO:tasks.workunit.client.1.smithi097.stdout:0/865: symlink da/d7e/dc4/dfc/d47/db3/dbf/l12f 0 2024-09-16T08:57:19.036 INFO:tasks.workunit.client.1.smithi097.stdout:0/866: symlink da/d7e/dc4/dfc/d47/db3/d60/d73/d85/l130 0 2024-09-16T08:57:19.041 INFO:tasks.workunit.client.1.smithi097.stdout:9/832: sync 2024-09-16T08:57:19.041 INFO:tasks.workunit.client.1.smithi097.stdout:9/833: fsync d2/d5/d3a/fd0 0 2024-09-16T08:57:19.041 INFO:tasks.workunit.client.1.smithi097.stdout:0/867: rename da/d7e/dc4/dfc/d94/d101/d12a to da/d24/d4b/dfb/d131 0 2024-09-16T08:57:19.042 INFO:tasks.workunit.client.1.smithi097.stdout:6/879: dwrite d1/db7/dfd/f96 [0,4194304] 0 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:6/880: creat d1/dd8/f129 x:0 0 0 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:6/881: truncate d1/dd3/d9e/dca/f2f 2205968 0 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:1/938: sync 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:7/778: sync 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:7/779: chown d4/d9/dc/c58 154474 1 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:4/764: sync 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:7/780: creat d4/d7/d7b/d43/daf/f108 x:0 0 0 2024-09-16T08:57:19.043 INFO:tasks.workunit.client.1.smithi097.stdout:7/781: write d4/d1b/da4/de0/ffc [340113,17501] 0 2024-09-16T08:57:19.045 INFO:tasks.workunit.client.1.smithi097.stdout:3/858: dwrite d7/d15/dee/d11f/d30/f53 [0,4194304] 0 2024-09-16T08:57:19.045 INFO:tasks.workunit.client.1.smithi097.stdout:0/868: truncate da/d7e/dc4/dfc/d47/db3/f7b 2958391 0 2024-09-16T08:57:19.045 INFO:tasks.workunit.client.1.smithi097.stdout:0/869: readlink da/d7e/dc4/dfc/d94/l11b 0 2024-09-16T08:57:19.045 INFO:tasks.workunit.client.1.smithi097.stdout:0/870: creat da/d1a/d9c/f132 x:0 0 0 2024-09-16T08:57:19.047 INFO:tasks.workunit.client.1.smithi097.stdout:6/882: creat d1/dd3/d9e/dca/d16/d4f/df8/f12a x:0 0 0 2024-09-16T08:57:19.049 INFO:tasks.workunit.client.1.smithi097.stdout:7/782: rename d4/d7/d77/lcd to d4/d7/d19/l109 0 2024-09-16T08:57:19.050 INFO:tasks.workunit.client.1.smithi097.stdout:7/783: truncate d4/d12/d1f/fc2 1201933 0 2024-09-16T08:57:19.050 INFO:tasks.workunit.client.1.smithi097.stdout:4/765: dread d0/d36/f47 [0,4194304] 0 2024-09-16T08:57:19.050 INFO:tasks.workunit.client.1.smithi097.stdout:3/859: getdents d7/de/d3c/d3f/d60/d6b 0 2024-09-16T08:57:19.053 INFO:tasks.workunit.client.1.smithi097.stdout:0/871: mknod da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dce/c133 0 2024-09-16T08:57:19.054 INFO:tasks.workunit.client.1.smithi097.stdout:7/784: creat d4/d7/d77/da8/de2/f10a x:0 0 0 2024-09-16T08:57:19.055 INFO:tasks.workunit.client.1.smithi097.stdout:4/766: creat d0/f10e x:0 0 0 2024-09-16T08:57:19.055 INFO:tasks.workunit.client.1.smithi097.stdout:4/767: fdatasync d0/d7/df/d83/ff5 0 2024-09-16T08:57:19.059 INFO:tasks.workunit.client.1.smithi097.stdout:3/860: creat d7/de/d3c/d3f/d60/dcb/d35/f124 x:0 0 0 2024-09-16T08:57:19.060 INFO:tasks.workunit.client.1.smithi097.stdout:0/872: creat da/d7e/f134 x:0 0 0 2024-09-16T08:57:19.060 INFO:tasks.workunit.client.1.smithi097.stdout:0/873: creat da/d1a/d41/db2/f135 x:0 0 0 2024-09-16T08:57:19.060 INFO:tasks.workunit.client.1.smithi097.stdout:0/874: chown da/d7e/dc4/dfc/d94/dc9 96893342 1 2024-09-16T08:57:19.061 INFO:tasks.workunit.client.1.smithi097.stdout:4/768: truncate d0/d7/df/f37 5511703 0 2024-09-16T08:57:19.061 INFO:tasks.workunit.client.1.smithi097.stdout:4/769: readlink d0/d35/d6c/lc7 0 2024-09-16T08:57:19.061 INFO:tasks.workunit.client.1.smithi097.stdout:4/770: truncate d0/d36/d4c/f80 418352 0 2024-09-16T08:57:19.062 INFO:tasks.workunit.client.1.smithi097.stdout:3/861: getdents d7/de/d3c/dcc 0 2024-09-16T08:57:19.062 INFO:tasks.workunit.client.1.smithi097.stdout:4/771: read d0/d36/da7/faa [18574,117611] 0 2024-09-16T08:57:19.062 INFO:tasks.workunit.client.1.smithi097.stdout:4/772: chown d0/d7/df/d83/fae 51596876 1 2024-09-16T08:57:19.064 INFO:tasks.workunit.client.1.smithi097.stdout:0/875: rmdir da/df/d13/db0 39 2024-09-16T08:57:19.064 INFO:tasks.workunit.client.1.smithi097.stdout:0/876: chown da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dce 2 1 2024-09-16T08:57:19.065 INFO:tasks.workunit.client.1.smithi097.stdout:7/785: read d4/d9/d1c/f38 [8355819,52899] 0 2024-09-16T08:57:19.065 INFO:tasks.workunit.client.1.smithi097.stdout:3/862: mknod d7/de/d3c/d4f/dfd/c125 0 2024-09-16T08:57:19.065 INFO:tasks.workunit.client.1.smithi097.stdout:3/863: write d7/de/d3c/d3f/d60/dcb/d35/d3d/f112 [1175602,101041] 0 2024-09-16T08:57:19.066 INFO:tasks.workunit.client.1.smithi097.stdout:5/920: dwrite db/d18/d1c/d42/d46/d7e/d64/fb3 [0,4194304] 0 2024-09-16T08:57:19.066 INFO:tasks.workunit.client.1.smithi097.stdout:5/921: creat db/d18/d20/f131 x:0 0 0 2024-09-16T08:57:19.066 INFO:tasks.workunit.client.1.smithi097.stdout:4/773: creat d0/d23/d5f/f10f x:0 0 0 2024-09-16T08:57:19.067 INFO:tasks.workunit.client.1.smithi097.stdout:0/877: mknod da/df/d8a/c136 0 2024-09-16T08:57:19.067 INFO:tasks.workunit.client.1.smithi097.stdout:7/786: getdents d4/d7/d30/ddf 0 2024-09-16T08:57:19.071 INFO:tasks.workunit.client.0.smithi031.stdout:3/487: mkdir d3/d14/d91 0 2024-09-16T08:57:19.074 INFO:tasks.workunit.client.1.smithi097.stdout:0/878: link da/d24/d4c/d80/d3b/f91 da/d7e/dc4/dfc/d47/db3/f137 0 2024-09-16T08:57:19.074 INFO:tasks.workunit.client.1.smithi097.stdout:0/879: read da/d7e/dc4/dfc/d47/db3/d60/d73/d85/f9f [2221922,24552] 0 2024-09-16T08:57:19.074 INFO:tasks.workunit.client.1.smithi097.stdout:0/880: write f8 [2370186,124231] 0 2024-09-16T08:57:19.074 INFO:tasks.workunit.client.1.smithi097.stdout:0/881: chown da/df/d13/c18 0 1 2024-09-16T08:57:19.076 INFO:tasks.workunit.client.1.smithi097.stdout:3/864: creat d7/de/d8d/d90/f126 x:0 0 0 2024-09-16T08:57:19.076 INFO:tasks.workunit.client.1.smithi097.stdout:7/787: symlink d4/d7/d77/da8/l10b 0 2024-09-16T08:57:19.076 INFO:tasks.workunit.client.1.smithi097.stdout:7/788: read - d4/d7/f107 zero size 2024-09-16T08:57:19.077 INFO:tasks.workunit.client.0.smithi031.stdout:2/307: creat d6/d20/f66 x:0 0 0 2024-09-16T08:57:19.077 INFO:tasks.workunit.client.1.smithi097.stdout:0/882: truncate da/d1a/d9c/f129 1433989 0 2024-09-16T08:57:19.079 INFO:tasks.workunit.client.1.smithi097.stdout:7/789: symlink d4/daa/l10c 0 2024-09-16T08:57:19.079 INFO:tasks.workunit.client.1.smithi097.stdout:0/883: read da/d7e/dc4/dfc/d47/fee [505776,18264] 0 2024-09-16T08:57:19.080 INFO:tasks.workunit.client.0.smithi031.stdout:6/496: rename d0/dd/l76 to d0/dd/d74/d19/d9a/l9e 0 2024-09-16T08:57:19.083 INFO:tasks.workunit.client.1.smithi097.stdout:7/790: rename d4/d9/l5e to d4/d9/dc/dc6/l10d 0 2024-09-16T08:57:19.084 INFO:tasks.workunit.client.0.smithi031.stdout:3/488: dread d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 [0,4194304] 0 2024-09-16T08:57:19.085 INFO:tasks.workunit.client.0.smithi031.stdout:3/489: getdents d3/d14/d37/d1a/d1d/d1e/d67/d7f 0 2024-09-16T08:57:19.085 INFO:tasks.workunit.client.1.smithi097.stdout:2/868: creat d3/d4/d68/d8b/f10d x:0 0 0 2024-09-16T08:57:19.086 INFO:tasks.workunit.client.0.smithi031.stdout:8/333: rename d4/d7/f1a to d4/d7/d36/d38/f73 0 2024-09-16T08:57:19.086 INFO:tasks.workunit.client.1.smithi097.stdout:5/922: dread db/d18/d20/d2b/f6f [0,4194304] 0 2024-09-16T08:57:19.087 INFO:tasks.workunit.client.1.smithi097.stdout:9/834: dwrite d2/d5/d3a/d3c/f54 [0,4194304] 0 2024-09-16T08:57:19.087 INFO:tasks.workunit.client.1.smithi097.stdout:9/835: chown d2/d5/d72/dc1/ddd/dee 794838551 1 2024-09-16T08:57:19.087 INFO:tasks.workunit.client.1.smithi097.stdout:9/836: stat d2/d5/db0/dc8/d74/d8c/d94/d96/dac/l68 0 2024-09-16T08:57:19.087 INFO:tasks.workunit.client.0.smithi031.stdout:6/497: creat d0/dd/d74/d5e/d7c/d3f/f9f x:0 0 0 2024-09-16T08:57:19.088 INFO:tasks.workunit.client.0.smithi031.stdout:5/422: dwrite d2/dc/d36/f52 [4194304,4194304] 0 2024-09-16T08:57:19.088 INFO:tasks.workunit.client.0.smithi031.stdout:5/423: creat d2/d4b/f8a x:0 0 0 2024-09-16T08:57:19.088 INFO:tasks.workunit.client.0.smithi031.stdout:5/424: creat d2/d4b/f8b x:0 0 0 2024-09-16T08:57:19.090 INFO:tasks.workunit.client.0.smithi031.stdout:1/352: rename d2/d15/d1f/c21 to d2/d4/d1b/d5c/d6b/c81 0 2024-09-16T08:57:19.092 INFO:tasks.workunit.client.1.smithi097.stdout:5/923: rmdir db/d18/d1c/d42 39 2024-09-16T08:57:19.092 INFO:tasks.workunit.client.1.smithi097.stdout:5/924: write db/f118 [803555,29534] 0 2024-09-16T08:57:19.092 INFO:tasks.workunit.client.1.smithi097.stdout:5/925: creat db/f132 x:0 0 0 2024-09-16T08:57:19.093 INFO:tasks.workunit.client.0.smithi031.stdout:1/353: read d2/d15/d5d/f23 [1420708,9960] 0 2024-09-16T08:57:19.095 INFO:tasks.workunit.client.0.smithi031.stdout:8/334: creat d4/d2d/d6e/f74 x:0 0 0 2024-09-16T08:57:19.095 INFO:tasks.workunit.client.0.smithi031.stdout:3/490: dread d3/d14/d37/d1a/d24/f77 [4194304,4194304] 0 2024-09-16T08:57:19.095 INFO:tasks.workunit.client.0.smithi031.stdout:3/491: dread - d3/d14/d37/f84 zero size 2024-09-16T08:57:19.098 INFO:tasks.workunit.client.1.smithi097.stdout:9/837: mknod d2/d5/d72/c107 0 2024-09-16T08:57:19.100 INFO:tasks.workunit.client.1.smithi097.stdout:2/869: dread d3/d4/d8/f25 [0,4194304] 0 2024-09-16T08:57:19.103 INFO:tasks.workunit.client.0.smithi031.stdout:9/356: rename d3/d5/da/d34/f3f to d3/d5/da/db/d21/d67/f7b 0 2024-09-16T08:57:19.104 INFO:tasks.workunit.client.0.smithi031.stdout:8/335: link d4/f29 d4/d7/d21/d37/d46/f75 0 2024-09-16T08:57:19.104 INFO:tasks.workunit.client.0.smithi031.stdout:8/336: stat d4/d7/f1c 0 2024-09-16T08:57:19.105 INFO:tasks.workunit.client.1.smithi097.stdout:5/926: getdents db/d18/d1c/d42/d46/d7e 0 2024-09-16T08:57:19.107 INFO:tasks.workunit.client.0.smithi031.stdout:1/354: dread d2/de/f2f [0,4194304] 0 2024-09-16T08:57:19.107 INFO:tasks.workunit.client.1.smithi097.stdout:2/870: dread d3/d1b/da7/fdf [0,4194304] 0 2024-09-16T08:57:19.110 INFO:tasks.workunit.client.1.smithi097.stdout:5/927: rename db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/fda to db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/f133 0 2024-09-16T08:57:19.110 INFO:tasks.workunit.client.1.smithi097.stdout:5/928: write db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/fb7 [819079,61417] 0 2024-09-16T08:57:19.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/337: link d4/f17 d4/d7/d21/d37/f76 0 2024-09-16T08:57:19.113 INFO:tasks.workunit.client.1.smithi097.stdout:2/871: getdents d3/d4/d9/d67/db2 0 2024-09-16T08:57:19.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/338: mkdir d4/d77 0 2024-09-16T08:57:19.116 INFO:tasks.workunit.client.1.smithi097.stdout:2/872: rename d3/d4/d68/d8b/d7c/d7d/d53/ff3 to d3/d4/d68/d8b/f10e 0 2024-09-16T08:57:19.116 INFO:tasks.workunit.client.0.smithi031.stdout:8/339: getdents d4/d2f 0 2024-09-16T08:57:19.118 INFO:tasks.workunit.client.0.smithi031.stdout:8/340: truncate d4/f6 3736498 0 2024-09-16T08:57:19.118 INFO:tasks.workunit.client.0.smithi031.stdout:8/341: write d4/d7/d21/d35/f5d [894233,70192] 0 2024-09-16T08:57:19.118 INFO:tasks.workunit.client.0.smithi031.stdout:8/342: readlink d4/d7/d36/l4e 0 2024-09-16T08:57:19.118 INFO:tasks.workunit.client.0.smithi031.stdout:8/343: mknod d4/d7/d21/d35/c78 0 2024-09-16T08:57:19.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:18 smithi031 ceph-mon[27844]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:19.125 INFO:tasks.workunit.client.0.smithi031.stdout:3/492: write d3/d14/d37/d1a/d1d/d3b/d3f/f62 [3568588,7871] 0 2024-09-16T08:57:19.125 INFO:tasks.workunit.client.0.smithi031.stdout:3/493: dread - d3/d14/d37/f84 zero size 2024-09-16T08:57:19.126 INFO:tasks.workunit.client.1.smithi097.stdout:8/793: dwrite d8/dd/d15/d16/ff6 [0,4194304] 0 2024-09-16T08:57:19.126 INFO:tasks.workunit.client.1.smithi097.stdout:6/883: dwrite d1/dd3/db0/f124 [0,4194304] 0 2024-09-16T08:57:19.126 INFO:tasks.workunit.client.1.smithi097.stdout:6/884: chown d1/dd3/f68 1 1 2024-09-16T08:57:19.127 INFO:tasks.workunit.client.1.smithi097.stdout:4/774: dwrite d0/d7/f10d [0,4194304] 0 2024-09-16T08:57:19.127 INFO:tasks.workunit.client.1.smithi097.stdout:4/775: read d0/fe [2097786,72282] 0 2024-09-16T08:57:19.130 INFO:tasks.workunit.client.1.smithi097.stdout:6/885: truncate d1/dc/d3c/f6d 28729 0 2024-09-16T08:57:19.131 INFO:tasks.workunit.client.1.smithi097.stdout:8/794: mknod d8/dd/d71/d9c/dac/cfb 0 2024-09-16T08:57:19.132 INFO:tasks.workunit.client.1.smithi097.stdout:8/795: read d8/dd/d15/d2c/d4a/d4d/d3f/fa6 [259325,120516] 0 2024-09-16T08:57:19.135 INFO:tasks.workunit.client.1.smithi097.stdout:6/886: mknod d1/dc/c12b 0 2024-09-16T08:57:19.135 INFO:tasks.workunit.client.1.smithi097.stdout:6/887: read - d1/dc/d3c/f123 zero size 2024-09-16T08:57:19.136 INFO:tasks.workunit.client.1.smithi097.stdout:8/796: symlink d8/dd/d15/d2c/d4a/d4d/d3d/db9/lfc 0 2024-09-16T08:57:19.137 INFO:tasks.workunit.client.1.smithi097.stdout:4/776: mknod d0/d39/c110 0 2024-09-16T08:57:19.139 INFO:tasks.workunit.client.1.smithi097.stdout:6/888: truncate d1/dd3/d9e/dca/d16/d86/f8a 290898 0 2024-09-16T08:57:19.139 INFO:tasks.workunit.client.1.smithi097.stdout:2/873: write d3/d4/d9/f5a [3304796,101999] 0 2024-09-16T08:57:19.140 INFO:tasks.workunit.client.1.smithi097.stdout:8/797: truncate d8/dd/d15/d16/d7c/fb7 2653426 0 2024-09-16T08:57:19.142 INFO:tasks.workunit.client.1.smithi097.stdout:4/777: rmdir d0/d36 39 2024-09-16T08:57:19.142 INFO:tasks.workunit.client.1.smithi097.stdout:4/778: write d0/d7/d52/dbf/fc6 [461094,118891] 0 2024-09-16T08:57:19.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/779: truncate d0/d36/f51 1508626 0 2024-09-16T08:57:19.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/780: write d0/d35/da4/dd1/f105 [556218,3789] 0 2024-09-16T08:57:19.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/781: write d0/d23/f10b [911493,11391] 0 2024-09-16T08:57:19.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/782: write d0/d23/ddc/ff6 [266309,34025] 0 2024-09-16T08:57:19.145 INFO:tasks.workunit.client.1.smithi097.stdout:4/783: write d0/d7/d52/d26/d8c/db0/fca [1025602,40116] 0 2024-09-16T08:57:19.146 INFO:tasks.workunit.client.0.smithi031.stdout:2/308: dwrite d6/fc [0,4194304] 0 2024-09-16T08:57:19.146 INFO:tasks.workunit.client.0.smithi031.stdout:5/425: dwrite d2/f59 [0,4194304] 0 2024-09-16T08:57:19.147 INFO:tasks.workunit.client.0.smithi031.stdout:2/309: chown d6/l59 514504340 1 2024-09-16T08:57:19.149 INFO:tasks.workunit.client.1.smithi097.stdout:7/791: dwrite d4/d1b/fb5 [4194304,4194304] 0 2024-09-16T08:57:19.149 INFO:tasks.workunit.client.1.smithi097.stdout:4/784: rename d0/d35/da4/dc5/fe2 to d0/d36/da7/f111 0 2024-09-16T08:57:19.149 INFO:tasks.workunit.client.0.smithi031.stdout:8/344: dread d4/d7/f51 [0,4194304] 0 2024-09-16T08:57:19.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/355: dwrite d2/d15/d32/f5f [0,4194304] 0 2024-09-16T08:57:19.150 INFO:tasks.workunit.client.0.smithi031.stdout:9/357: dwrite d3/d5/da/db/f1e [0,4194304] 0 2024-09-16T08:57:19.150 INFO:tasks.workunit.client.1.smithi097.stdout:0/884: dwrite da/d7e/dc4/dfc/d47/db3/d60/d73/f77 [0,4194304] 0 2024-09-16T08:57:19.150 INFO:tasks.workunit.client.1.smithi097.stdout:5/929: dwrite db/d1a/f92 [0,4194304] 0 2024-09-16T08:57:19.150 INFO:tasks.workunit.client.1.smithi097.stdout:5/930: chown db/d1a/lb9 308 1 2024-09-16T08:57:19.150 INFO:tasks.workunit.client.1.smithi097.stdout:5/931: getdents db/d18/d1c/d42/d124 0 2024-09-16T08:57:19.151 INFO:tasks.workunit.client.1.smithi097.stdout:3/865: dwrite d7/d15/dee/d11f/d30/d88/fc1 [0,4194304] 0 2024-09-16T08:57:19.152 INFO:tasks.workunit.client.1.smithi097.stdout:3/866: chown d7/de/d3c/dde/ff0 895953156 1 2024-09-16T08:57:19.157 INFO:tasks.workunit.client.0.smithi031.stdout:5/426: mkdir d2/dc/d8c 0 2024-09-16T08:57:19.157 INFO:tasks.workunit.client.0.smithi031.stdout:5/427: write d2/dc/d25/d68/f6b [535665,63431] 0 2024-09-16T08:57:19.157 INFO:tasks.workunit.client.0.smithi031.stdout:5/428: readlink d2/dc/d1d/l1e 0 2024-09-16T08:57:19.158 INFO:tasks.workunit.client.1.smithi097.stdout:7/792: truncate d4/d1b/d41/f52 4628655 0 2024-09-16T08:57:19.161 INFO:tasks.workunit.client.1.smithi097.stdout:4/785: creat d0/d36/da7/f112 x:0 0 0 2024-09-16T08:57:19.162 INFO:tasks.workunit.client.0.smithi031.stdout:8/345: mkdir d4/d7/d21/d35/d79 0 2024-09-16T08:57:19.164 INFO:tasks.workunit.client.0.smithi031.stdout:1/356: creat d2/d4/f82 x:0 0 0 2024-09-16T08:57:19.169 INFO:tasks.workunit.client.0.smithi031.stdout:9/358: link d3/d5/da/l45 d3/l7c 0 2024-09-16T08:57:19.169 INFO:tasks.workunit.client.0.smithi031.stdout:9/359: dread - d3/d5/da/f4f zero size 2024-09-16T08:57:19.171 INFO:tasks.workunit.client.1.smithi097.stdout:0/885: rename da/d7e/dc4/dfc/d47/db3/dbf/d12b/lbb to da/df/d13/db0/l138 0 2024-09-16T08:57:19.171 INFO:tasks.workunit.client.1.smithi097.stdout:0/886: write da/df/d8a/fda [1135679,52639] 0 2024-09-16T08:57:19.172 INFO:tasks.workunit.client.1.smithi097.stdout:0/887: write da/d7e/dc4/dfc/f6b [4330185,87578] 0 2024-09-16T08:57:19.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/346: write d4/d34/f6a [106430,2717] 0 2024-09-16T08:57:19.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/347: readlink d4/d7/d36/l4e 0 2024-09-16T08:57:19.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/348: readlink d4/d7/d36/l4e 0 2024-09-16T08:57:19.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/349: chown d4/d7/d21/d35/c78 22134 1 2024-09-16T08:57:19.173 INFO:tasks.workunit.client.1.smithi097.stdout:5/932: mkdir db/d22/d58/d134 0 2024-09-16T08:57:19.174 INFO:tasks.workunit.client.1.smithi097.stdout:8/798: dwrite d8/dd/d15/d2c/d4a/d4e/f70 [0,4194304] 0 2024-09-16T08:57:19.177 INFO:tasks.workunit.client.1.smithi097.stdout:2/874: dwrite d3/d4/d68/d8b/d7c/d7d/fcd [0,4194304] 0 2024-09-16T08:57:19.178 INFO:tasks.workunit.client.1.smithi097.stdout:2/875: chown d3/d4/d68/d8b/d7c/d7d/fb5 1439554 1 2024-09-16T08:57:19.178 INFO:tasks.workunit.client.0.smithi031.stdout:2/310: getdents d6/da/d4b/d4c 0 2024-09-16T08:57:19.180 INFO:tasks.workunit.client.1.smithi097.stdout:3/867: dwrite d7/de/d6c/f4a [0,4194304] 0 2024-09-16T08:57:19.186 INFO:tasks.workunit.client.0.smithi031.stdout:8/350: symlink d4/d7/d66/l7a 0 2024-09-16T08:57:19.186 INFO:tasks.workunit.client.0.smithi031.stdout:8/351: chown d4/d7/c1b 635028 1 2024-09-16T08:57:19.186 INFO:tasks.workunit.client.0.smithi031.stdout:6/498: getdents d0/dd/d74/d5e/d7c/d3f 0 2024-09-16T08:57:19.186 INFO:tasks.workunit.client.0.smithi031.stdout:6/499: chown d0/dd/d2b/l39 835399 1 2024-09-16T08:57:19.186 INFO:tasks.workunit.client.0.smithi031.stdout:6/500: write d0/d10/f60 [772791,4176] 0 2024-09-16T08:57:19.188 INFO:tasks.workunit.client.1.smithi097.stdout:4/786: dwrite d0/f2a [4194304,4194304] 0 2024-09-16T08:57:19.188 INFO:tasks.workunit.client.1.smithi097.stdout:4/787: read - d0/d23/d5f/f93 zero size 2024-09-16T08:57:19.195 INFO:tasks.workunit.client.0.smithi031.stdout:6/501: getdents d0/dd/d74/d5e/d7c/d56 0 2024-09-16T08:57:19.197 INFO:tasks.workunit.client.0.smithi031.stdout:2/311: rmdir d6/d2e/d40 39 2024-09-16T08:57:19.197 INFO:tasks.workunit.client.0.smithi031.stdout:2/312: readlink d6/da/l55 0 2024-09-16T08:57:19.200 INFO:tasks.workunit.client.0.smithi031.stdout:1/357: dwrite f0 [0,4194304] 0 2024-09-16T08:57:19.200 INFO:tasks.workunit.client.0.smithi031.stdout:1/358: truncate d2/f53 659056 0 2024-09-16T08:57:19.200 INFO:tasks.workunit.client.0.smithi031.stdout:6/502: link d0/dd/d35/d6c/d81/l87 d0/dd/d74/d5e/la0 0 2024-09-16T08:57:19.200 INFO:tasks.workunit.client.0.smithi031.stdout:6/503: fdatasync d0/f91 0 2024-09-16T08:57:19.202 INFO:tasks.workunit.client.1.smithi097.stdout:5/933: dwrite db/d22/f96 [0,4194304] 0 2024-09-16T08:57:19.204 INFO:tasks.workunit.client.0.smithi031.stdout:6/504: creat d0/dd/d74/d19/fa1 x:0 0 0 2024-09-16T08:57:19.206 INFO:tasks.workunit.client.1.smithi097.stdout:7/793: mkdir d4/d9/d51/db7/d10e 0 2024-09-16T08:57:19.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/359: getdents d2 0 2024-09-16T08:57:19.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/360: fsync d2/d4/d1b/d5c/d6b/f7b 0 2024-09-16T08:57:19.207 INFO:tasks.workunit.client.0.smithi031.stdout:1/361: fsync d2/de/f13 0 2024-09-16T08:57:19.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/352: dwrite d4/d7/f67 [0,4194304] 0 2024-09-16T08:57:19.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/353: truncate d4/d5b/f62 748888 0 2024-09-16T08:57:19.207 INFO:tasks.workunit.client.0.smithi031.stdout:8/354: truncate d4/d7/f67 4625014 0 2024-09-16T08:57:19.208 INFO:tasks.workunit.client.1.smithi097.stdout:0/888: mknod da/d24/d4b/c139 0 2024-09-16T08:57:19.209 INFO:tasks.workunit.client.0.smithi031.stdout:6/505: mknod d0/ca2 0 2024-09-16T08:57:19.211 INFO:tasks.workunit.client.0.smithi031.stdout:1/362: mknod d2/d15/c83 0 2024-09-16T08:57:19.211 INFO:tasks.workunit.client.0.smithi031.stdout:1/363: write d2/d15/d4a/f6a [1051464,95954] 0 2024-09-16T08:57:19.211 INFO:tasks.workunit.client.0.smithi031.stdout:1/364: chown d2/d4/c25 592338 1 2024-09-16T08:57:19.212 INFO:tasks.workunit.client.1.smithi097.stdout:2/876: mkdir d3/d4/d68/d8b/d61/dac/dd9/d10f 0 2024-09-16T08:57:19.213 INFO:tasks.workunit.client.0.smithi031.stdout:2/313: dread d6/f2d [0,4194304] 0 2024-09-16T08:57:19.218 INFO:tasks.workunit.client.0.smithi031.stdout:1/365: mknod d2/d15/d1e/c84 0 2024-09-16T08:57:19.218 INFO:tasks.workunit.client.0.smithi031.stdout:1/366: dread - d2/d4/d52/f74 zero size 2024-09-16T08:57:19.218 INFO:tasks.workunit.client.0.smithi031.stdout:1/367: creat d2/d15/d38/f85 x:0 0 0 2024-09-16T08:57:19.221 INFO:tasks.workunit.client.0.smithi031.stdout:2/314: symlink d6/da/d4b/l67 0 2024-09-16T08:57:19.226 INFO:tasks.workunit.client.1.smithi097.stdout:7/794: dread d4/d7/fbd [0,4194304] 0 2024-09-16T08:57:19.226 INFO:tasks.workunit.client.1.smithi097.stdout:3/868: getdents d7/de/d3c/d3f/d60/dcb/d35/d3d 0 2024-09-16T08:57:19.227 INFO:tasks.workunit.client.0.smithi031.stdout:2/315: mkdir d6/da/d4b/d4c/d68 0 2024-09-16T08:57:19.229 INFO:tasks.workunit.client.0.smithi031.stdout:2/316: mkdir d6/da/d37/d42/d69 0 2024-09-16T08:57:19.229 INFO:tasks.workunit.client.0.smithi031.stdout:2/317: creat d6/da/f6a x:0 0 0 2024-09-16T08:57:19.234 INFO:tasks.workunit.client.0.smithi031.stdout:2/318: symlink d6/da/d37/l6b 0 2024-09-16T08:57:19.235 INFO:tasks.workunit.client.1.smithi097.stdout:5/934: unlink db/ffb 0 2024-09-16T08:57:19.236 INFO:tasks.workunit.client.1.smithi097.stdout:2/877: truncate d3/d4/d68/d8b/d7c/d7d/d53/fdc 2766151 0 2024-09-16T08:57:19.243 INFO:tasks.workunit.client.1.smithi097.stdout:3/869: truncate d7/de/d8d/f11d 3986532 0 2024-09-16T08:57:19.245 INFO:tasks.workunit.client.1.smithi097.stdout:4/788: dread d0/d39/d71/f77 [0,4194304] 0 2024-09-16T08:57:19.245 INFO:tasks.workunit.client.1.smithi097.stdout:2/878: getdents d3/d4/d9/d78 0 2024-09-16T08:57:19.245 INFO:tasks.workunit.client.0.smithi031.stdout:8/355: dwrite d4/f29 [0,4194304] 0 2024-09-16T08:57:19.246 INFO:tasks.workunit.client.0.smithi031.stdout:2/319: rename d6/d41 to d6/d2e/d40/d6c 0 2024-09-16T08:57:19.247 INFO:tasks.workunit.client.0.smithi031.stdout:2/320: dread - d6/d20/f66 zero size 2024-09-16T08:57:19.247 INFO:tasks.workunit.client.1.smithi097.stdout:7/795: getdents d4/d7/d7b/d43/daf 0 2024-09-16T08:57:19.247 INFO:tasks.workunit.client.1.smithi097.stdout:7/796: chown d4/d12/d1f/d22/ff1 384336321 1 2024-09-16T08:57:19.248 INFO:tasks.workunit.client.1.smithi097.stdout:7/797: read - d4/d7/d7b/d43/daf/f108 zero size 2024-09-16T08:57:19.248 INFO:tasks.workunit.client.1.smithi097.stdout:5/935: write db/d18/d1c/d42/d46/f47 [2451792,64807] 0 2024-09-16T08:57:19.249 INFO:tasks.workunit.client.1.smithi097.stdout:4/789: mknod d0/d23/d5f/c113 0 2024-09-16T08:57:19.250 INFO:tasks.workunit.client.1.smithi097.stdout:2/879: symlink d3/d1b/dbf/l110 0 2024-09-16T08:57:19.251 INFO:tasks.workunit.client.0.smithi031.stdout:8/356: mkdir d4/d5b/d7b 0 2024-09-16T08:57:19.252 INFO:tasks.workunit.client.1.smithi097.stdout:7/798: creat d4/d9/dc/f10f x:0 0 0 2024-09-16T08:57:19.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/321: symlink d6/da/d37/l6d 0 2024-09-16T08:57:19.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/322: write d6/d2e/d40/d6c/d2c/d53/d60/f39 [226120,83101] 0 2024-09-16T08:57:19.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/323: chown d6/da/l45 1 1 2024-09-16T08:57:19.255 INFO:tasks.workunit.client.0.smithi031.stdout:8/357: mknod d4/d7/d21/d37/c7c 0 2024-09-16T08:57:19.257 INFO:tasks.workunit.client.1.smithi097.stdout:0/889: dwrite da/d24/d4b/fa1 [0,4194304] 0 2024-09-16T08:57:19.257 INFO:tasks.workunit.client.1.smithi097.stdout:0/890: chown da/d7e/dc4/dfc/d47/db3/d60/c87 610723 1 2024-09-16T08:57:19.257 INFO:tasks.workunit.client.1.smithi097.stdout:0/891: fsync da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/fc7 0 2024-09-16T08:57:19.258 INFO:tasks.workunit.client.1.smithi097.stdout:0/892: read da/d24/d4b/f126 [1306858,76660] 0 2024-09-16T08:57:19.258 INFO:tasks.workunit.client.1.smithi097.stdout:0/893: fsync da/d24/d76/f8b 0 2024-09-16T08:57:19.258 INFO:tasks.workunit.client.1.smithi097.stdout:4/790: creat d0/d7/df/d15/f114 x:0 0 0 2024-09-16T08:57:19.258 INFO:tasks.workunit.client.0.smithi031.stdout:6/506: dwrite d0/f15 [0,4194304] 0 2024-09-16T08:57:19.258 INFO:tasks.workunit.client.0.smithi031.stdout:9/360: truncate d3/d5/da/db/d21/f3d 3864091 0 2024-09-16T08:57:19.258 INFO:tasks.workunit.client.0.smithi031.stdout:9/361: fsync d3/d48/f68 0 2024-09-16T08:57:19.259 INFO:tasks.workunit.client.0.smithi031.stdout:9/362: chown d3/d5/d73 767444 1 2024-09-16T08:57:19.259 INFO:tasks.workunit.client.1.smithi097.stdout:3/870: write d7/fa [1649778,7293] 0 2024-09-16T08:57:19.259 INFO:tasks.workunit.client.1.smithi097.stdout:5/936: dread db/d18/d20/f29 [0,4194304] 0 2024-09-16T08:57:19.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/368: dwrite d2/d4/f28 [0,4194304] 0 2024-09-16T08:57:19.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/369: readlink d2/d15/d5d/l48 0 2024-09-16T08:57:19.260 INFO:tasks.workunit.client.0.smithi031.stdout:1/370: fsync d2/d4/d1b/f78 0 2024-09-16T08:57:19.261 INFO:tasks.workunit.client.0.smithi031.stdout:6/507: unlink d0/f72 0 2024-09-16T08:57:19.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:18 smithi097 ceph-mon[35478]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:19.262 INFO:tasks.workunit.client.1.smithi097.stdout:0/894: getdents da/d7e/dc4/dfc/d47/db3/dbf/dcc 0 2024-09-16T08:57:19.262 INFO:tasks.workunit.client.1.smithi097.stdout:0/895: truncate da/d7e/dc4/dfc/d47/fee 1313861 0 2024-09-16T08:57:19.265 INFO:tasks.workunit.client.0.smithi031.stdout:9/363: rename d3/d5/da/c51 to d3/d5/da/d34/d3c/d42/c7d 0 2024-09-16T08:57:19.269 INFO:tasks.workunit.client.1.smithi097.stdout:3/871: unlink d7/de/d6c/f6f 0 2024-09-16T08:57:19.270 INFO:tasks.workunit.client.0.smithi031.stdout:8/358: dread d4/d7/d21/d37/d46/f75 [0,4194304] 0 2024-09-16T08:57:19.271 INFO:tasks.workunit.client.1.smithi097.stdout:4/791: unlink d0/d39/f6f 0 2024-09-16T08:57:19.271 INFO:tasks.workunit.client.1.smithi097.stdout:4/792: write d0/d23/f3a [9204315,127089] 0 2024-09-16T08:57:19.271 INFO:tasks.workunit.client.1.smithi097.stdout:4/793: write d0/d39/d71/f77 [5198131,74986] 0 2024-09-16T08:57:19.271 INFO:tasks.workunit.client.1.smithi097.stdout:4/794: truncate d0/d35/da4/dc5/de7/fec 175723 0 2024-09-16T08:57:19.273 INFO:tasks.workunit.client.1.smithi097.stdout:3/872: mknod d7/d15/dee/d11f/d123/c127 0 2024-09-16T08:57:19.273 INFO:tasks.workunit.client.1.smithi097.stdout:3/873: write d7/de/d3c/dbe/fce [980339,120355] 0 2024-09-16T08:57:19.275 INFO:tasks.workunit.client.1.smithi097.stdout:4/795: write d0/d39/f6a [4015043,110244] 0 2024-09-16T08:57:19.276 INFO:tasks.workunit.client.1.smithi097.stdout:3/874: symlink d7/de/d3c/dcc/l128 0 2024-09-16T08:57:19.279 INFO:tasks.workunit.client.1.smithi097.stdout:3/875: mknod d7/d15/dee/d11f/d30/d69/c129 0 2024-09-16T08:57:19.280 INFO:tasks.workunit.client.1.smithi097.stdout:0/896: dwrite da/d1a/f2b [0,4194304] 0 2024-09-16T08:57:19.282 INFO:tasks.workunit.client.0.smithi031.stdout:6/508: mknod d0/ca3 0 2024-09-16T08:57:19.285 INFO:tasks.workunit.client.1.smithi097.stdout:0/897: mknod da/df/d8a/dac/c13a 0 2024-09-16T08:57:19.286 INFO:tasks.workunit.client.0.smithi031.stdout:1/371: rename d2/d15/d1f/d43/l4d to d2/d15/d4a/l86 0 2024-09-16T08:57:19.286 INFO:tasks.workunit.client.0.smithi031.stdout:1/372: chown d2/d4/d1b/d5c/c64 1370 1 2024-09-16T08:57:19.286 INFO:tasks.workunit.client.0.smithi031.stdout:8/359: dread d4/f8 [12582912,4194304] 0 2024-09-16T08:57:19.287 INFO:tasks.workunit.client.0.smithi031.stdout:9/364: mknod d3/d5/da/d34/c7e 0 2024-09-16T08:57:19.292 INFO:tasks.workunit.client.0.smithi031.stdout:6/509: rename d0/f84 to d0/dd/d74/d5e/d7c/d3f/d92/fa4 0 2024-09-16T08:57:19.292 INFO:tasks.workunit.client.0.smithi031.stdout:6/510: fdatasync d0/dd/f2f 0 2024-09-16T08:57:19.294 INFO:tasks.workunit.client.0.smithi031.stdout:1/373: mkdir d2/d15/d32/d87 0 2024-09-16T08:57:19.294 INFO:tasks.workunit.client.1.smithi097.stdout:0/898: dread da/d1a/f1c [0,4194304] 0 2024-09-16T08:57:19.294 INFO:tasks.workunit.client.1.smithi097.stdout:0/899: readlink da/df/d8a/dac/l121 0 2024-09-16T08:57:19.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/360: creat d4/d5b/d7b/f7d x:0 0 0 2024-09-16T08:57:19.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/361: fsync d4/d7/d21/f23 0 2024-09-16T08:57:19.296 INFO:tasks.workunit.client.1.smithi097.stdout:2/880: fsync d3/d1b/f83 0 2024-09-16T08:57:19.303 INFO:tasks.workunit.client.1.smithi097.stdout:2/881: dread d3/d4/d9/d67/fb6 [0,4194304] 0 2024-09-16T08:57:19.310 INFO:tasks.workunit.client.0.smithi031.stdout:1/374: dread d2/de/f13 [0,4194304] 0 2024-09-16T08:57:19.313 INFO:tasks.workunit.client.0.smithi031.stdout:8/362: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:19.315 INFO:tasks.workunit.client.1.smithi097.stdout:4/796: dwrite d0/d23/ddc/ffc [0,4194304] 0 2024-09-16T08:57:19.315 INFO:tasks.workunit.client.1.smithi097.stdout:4/797: dread - d0/d7/ff4 zero size 2024-09-16T08:57:19.316 INFO:tasks.workunit.client.0.smithi031.stdout:9/365: dwrite d3/d5/da/db/d21/d67/f7b [0,4194304] 0 2024-09-16T08:57:19.316 INFO:tasks.workunit.client.0.smithi031.stdout:9/366: write d3/d5/da/db/f39 [35488,37055] 0 2024-09-16T08:57:19.316 INFO:tasks.workunit.client.1.smithi097.stdout:3/876: dwrite d7/de/d3c/d3f/d60/d6b/fa3 [0,4194304] 0 2024-09-16T08:57:19.316 INFO:tasks.workunit.client.1.smithi097.stdout:3/877: creat d7/d15/d7d/f12a x:0 0 0 2024-09-16T08:57:19.317 INFO:tasks.workunit.client.1.smithi097.stdout:3/878: dread d7/de/d3c/d3f/d60/d6b/f86 [0,4194304] 0 2024-09-16T08:57:19.317 INFO:tasks.workunit.client.1.smithi097.stdout:3/879: read d7/fa [3884224,121571] 0 2024-09-16T08:57:19.317 INFO:tasks.workunit.client.1.smithi097.stdout:3/880: chown d7/de/d3c/d3f/d60 917 1 2024-09-16T08:57:19.361 INFO:tasks.workunit.client.0.smithi031.stdout:6/511: truncate d0/dd/d74/d5e/d7c/d56/f78 2775535 0 2024-09-16T08:57:19.362 INFO:tasks.workunit.client.0.smithi031.stdout:6/512: readlink d0/dd/d74/l4f 0 2024-09-16T08:57:19.363 INFO:tasks.workunit.client.1.smithi097.stdout:0/900: creat da/d7e/dc4/dfc/d47/f13b x:0 0 0 2024-09-16T08:57:19.364 INFO:tasks.workunit.client.0.smithi031.stdout:1/375: creat d2/d15/d32/f88 x:0 0 0 2024-09-16T08:57:19.365 INFO:tasks.workunit.client.1.smithi097.stdout:2/882: unlink d3/d4/d68/d8b/d7c/d7d/d53/da3/fe3 0 2024-09-16T08:57:19.367 INFO:tasks.workunit.client.0.smithi031.stdout:9/367: unlink d3/d5/da/l60 0 2024-09-16T08:57:19.373 INFO:tasks.workunit.client.0.smithi031.stdout:1/376: creat d2/d4/d67/f89 x:0 0 0 2024-09-16T08:57:19.377 INFO:tasks.workunit.client.0.smithi031.stdout:9/368: symlink d3/d5/da/d34/d3c/l7f 0 2024-09-16T08:57:19.377 INFO:tasks.workunit.client.0.smithi031.stdout:9/369: chown d3/d5/da/db/d21/d67 3 1 2024-09-16T08:57:19.379 INFO:tasks.workunit.client.0.smithi031.stdout:1/377: write d2/de/f2f [2383586,29336] 0 2024-09-16T08:57:19.379 INFO:tasks.workunit.client.0.smithi031.stdout:9/370: rmdir d3/d5/da/d34/d3c 39 2024-09-16T08:57:19.381 INFO:tasks.workunit.client.1.smithi097.stdout:0/901: creat da/d7e/dc4/dfc/d94/dd5/f13c x:0 0 0 2024-09-16T08:57:19.384 INFO:tasks.workunit.client.0.smithi031.stdout:1/378: dread d2/d4/d1b/f2b [0,4194304] 0 2024-09-16T08:57:19.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/379: fdatasync d2/d4/d1b/f3b 0 2024-09-16T08:57:19.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/380: mknod d2/d4/c8a 0 2024-09-16T08:57:19.385 INFO:tasks.workunit.client.1.smithi097.stdout:1/939: sync 2024-09-16T08:57:19.385 INFO:tasks.workunit.client.1.smithi097.stdout:1/940: readlink d5/db/dc/d19/l9e 0 2024-09-16T08:57:19.387 INFO:tasks.workunit.client.1.smithi097.stdout:0/902: dread f7 [0,4194304] 0 2024-09-16T08:57:19.387 INFO:tasks.workunit.client.1.smithi097.stdout:1/941: truncate d5/d17/d54/d6b/d71/f76 2272287 0 2024-09-16T08:57:19.389 INFO:tasks.workunit.client.1.smithi097.stdout:2/883: dwrite d3/d4/d9/d67/fd0 [0,4194304] 0 2024-09-16T08:57:19.389 INFO:tasks.workunit.client.1.smithi097.stdout:0/903: rename da/df/d8a/dac/l12c to da/d24/d4c/d80/d42/l13d 0 2024-09-16T08:57:19.390 INFO:tasks.workunit.client.1.smithi097.stdout:0/904: stat l3 0 2024-09-16T08:57:19.390 INFO:tasks.workunit.client.1.smithi097.stdout:0/905: truncate da/d1a/f49 5187730 0 2024-09-16T08:57:19.390 INFO:tasks.workunit.client.1.smithi097.stdout:0/906: readlink da/df/d8a/l108 0 2024-09-16T08:57:19.390 INFO:tasks.workunit.client.0.smithi031.stdout:6/513: dwrite d0/dd/d74/f42 [0,4194304] 0 2024-09-16T08:57:19.391 INFO:tasks.workunit.client.1.smithi097.stdout:2/884: mknod d3/d4/d68/d8b/d61/dac/dd9/de6/c111 0 2024-09-16T08:57:19.391 INFO:tasks.workunit.client.0.smithi031.stdout:1/381: dread d2/f29 [0,4194304] 0 2024-09-16T08:57:19.394 INFO:tasks.workunit.client.1.smithi097.stdout:0/907: link da/d24/d4c/d80/d3b/f6a da/d7e/dc4/dfc/d47/db3/dbf/f13e 0 2024-09-16T08:57:19.395 INFO:tasks.workunit.client.1.smithi097.stdout:0/908: truncate da/d24/d4c/d80/f3c 4007901 0 2024-09-16T08:57:19.395 INFO:tasks.workunit.client.0.smithi031.stdout:4/370: sync 2024-09-16T08:57:19.395 INFO:tasks.workunit.client.0.smithi031.stdout:0/406: sync 2024-09-16T08:57:19.395 INFO:tasks.workunit.client.0.smithi031.stdout:4/371: write d0/d2/f3d [7613,84860] 0 2024-09-16T08:57:19.397 INFO:tasks.workunit.client.1.smithi097.stdout:2/885: creat d3/d1b/f112 x:0 0 0 2024-09-16T08:57:19.399 INFO:tasks.workunit.client.0.smithi031.stdout:9/371: dwrite d3/d5/da/f33 [0,4194304] 0 2024-09-16T08:57:19.400 INFO:tasks.workunit.client.1.smithi097.stdout:1/942: rename d5/db/dc/d19/lec to d5/d17/d54/d6b/da8/dc5/dc7/df3/l12f 0 2024-09-16T08:57:19.401 INFO:tasks.workunit.client.0.smithi031.stdout:0/407: creat d6/d12/d13/f84 x:0 0 0 2024-09-16T08:57:19.403 INFO:tasks.workunit.client.1.smithi097.stdout:0/909: mkdir da/d13f 0 2024-09-16T08:57:19.405 INFO:tasks.workunit.client.1.smithi097.stdout:2/886: mknod d3/d4/d68/d8b/d7c/c113 0 2024-09-16T08:57:19.406 INFO:tasks.workunit.client.1.smithi097.stdout:1/943: truncate d5/f12 1957029 0 2024-09-16T08:57:19.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/514: rmdir d0 39 2024-09-16T08:57:19.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/515: getdents d0/dd/d74/d5e/d7c/d3f/d65 0 2024-09-16T08:57:19.415 INFO:tasks.workunit.client.0.smithi031.stdout:1/382: rmdir d2/d15/d32 39 2024-09-16T08:57:19.417 INFO:tasks.workunit.client.1.smithi097.stdout:1/944: mknod d5/d14/c130 0 2024-09-16T08:57:19.419 INFO:tasks.workunit.client.1.smithi097.stdout:3/881: fsync d7/d15/d7d/f12a 0 2024-09-16T08:57:19.421 INFO:tasks.workunit.client.0.smithi031.stdout:0/408: link d6/c10 d6/d12/d13/d3c/c85 0 2024-09-16T08:57:19.423 INFO:tasks.workunit.client.0.smithi031.stdout:6/516: unlink d0/dd/d74/f42 0 2024-09-16T08:57:19.423 INFO:tasks.workunit.client.1.smithi097.stdout:3/882: dread d7/de/f23 [0,4194304] 0 2024-09-16T08:57:19.423 INFO:tasks.workunit.client.1.smithi097.stdout:3/883: dread - d7/de/d8d/d90/f126 zero size 2024-09-16T08:57:19.424 INFO:tasks.workunit.client.0.smithi031.stdout:0/409: unlink d6/l3d 0 2024-09-16T08:57:19.424 INFO:tasks.workunit.client.0.smithi031.stdout:0/410: fsync d6/d12/f6a 0 2024-09-16T08:57:19.425 INFO:tasks.workunit.client.0.smithi031.stdout:9/372: dwrite d3/d5/da/d34/f5c [0,4194304] 0 2024-09-16T08:57:19.425 INFO:tasks.workunit.client.1.smithi097.stdout:2/887: dwrite d3/d4/d8/d51/f63 [0,4194304] 0 2024-09-16T08:57:19.425 INFO:tasks.workunit.client.1.smithi097.stdout:2/888: chown d3/d4/d68/d8b/d7c/d7d/d53/f50 140794 1 2024-09-16T08:57:19.426 INFO:tasks.workunit.client.0.smithi031.stdout:6/517: symlink d0/la5 0 2024-09-16T08:57:19.427 INFO:tasks.workunit.client.1.smithi097.stdout:3/884: symlink d7/d15/d7d/l12b 0 2024-09-16T08:57:19.428 INFO:tasks.workunit.client.0.smithi031.stdout:0/411: mknod d6/d12/d47/d7c/c86 0 2024-09-16T08:57:19.434 INFO:tasks.workunit.client.1.smithi097.stdout:2/889: mkdir d3/d4/d68/d8b/d7c/d7d/d53/d114 0 2024-09-16T08:57:19.435 INFO:tasks.workunit.client.0.smithi031.stdout:6/518: mkdir d0/dd/d74/da6 0 2024-09-16T08:57:19.437 INFO:tasks.workunit.client.1.smithi097.stdout:3/885: mknod d7/de/d3c/d4f/c12c 0 2024-09-16T08:57:19.437 INFO:tasks.workunit.client.0.smithi031.stdout:1/383: dwrite d2/f3c [0,4194304] 0 2024-09-16T08:57:19.437 INFO:tasks.workunit.client.0.smithi031.stdout:1/384: creat d2/d4/d36/f8b x:0 0 0 2024-09-16T08:57:19.438 INFO:tasks.workunit.client.0.smithi031.stdout:0/412: getdents d6/d12/d13/d36 0 2024-09-16T08:57:19.439 INFO:tasks.workunit.client.0.smithi031.stdout:9/373: dread d3/d5/da/db/d21/d67/f7b [0,4194304] 0 2024-09-16T08:57:19.439 INFO:tasks.workunit.client.0.smithi031.stdout:9/374: fsync d3/d48/f49 0 2024-09-16T08:57:19.441 INFO:tasks.workunit.client.1.smithi097.stdout:1/945: dwrite d5/db/dc/d11d/fd8 [0,4194304] 0 2024-09-16T08:57:19.442 INFO:tasks.workunit.client.1.smithi097.stdout:2/890: rename d3/d4/d68/f72 to d3/d4/d9/d78/f115 0 2024-09-16T08:57:19.445 INFO:tasks.workunit.client.1.smithi097.stdout:3/886: creat d7/d15/dee/d11f/d63/f12d x:0 0 0 2024-09-16T08:57:19.447 INFO:tasks.workunit.client.0.smithi031.stdout:1/385: mkdir d2/d4/d67/d8c 0 2024-09-16T08:57:19.450 INFO:tasks.workunit.client.0.smithi031.stdout:1/386: symlink d2/de/d6c/l8d 0 2024-09-16T08:57:19.451 INFO:tasks.workunit.client.1.smithi097.stdout:1/946: mkdir d5/d70/d9d/d12c/d131 0 2024-09-16T08:57:19.453 INFO:tasks.workunit.client.1.smithi097.stdout:2/891: link d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4/cf1 d3/d4/d68/c116 0 2024-09-16T08:57:19.453 INFO:tasks.workunit.client.0.smithi031.stdout:0/413: getdents d6/d12/d13/d3c 0 2024-09-16T08:57:19.455 INFO:tasks.workunit.client.1.smithi097.stdout:3/887: mknod d7/de/d3c/d3f/d60/d7a/c12e 0 2024-09-16T08:57:19.463 INFO:tasks.workunit.client.0.smithi031.stdout:9/375: dwrite d3/d5/da/d34/f77 [0,4194304] 0 2024-09-16T08:57:19.464 INFO:tasks.workunit.client.0.smithi031.stdout:6/519: dwrite d0/ff [0,4194304] 0 2024-09-16T08:57:19.464 INFO:tasks.workunit.client.0.smithi031.stdout:6/520: dread - d0/f7f zero size 2024-09-16T08:57:19.465 INFO:tasks.workunit.client.1.smithi097.stdout:2/892: write d3/d4/d77/ff5 [2976332,92323] 0 2024-09-16T08:57:19.466 INFO:tasks.workunit.client.0.smithi031.stdout:9/376: mkdir d3/d5/da/db/d21/d3b/d80 0 2024-09-16T08:57:19.467 INFO:tasks.workunit.client.1.smithi097.stdout:1/947: dread d5/d14/f122 [0,4194304] 0 2024-09-16T08:57:19.467 INFO:tasks.workunit.client.0.smithi031.stdout:1/387: dread d2/f17 [0,4194304] 0 2024-09-16T08:57:19.467 INFO:tasks.workunit.client.0.smithi031.stdout:1/388: chown d2/d4/d36/c4f 28 1 2024-09-16T08:57:19.467 INFO:tasks.workunit.client.0.smithi031.stdout:1/389: creat d2/d4/d1b/d5c/f8e x:0 0 0 2024-09-16T08:57:19.468 INFO:tasks.workunit.client.1.smithi097.stdout:1/948: chown d5/db/dc/d19/c2b 4015431 1 2024-09-16T08:57:19.469 INFO:tasks.workunit.client.0.smithi031.stdout:1/390: getdents d2/d15 0 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:8/799: sync 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:9/838: sync 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:6/889: sync 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:5/937: sync 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:6/890: getdents d1/dd8/d110 0 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:6/891: write d1/dd3/d9e/dca/d16/d86/fff [170327,31047] 0 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:4/798: sync 2024-09-16T08:57:19.473 INFO:tasks.workunit.client.1.smithi097.stdout:7/799: sync 2024-09-16T08:57:19.477 INFO:tasks.workunit.client.0.smithi031.stdout:9/377: dread d3/d5/da/d34/f35 [0,4194304] 0 2024-09-16T08:57:19.482 INFO:tasks.workunit.client.0.smithi031.stdout:0/414: dwrite d6/f82 [0,4194304] 0 2024-09-16T08:57:19.509 INFO:tasks.workunit.client.1.smithi097.stdout:5/938: dwrite db/d18/d1c/d42/d46/d7e/d8f/fc2 [0,4194304] 0 2024-09-16T08:57:19.510 INFO:tasks.workunit.client.0.smithi031.stdout:1/391: dwrite d2/d4/d1b/d5c/f66 [0,4194304] 0 2024-09-16T08:57:19.510 INFO:tasks.workunit.client.0.smithi031.stdout:1/392: write d2/d15/d38/f76 [950926,86639] 0 2024-09-16T08:57:19.510 INFO:tasks.workunit.client.0.smithi031.stdout:9/378: dwrite d3/d5/da/f2c [0,4194304] 0 2024-09-16T08:57:19.510 INFO:tasks.workunit.client.1.smithi097.stdout:2/893: dwrite d3/d1b/f97 [0,4194304] 0 2024-09-16T08:57:19.510 INFO:tasks.workunit.client.1.smithi097.stdout:2/894: truncate d3/d4/d9/d78/dcc/f109 143632 0 2024-09-16T08:57:19.511 INFO:tasks.workunit.client.1.smithi097.stdout:2/895: truncate d3/d4/d9/d78/f115 1814979 0 2024-09-16T08:57:19.511 INFO:tasks.workunit.client.1.smithi097.stdout:4/799: dwrite d0/d7/df/d15/f114 [0,4194304] 0 2024-09-16T08:57:19.515 INFO:tasks.workunit.client.1.smithi097.stdout:8/800: rmdir d8/dd/d15/d2c/d4a 39 2024-09-16T08:57:19.515 INFO:tasks.workunit.client.1.smithi097.stdout:8/801: readlink d8/dd/d15/d16/l88 0 2024-09-16T08:57:19.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/415: link d6/d12/l2b d6/d12/d47/d5b/l87 0 2024-09-16T08:57:19.516 INFO:tasks.workunit.client.0.smithi031.stdout:0/416: write d6/d12/f6a [154072,29275] 0 2024-09-16T08:57:19.517 INFO:tasks.workunit.client.1.smithi097.stdout:9/839: unlink d2/d5/db0/dc8/l49 0 2024-09-16T08:57:19.517 INFO:tasks.workunit.client.1.smithi097.stdout:9/840: dread - d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/fbe zero size 2024-09-16T08:57:19.518 INFO:tasks.workunit.client.1.smithi097.stdout:9/841: chown d2/d5/f21 458193010 1 2024-09-16T08:57:19.519 INFO:tasks.workunit.client.1.smithi097.stdout:6/892: mknod d1/dd3/d9e/dca/d81/c12c 0 2024-09-16T08:57:19.519 INFO:tasks.workunit.client.1.smithi097.stdout:6/893: dread - d1/dd3/d9e/dca/d81/f108 zero size 2024-09-16T08:57:19.519 INFO:tasks.workunit.client.1.smithi097.stdout:6/894: creat d1/dd3/d9e/dca/d81/dc4/dc6/f12d x:0 0 0 2024-09-16T08:57:19.521 INFO:tasks.workunit.client.0.smithi031.stdout:1/393: symlink d2/d15/d32/d87/l8f 0 2024-09-16T08:57:19.522 INFO:tasks.workunit.client.1.smithi097.stdout:4/800: dread d0/d7/df/f37 [0,4194304] 0 2024-09-16T08:57:19.524 INFO:tasks.workunit.client.1.smithi097.stdout:5/939: getdents db/d6e 0 2024-09-16T08:57:19.524 INFO:tasks.workunit.client.1.smithi097.stdout:5/940: chown db/d18/d20/d2b/f7c 169009 1 2024-09-16T08:57:19.526 INFO:tasks.workunit.client.1.smithi097.stdout:2/896: truncate d3/d4/d9/fe9 328164 0 2024-09-16T08:57:19.529 INFO:tasks.workunit.client.0.smithi031.stdout:1/394: dread d2/d15/d32/f55 [0,4194304] 0 2024-09-16T08:57:19.530 INFO:tasks.workunit.client.0.smithi031.stdout:0/417: dread d6/d12/d13/d3c/f45 [0,4194304] 0 2024-09-16T08:57:19.531 INFO:tasks.workunit.client.0.smithi031.stdout:9/379: mkdir d3/d5/d73/d81 0 2024-09-16T08:57:19.533 INFO:tasks.workunit.client.1.smithi097.stdout:8/802: creat d8/dd/ffd x:0 0 0 2024-09-16T08:57:19.535 INFO:tasks.workunit.client.0.smithi031.stdout:1/395: mknod d2/d15/c90 0 2024-09-16T08:57:19.537 INFO:tasks.workunit.client.1.smithi097.stdout:9/842: mkdir d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/d108 0 2024-09-16T08:57:19.537 INFO:tasks.workunit.client.1.smithi097.stdout:9/843: chown d2/d5/f69 23 1 2024-09-16T08:57:19.537 INFO:tasks.workunit.client.1.smithi097.stdout:2/897: write d3/d1b/da7/fdf [3357533,42304] 0 2024-09-16T08:57:19.537 INFO:tasks.workunit.client.1.smithi097.stdout:2/898: chown d3/d4/d9/d67 91116 1 2024-09-16T08:57:19.540 INFO:tasks.workunit.client.0.smithi031.stdout:9/380: mkdir d3/d82 0 2024-09-16T08:57:19.540 INFO:tasks.workunit.client.0.smithi031.stdout:9/381: fsync d3/d5/da/f2b 0 2024-09-16T08:57:19.541 INFO:tasks.workunit.client.1.smithi097.stdout:6/895: symlink d1/dd3/d9e/dca/l12e 0 2024-09-16T08:57:19.542 INFO:tasks.workunit.client.0.smithi031.stdout:1/396: rename d2/d4/l80 to d2/d4/d1b/l91 0 2024-09-16T08:57:19.543 INFO:tasks.workunit.client.1.smithi097.stdout:7/800: getdents d4/d9/d1c/d5f 0 2024-09-16T08:57:19.543 INFO:tasks.workunit.client.1.smithi097.stdout:7/801: write d4/d7/d7b/d43/daf/ffd [68749,45689] 0 2024-09-16T08:57:19.544 INFO:tasks.workunit.client.0.smithi031.stdout:0/418: mknod d6/d12/d13/c88 0 2024-09-16T08:57:19.544 INFO:tasks.workunit.client.0.smithi031.stdout:0/419: write d6/d12/d13/f84 [177635,105467] 0 2024-09-16T08:57:19.545 INFO:tasks.workunit.client.1.smithi097.stdout:5/941: symlink db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/l135 0 2024-09-16T08:57:19.545 INFO:tasks.workunit.client.1.smithi097.stdout:5/942: dread - db/d18/d1c/d42/d46/d7e/d64/fbd zero size 2024-09-16T08:57:19.545 INFO:tasks.workunit.client.1.smithi097.stdout:5/943: chown db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/ca0 53 1 2024-09-16T08:57:19.546 INFO:tasks.workunit.client.0.smithi031.stdout:9/382: link d3/d5/da/f4f d3/d82/f83 0 2024-09-16T08:57:19.546 INFO:tasks.workunit.client.0.smithi031.stdout:9/383: write d3/d5/da/f2b [3864714,43243] 0 2024-09-16T08:57:19.547 INFO:tasks.workunit.client.0.smithi031.stdout:0/420: stat d6/d12/d13/c14 0 2024-09-16T08:57:19.548 INFO:tasks.workunit.client.1.smithi097.stdout:4/801: dwrite d0/f3e [0,4194304] 0 2024-09-16T08:57:19.549 INFO:tasks.workunit.client.1.smithi097.stdout:7/802: dread d4/d9/d1c/f28 [0,4194304] 0 2024-09-16T08:57:19.549 INFO:tasks.workunit.client.1.smithi097.stdout:9/844: rename d2/d5/db0/dc8/d74/d8c/d94/d96/l98 to d2/d5/db0/dc8/d63/da8/l109 0 2024-09-16T08:57:19.550 INFO:tasks.workunit.client.1.smithi097.stdout:1/949: dwrite d5/d17/d54/d6b/d71/f76 [0,4194304] 0 2024-09-16T08:57:19.550 INFO:tasks.workunit.client.0.smithi031.stdout:1/397: fsync d2/d15/d38/f76 0 2024-09-16T08:57:19.551 INFO:tasks.workunit.client.0.smithi031.stdout:7/469: sync 2024-09-16T08:57:19.551 INFO:tasks.workunit.client.0.smithi031.stdout:7/470: write d2/d2d/f43 [482424,69574] 0 2024-09-16T08:57:19.552 INFO:tasks.workunit.client.1.smithi097.stdout:2/899: symlink d3/d1b/l117 0 2024-09-16T08:57:19.555 INFO:tasks.workunit.client.0.smithi031.stdout:9/384: rename d3/d82/f83 to d3/d5/da/d34/d3c/d5e/d62/f84 0 2024-09-16T08:57:19.557 INFO:tasks.workunit.client.1.smithi097.stdout:6/896: unlink d1/dd3/d9e/dca/f8d 0 2024-09-16T08:57:19.557 INFO:tasks.workunit.client.1.smithi097.stdout:6/897: chown d1/dd3 56 1 2024-09-16T08:57:19.557 INFO:tasks.workunit.client.1.smithi097.stdout:6/898: fdatasync d1/dd3/d9e/dca/d16/f21 0 2024-09-16T08:57:19.557 INFO:tasks.workunit.client.0.smithi031.stdout:0/421: symlink d6/d12/d13/d3c/l89 0 2024-09-16T08:57:19.557 INFO:tasks.workunit.client.0.smithi031.stdout:0/422: chown d6/c10 8343 1 2024-09-16T08:57:19.562 INFO:tasks.workunit.client.0.smithi031.stdout:0/423: read d6/d12/d13/d3c/f45 [1517796,92312] 0 2024-09-16T08:57:19.563 INFO:tasks.workunit.client.1.smithi097.stdout:5/944: creat db/d18/d1c/f136 x:0 0 0 2024-09-16T08:57:19.563 INFO:tasks.workunit.client.1.smithi097.stdout:5/945: read - db/d1a/d6d/dce/f106 zero size 2024-09-16T08:57:19.565 INFO:tasks.workunit.client.1.smithi097.stdout:8/803: getdents d8/dd/d42 0 2024-09-16T08:57:19.565 INFO:tasks.workunit.client.1.smithi097.stdout:8/804: chown d8/dd/d71/d9c/dac/cfb 2 1 2024-09-16T08:57:19.569 INFO:tasks.workunit.client.1.smithi097.stdout:4/802: link d0/d35/d6c/fd9 d0/d23/ddc/f115 0 2024-09-16T08:57:19.570 INFO:tasks.workunit.client.0.smithi031.stdout:0/424: creat d6/d12/d13/d36/f8a x:0 0 0 2024-09-16T08:57:19.570 INFO:tasks.workunit.client.0.smithi031.stdout:0/425: truncate d6/d12/d13/f39 897279 0 2024-09-16T08:57:19.570 INFO:tasks.workunit.client.0.smithi031.stdout:0/426: stat d6/d12/d47/d7c 0 2024-09-16T08:57:19.571 INFO:tasks.workunit.client.1.smithi097.stdout:7/803: mknod d4/d9/d51/db7/df9/c110 0 2024-09-16T08:57:19.572 INFO:tasks.workunit.client.1.smithi097.stdout:9/845: truncate d2/d5/d56/f5d 1393814 0 2024-09-16T08:57:19.572 INFO:tasks.workunit.client.1.smithi097.stdout:9/846: write d2/d5/f29 [358291,63270] 0 2024-09-16T08:57:19.572 INFO:tasks.workunit.client.1.smithi097.stdout:9/847: chown d2/d5/d72/da2/cbf 4154805 1 2024-09-16T08:57:19.573 INFO:tasks.workunit.client.0.smithi031.stdout:1/398: dwrite d2/f14 [4194304,4194304] 0 2024-09-16T08:57:19.573 INFO:tasks.workunit.client.0.smithi031.stdout:1/399: creat d2/d4/d36/f92 x:0 0 0 2024-09-16T08:57:19.573 INFO:tasks.workunit.client.0.smithi031.stdout:1/400: write d2/d4/d36/f8b [532411,119184] 0 2024-09-16T08:57:19.573 INFO:tasks.workunit.client.0.smithi031.stdout:9/385: dwrite d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:19.575 INFO:tasks.workunit.client.0.smithi031.stdout:1/401: rmdir d2/d15/d5d 39 2024-09-16T08:57:19.578 INFO:tasks.workunit.client.0.smithi031.stdout:9/386: mkdir d3/d85 0 2024-09-16T08:57:19.578 INFO:tasks.workunit.client.0.smithi031.stdout:9/387: write d3/d5/da/f6b [76946,118773] 0 2024-09-16T08:57:19.580 INFO:tasks.workunit.client.0.smithi031.stdout:1/402: truncate d2/d4/f54 340090 0 2024-09-16T08:57:19.581 INFO:tasks.workunit.client.1.smithi097.stdout:1/950: rmdir d5/d17/d54/d6b/da8/dc5/dc7 39 2024-09-16T08:57:19.582 INFO:tasks.workunit.client.1.smithi097.stdout:9/848: dread d2/f50 [0,4194304] 0 2024-09-16T08:57:19.583 INFO:tasks.workunit.client.0.smithi031.stdout:9/388: creat d3/d5/d73/f86 x:0 0 0 2024-09-16T08:57:19.584 INFO:tasks.workunit.client.1.smithi097.stdout:2/900: symlink d3/d4/d8/d51/l118 0 2024-09-16T08:57:19.589 INFO:tasks.workunit.client.1.smithi097.stdout:4/803: rmdir d0/d7/df/d83/df8/dbc/d108 0 2024-09-16T08:57:19.589 INFO:tasks.workunit.client.1.smithi097.stdout:4/804: write d0/d7/df/d15/f63 [711189,33192] 0 2024-09-16T08:57:19.589 INFO:tasks.workunit.client.1.smithi097.stdout:4/805: creat d0/d7/d52/d26/d8c/db0/f116 x:0 0 0 2024-09-16T08:57:19.591 INFO:tasks.workunit.client.1.smithi097.stdout:7/804: rename d4/d7/d77/ce9 to d4/d7/d77/c111 0 2024-09-16T08:57:19.592 INFO:tasks.workunit.client.1.smithi097.stdout:5/946: dwrite db/d18/d20/fc9 [0,4194304] 0 2024-09-16T08:57:19.592 INFO:tasks.workunit.client.1.smithi097.stdout:5/947: write db/d18/d1c/fad [355222,100406] 0 2024-09-16T08:57:19.593 INFO:tasks.workunit.client.1.smithi097.stdout:8/805: dwrite d8/dd/d15/d2c/d48/d56/f6d [0,4194304] 0 2024-09-16T08:57:19.595 INFO:tasks.workunit.client.1.smithi097.stdout:1/951: stat d5/d14/c44 0 2024-09-16T08:57:19.595 INFO:tasks.workunit.client.1.smithi097.stdout:1/952: read - d5/d17/d54/d6b/da8/dc5/dc7/df3/f105 zero size 2024-09-16T08:57:19.595 INFO:tasks.workunit.client.1.smithi097.stdout:1/953: truncate d5/f35 200457 0 2024-09-16T08:57:19.596 INFO:tasks.workunit.client.1.smithi097.stdout:9/849: creat d2/d5/d72/dc1/f10a x:0 0 0 2024-09-16T08:57:19.597 INFO:tasks.workunit.client.1.smithi097.stdout:9/850: dread d2/d5/d3a/d3c/f85 [0,4194304] 0 2024-09-16T08:57:19.598 INFO:tasks.workunit.client.1.smithi097.stdout:9/851: write d2/d3f/fd4 [1711286,35461] 0 2024-09-16T08:57:19.598 INFO:tasks.workunit.client.1.smithi097.stdout:9/852: chown d2/d5/fbd 0 1 2024-09-16T08:57:19.598 INFO:tasks.workunit.client.1.smithi097.stdout:4/806: rename d0/d7/d52/d26/d3d/f76 to d0/d7/df/d15/d1f/f117 0 2024-09-16T08:57:19.599 INFO:tasks.workunit.client.0.smithi031.stdout:1/403: symlink d2/d4/d1b/d5c/d6b/l93 0 2024-09-16T08:57:19.599 INFO:tasks.workunit.client.0.smithi031.stdout:1/404: creat d2/d4/d1b/d5c/f94 x:0 0 0 2024-09-16T08:57:19.599 INFO:tasks.workunit.client.0.smithi031.stdout:1/405: write d2/d4/d1b/d5c/f8e [868471,13760] 0 2024-09-16T08:57:19.600 INFO:tasks.workunit.client.1.smithi097.stdout:2/901: dwrite d3/f9b [4194304,4194304] 0 2024-09-16T08:57:19.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/910: sync 2024-09-16T08:57:19.600 INFO:tasks.workunit.client.1.smithi097.stdout:0/911: chown da/d1a/d41/db2/fd6 46980267 1 2024-09-16T08:57:19.601 INFO:tasks.workunit.client.1.smithi097.stdout:7/805: truncate d4/d7/d77/f9e 463295 0 2024-09-16T08:57:19.601 INFO:tasks.workunit.client.0.smithi031.stdout:9/389: symlink d3/d5/da/db/l87 0 2024-09-16T08:57:19.601 INFO:tasks.workunit.client.0.smithi031.stdout:9/390: fdatasync d3/d5/da/db/d21/f41 0 2024-09-16T08:57:19.605 INFO:tasks.workunit.client.0.smithi031.stdout:1/406: mkdir d2/d4/d67/d95 0 2024-09-16T08:57:19.606 INFO:tasks.workunit.client.1.smithi097.stdout:5/948: symlink db/d22/d58/l137 0 2024-09-16T08:57:19.606 INFO:tasks.workunit.client.1.smithi097.stdout:5/949: creat db/d6e/d79/f138 x:0 0 0 2024-09-16T08:57:19.606 INFO:tasks.workunit.client.1.smithi097.stdout:5/950: creat db/d6e/d81/f139 x:0 0 0 2024-09-16T08:57:19.607 INFO:tasks.workunit.client.0.smithi031.stdout:9/391: mknod d3/d5/d73/d81/c88 0 2024-09-16T08:57:19.607 INFO:tasks.workunit.client.1.smithi097.stdout:1/954: getdents d5/d70/d9d 0 2024-09-16T08:57:19.610 INFO:tasks.workunit.client.1.smithi097.stdout:9/853: creat d2/d5/f10b x:0 0 0 2024-09-16T08:57:19.610 INFO:tasks.workunit.client.1.smithi097.stdout:9/854: chown d2/c24 4031125 1 2024-09-16T08:57:19.610 INFO:tasks.workunit.client.1.smithi097.stdout:9/855: write d2/d5/d3a/d3c/d73/f7a [3708747,91935] 0 2024-09-16T08:57:19.610 INFO:tasks.workunit.client.1.smithi097.stdout:9/856: stat d2/d5/db0/dc8/d63 0 2024-09-16T08:57:19.610 INFO:tasks.workunit.client.1.smithi097.stdout:9/857: truncate d2/d5/db0/dc8/d74/d8c/d94/d96/fe7 13260 0 2024-09-16T08:57:19.610 INFO:tasks.workunit.client.1.smithi097.stdout:9/858: creat d2/d5/d72/dc1/de6/f10c x:0 0 0 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:1/407: creat d2/f96 x:0 0 0 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:1/408: read - d2/f2c zero size 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:3/494: sync 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:5/429: sync 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:4/372: sync 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:8/363: sync 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:2/324: sync 2024-09-16T08:57:19.611 INFO:tasks.workunit.client.0.smithi031.stdout:2/325: write d6/da/f6a [109835,93734] 0 2024-09-16T08:57:19.612 INFO:tasks.workunit.client.0.smithi031.stdout:2/326: dread - d6/da/d4b/d4c/f5f zero size 2024-09-16T08:57:19.613 INFO:tasks.workunit.client.0.smithi031.stdout:9/392: creat d3/d5/da/f89 x:0 0 0 2024-09-16T08:57:19.615 INFO:tasks.workunit.client.0.smithi031.stdout:2/327: write d6/da/f27 [562398,85699] 0 2024-09-16T08:57:19.615 INFO:tasks.workunit.client.0.smithi031.stdout:2/328: write d6/da/f27 [624343,115908] 0 2024-09-16T08:57:19.615 INFO:tasks.workunit.client.1.smithi097.stdout:2/902: rename d3/d4/d9/d78/f115 to d3/d4/d68/d8b/d7c/d7d/d53/d114/f119 0 2024-09-16T08:57:19.616 INFO:tasks.workunit.client.0.smithi031.stdout:1/409: rmdir d2/d4/d67/d95 0 2024-09-16T08:57:19.618 INFO:tasks.workunit.client.0.smithi031.stdout:4/373: rmdir d0/d2/d16/d1e 39 2024-09-16T08:57:19.618 INFO:tasks.workunit.client.0.smithi031.stdout:4/374: write d0/d5b/f69 [147404,83538] 0 2024-09-16T08:57:19.618 INFO:tasks.workunit.client.1.smithi097.stdout:9/859: dread d2/d5/db0/dc8/d74/d8c/d94/f99 [4194304,4194304] 0 2024-09-16T08:57:19.620 INFO:tasks.workunit.client.0.smithi031.stdout:4/375: write d0/f38 [4275899,58163] 0 2024-09-16T08:57:19.620 INFO:tasks.workunit.client.0.smithi031.stdout:4/376: chown d0/d2/d16/f6e 672831226 1 2024-09-16T08:57:19.620 INFO:tasks.workunit.client.0.smithi031.stdout:4/377: chown d0/c15 23 1 2024-09-16T08:57:19.621 INFO:tasks.workunit.client.0.smithi031.stdout:3/495: creat d3/d14/d37/d1a/d1d/f92 x:0 0 0 2024-09-16T08:57:19.621 INFO:tasks.workunit.client.0.smithi031.stdout:3/496: write d3/d14/d37/d33/f39 [1090294,3168] 0 2024-09-16T08:57:19.622 INFO:tasks.workunit.client.0.smithi031.stdout:9/393: dread d3/d5/da/d34/f77 [0,4194304] 0 2024-09-16T08:57:19.623 INFO:tasks.workunit.client.0.smithi031.stdout:3/497: write d3/d14/d37/d1a/d1d/d3b/d3f/f62 [4899243,128562] 0 2024-09-16T08:57:19.625 INFO:tasks.workunit.client.0.smithi031.stdout:2/329: mknod d6/d2e/d40/d6c/d2c/c6e 0 2024-09-16T08:57:19.625 INFO:tasks.workunit.client.0.smithi031.stdout:2/330: chown d6/da/d37/c38 410 1 2024-09-16T08:57:19.627 INFO:tasks.workunit.client.0.smithi031.stdout:1/410: creat d2/d15/d32/f97 x:0 0 0 2024-09-16T08:57:19.631 INFO:tasks.workunit.client.0.smithi031.stdout:4/378: rename d0/d2/d16/c73 to d0/d5b/c7b 0 2024-09-16T08:57:19.632 INFO:tasks.workunit.client.1.smithi097.stdout:7/806: mkdir d4/d7/d7b/d43/daf/d112 0 2024-09-16T08:57:19.632 INFO:tasks.workunit.client.1.smithi097.stdout:7/807: chown d4/d9/dc/f16 1 1 2024-09-16T08:57:19.632 INFO:tasks.workunit.client.1.smithi097.stdout:7/808: creat d4/d7/d79/f113 x:0 0 0 2024-09-16T08:57:19.635 INFO:tasks.workunit.client.0.smithi031.stdout:9/394: link d3/d5/da/db/c25 d3/d5/d6a/c8a 0 2024-09-16T08:57:19.636 INFO:tasks.workunit.client.0.smithi031.stdout:5/430: dwrite d2/f34 [0,4194304] 0 2024-09-16T08:57:19.638 INFO:tasks.workunit.client.0.smithi031.stdout:4/379: symlink d0/d5b/l7c 0 2024-09-16T08:57:19.638 INFO:tasks.workunit.client.0.smithi031.stdout:4/380: truncate d0/d2/f41 210347 0 2024-09-16T08:57:19.638 INFO:tasks.workunit.client.0.smithi031.stdout:4/381: fsync d0/d2/d16/f30 0 2024-09-16T08:57:19.639 INFO:tasks.workunit.client.0.smithi031.stdout:4/382: chown d0/d2/d16/c1d 68919765 1 2024-09-16T08:57:19.639 INFO:tasks.workunit.client.0.smithi031.stdout:4/383: creat d0/d5b/f7d x:0 0 0 2024-09-16T08:57:19.639 INFO:tasks.workunit.client.0.smithi031.stdout:4/384: readlink d0/l45 0 2024-09-16T08:57:19.639 INFO:tasks.workunit.client.0.smithi031.stdout:5/431: truncate d2/dc/f43 3115746 0 2024-09-16T08:57:19.642 INFO:tasks.workunit.client.1.smithi097.stdout:1/955: dread d5/d17/f90 [0,4194304] 0 2024-09-16T08:57:19.642 INFO:tasks.workunit.client.1.smithi097.stdout:1/956: write d5/d17/f24 [1616745,84960] 0 2024-09-16T08:57:19.642 INFO:tasks.workunit.client.1.smithi097.stdout:1/957: fdatasync d5/d2c/dcb/f8d 0 2024-09-16T08:57:19.642 INFO:tasks.workunit.client.1.smithi097.stdout:1/958: readlink d5/d2c/dcb/l110 0 2024-09-16T08:57:19.643 INFO:tasks.workunit.client.0.smithi031.stdout:4/385: truncate d0/d2/f13 1531264 0 2024-09-16T08:57:19.643 INFO:tasks.workunit.client.0.smithi031.stdout:4/386: chown d0/c1c 862 1 2024-09-16T08:57:19.644 INFO:tasks.workunit.client.1.smithi097.stdout:0/912: rename da/f11f to da/d7e/dc4/dfc/d94/dd5/f140 0 2024-09-16T08:57:19.644 INFO:tasks.workunit.client.0.smithi031.stdout:5/432: truncate d2/dc/f2d 392176 0 2024-09-16T08:57:19.644 INFO:tasks.workunit.client.0.smithi031.stdout:5/433: readlink d2/dc/l6f 0 2024-09-16T08:57:19.645 INFO:tasks.workunit.client.1.smithi097.stdout:2/903: mkdir d3/d4/d68/d8b/d7c/d7d/d53/da3/de8/df8/d11a 0 2024-09-16T08:57:19.646 INFO:tasks.workunit.client.0.smithi031.stdout:4/387: creat d0/d2/f7e x:0 0 0 2024-09-16T08:57:19.647 INFO:tasks.workunit.client.1.smithi097.stdout:7/809: mknod d4/daa/c114 0 2024-09-16T08:57:19.648 INFO:tasks.workunit.client.0.smithi031.stdout:5/434: mknod d2/dc/c8d 0 2024-09-16T08:57:19.650 INFO:tasks.workunit.client.1.smithi097.stdout:1/959: creat d5/db/dc/d19/d125/f132 x:0 0 0 2024-09-16T08:57:19.652 INFO:tasks.workunit.client.1.smithi097.stdout:9/860: rename d2/d5/db0/dc8/d63/da8/dca/df5/dd8 to d2/d5/db0/dc8/d74/d8c/dad/d10d 0 2024-09-16T08:57:19.654 INFO:tasks.workunit.client.0.smithi031.stdout:5/435: unlink d2/dc/d25/c2c 0 2024-09-16T08:57:19.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/364: dwrite d4/f43 [0,4194304] 0 2024-09-16T08:57:19.655 INFO:tasks.workunit.client.1.smithi097.stdout:0/913: creat da/d24/d4c/d80/dab/f141 x:0 0 0 2024-09-16T08:57:19.655 INFO:tasks.workunit.client.1.smithi097.stdout:0/914: fdatasync da/d24/d4b/f52 0 2024-09-16T08:57:19.656 INFO:tasks.workunit.client.1.smithi097.stdout:1/960: mknod d5/d103/c133 0 2024-09-16T08:57:19.659 INFO:tasks.workunit.client.1.smithi097.stdout:2/904: rename d3/f9b to d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4/f11b 0 2024-09-16T08:57:19.659 INFO:tasks.workunit.client.1.smithi097.stdout:1/961: dread d5/d17/f90 [0,4194304] 0 2024-09-16T08:57:19.659 INFO:tasks.workunit.client.1.smithi097.stdout:1/962: fdatasync d5/db/dc/f41 0 2024-09-16T08:57:19.659 INFO:tasks.workunit.client.1.smithi097.stdout:1/963: chown d5/db/dc/f41 5939 1 2024-09-16T08:57:19.659 INFO:tasks.workunit.client.1.smithi097.stdout:1/964: write d5/d2c/dcb/de2/fe5 [551471,18774] 0 2024-09-16T08:57:19.659 INFO:tasks.workunit.client.0.smithi031.stdout:5/436: creat d2/dc/d25/d21/f8e x:0 0 0 2024-09-16T08:57:19.661 INFO:tasks.workunit.client.1.smithi097.stdout:9/861: rename d2/d5/c26 to d2/d5/d3a/d70/c10e 0 2024-09-16T08:57:19.662 INFO:tasks.workunit.client.1.smithi097.stdout:2/905: creat d3/d4/d77/f11c x:0 0 0 2024-09-16T08:57:19.662 INFO:tasks.workunit.client.1.smithi097.stdout:2/906: chown d3/d4/d68/d8b/d7c/d7d/d53/d47/dd4/cf1 5255 1 2024-09-16T08:57:19.662 INFO:tasks.workunit.client.0.smithi031.stdout:8/365: getdents d4/d5b/d7b 0 2024-09-16T08:57:19.663 INFO:tasks.workunit.client.1.smithi097.stdout:1/965: mkdir d5/d17/d54/d6b/d71/d94/d134 0 2024-09-16T08:57:19.664 INFO:tasks.workunit.client.0.smithi031.stdout:8/366: rename d4/d7/f1c to d4/d7/d36/f7e 0 2024-09-16T08:57:19.665 INFO:tasks.workunit.client.1.smithi097.stdout:9/862: rename d2/d3f/f4b to d2/d5/d3a/d3c/f10f 0 2024-09-16T08:57:19.665 INFO:tasks.workunit.client.1.smithi097.stdout:1/966: rmdir d5/d17/d54/d6b/d71/dd3 39 2024-09-16T08:57:19.666 INFO:tasks.workunit.client.0.smithi031.stdout:8/367: unlink d4/d7/f67 0 2024-09-16T08:57:19.666 INFO:tasks.workunit.client.0.smithi031.stdout:8/368: readlink d4/d7/l6f 0 2024-09-16T08:57:19.666 INFO:tasks.workunit.client.0.smithi031.stdout:8/369: chown d4/d7/d36/d38/l3e 366001482 1 2024-09-16T08:57:19.666 INFO:tasks.workunit.client.0.smithi031.stdout:8/370: chown d4/l12 3756 1 2024-09-16T08:57:19.667 INFO:tasks.workunit.client.1.smithi097.stdout:9/863: rename d2/d5/db0/dc8/d74/d8c/d94/d96/dac/db3 to d2/d5/db0/dc8/d74/de0/d110 0 2024-09-16T08:57:19.668 INFO:tasks.workunit.client.1.smithi097.stdout:0/915: dread da/df/f16 [0,4194304] 0 2024-09-16T08:57:19.669 INFO:tasks.workunit.client.0.smithi031.stdout:8/371: rename d4/cc to d4/d34/c7f 0 2024-09-16T08:57:19.672 INFO:tasks.workunit.client.1.smithi097.stdout:1/967: rename d5/d2c/d3b/cb2 to d5/db/dc/d11d/c135 0 2024-09-16T08:57:19.674 INFO:tasks.workunit.client.1.smithi097.stdout:9/864: mknod d2/d5/d3a/d3c/d73/c111 0 2024-09-16T08:57:19.674 INFO:tasks.workunit.client.0.smithi031.stdout:5/437: dread d2/dc/f16 [0,4194304] 0 2024-09-16T08:57:19.676 INFO:tasks.workunit.client.1.smithi097.stdout:1/968: symlink d5/d17/d54/d6b/da8/l136 0 2024-09-16T08:57:19.677 INFO:tasks.workunit.client.1.smithi097.stdout:9/865: symlink d2/d5/db0/dc8/d63/da8/dca/df5/l112 0 2024-09-16T08:57:19.678 INFO:tasks.workunit.client.1.smithi097.stdout:9/866: fsync d2/d5/fa 0 2024-09-16T08:57:19.679 INFO:tasks.workunit.client.0.smithi031.stdout:3/498: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/f74 [0,4194304] 0 2024-09-16T08:57:19.679 INFO:tasks.workunit.client.1.smithi097.stdout:1/969: creat d5/db/f137 x:0 0 0 2024-09-16T08:57:19.680 INFO:tasks.workunit.client.1.smithi097.stdout:0/916: read da/d7e/dc4/dfc/d47/db3/f137 [157854,16649] 0 2024-09-16T08:57:19.680 INFO:tasks.workunit.client.1.smithi097.stdout:9/867: rmdir d2/df8 0 2024-09-16T08:57:19.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/388: dwrite d0/d2/d3a/f49 [0,4194304] 0 2024-09-16T08:57:19.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/389: creat d0/d3c/f7f x:0 0 0 2024-09-16T08:57:19.682 INFO:tasks.workunit.client.0.smithi031.stdout:4/390: chown d0/f38 1863 1 2024-09-16T08:57:19.682 INFO:tasks.workunit.client.1.smithi097.stdout:1/970: mkdir d5/d17/d54/d6b/d71/d94/d138 0 2024-09-16T08:57:19.683 INFO:tasks.workunit.client.1.smithi097.stdout:1/971: write d5/db/dc/d11d/ff8 [2453,107976] 0 2024-09-16T08:57:19.683 INFO:tasks.workunit.client.0.smithi031.stdout:3/499: mkdir d3/d14/d93 0 2024-09-16T08:57:19.683 INFO:tasks.workunit.client.0.smithi031.stdout:3/500: truncate d3/d14/d37/d1a/f8f 1013325 0 2024-09-16T08:57:19.684 INFO:tasks.workunit.client.1.smithi097.stdout:7/810: dwrite d4/f64 [0,4194304] 0 2024-09-16T08:57:19.684 INFO:tasks.workunit.client.1.smithi097.stdout:0/917: mknod da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dad/c142 0 2024-09-16T08:57:19.684 INFO:tasks.workunit.client.1.smithi097.stdout:0/918: write da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/fc6 [655612,102755] 0 2024-09-16T08:57:19.685 INFO:tasks.workunit.client.0.smithi031.stdout:2/331: dwrite d6/d20/f66 [0,4194304] 0 2024-09-16T08:57:19.685 INFO:tasks.workunit.client.0.smithi031.stdout:1/411: dwrite d2/d4/f12 [0,4194304] 0 2024-09-16T08:57:19.685 INFO:tasks.workunit.client.0.smithi031.stdout:1/412: write d2/d4/f31 [524739,47155] 0 2024-09-16T08:57:19.686 INFO:tasks.workunit.client.1.smithi097.stdout:9/868: mknod d2/d5/db0/dc8/d74/d8c/d94/d96/dac/dfd/c113 0 2024-09-16T08:57:19.689 INFO:tasks.workunit.client.1.smithi097.stdout:1/972: mkdir d5/d17/d54/d6b/da8/dc5/dc7/df3/d139 0 2024-09-16T08:57:19.692 INFO:tasks.workunit.client.1.smithi097.stdout:7/811: mknod d4/d7/d19/c115 0 2024-09-16T08:57:19.692 INFO:tasks.workunit.client.1.smithi097.stdout:7/812: chown d4/d9/dc/f6b 7377 1 2024-09-16T08:57:19.692 INFO:tasks.workunit.client.1.smithi097.stdout:2/907: dwrite d3/d4/d68/d8b/d61/dac/fc0 [0,4194304] 0 2024-09-16T08:57:19.693 INFO:tasks.workunit.client.0.smithi031.stdout:8/372: dwrite d4/d7/d21/d37/d46/f75 [0,4194304] 0 2024-09-16T08:57:19.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/391: mknod d0/d5b/c80 0 2024-09-16T08:57:19.695 INFO:tasks.workunit.client.0.smithi031.stdout:4/392: fdatasync d0/d2/f1a 0 2024-09-16T08:57:19.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/393: truncate d0/d2/d3a/f49 4848665 0 2024-09-16T08:57:19.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/394: fdatasync d0/f38 0 2024-09-16T08:57:19.696 INFO:tasks.workunit.client.0.smithi031.stdout:4/395: chown d0/d2/d16/d1e/f32 8 1 2024-09-16T08:57:19.702 INFO:tasks.workunit.client.1.smithi097.stdout:0/919: dwrite da/d7e/dc4/dfc/d47/db3/dbf/d12b/d84/fc5 [0,4194304] 0 2024-09-16T08:57:19.704 INFO:tasks.workunit.client.1.smithi097.stdout:9/869: symlink d2/d5/d8d/db1/l114 0 2024-09-16T08:57:19.704 INFO:tasks.workunit.client.1.smithi097.stdout:9/870: write d2/d5/d8d/f93 [533229,32362] 0 2024-09-16T08:57:19.705 INFO:tasks.workunit.client.1.smithi097.stdout:0/920: dread da/d7e/dc4/dfc/d94/dd5/fd7 [0,4194304] 0 2024-09-16T08:57:19.708 INFO:tasks.workunit.client.1.smithi097.stdout:7/813: symlink d4/d1b/da4/l116 0 2024-09-16T08:57:19.708 INFO:tasks.workunit.client.1.smithi097.stdout:7/814: fsync d4/d9/d1c/fbf 0 2024-09-16T08:57:19.709 INFO:tasks.workunit.client.0.smithi031.stdout:2/332: creat d6/d2e/d40/d6c/d2c/d53/d60/f6f x:0 0 0 2024-09-16T08:57:19.709 INFO:tasks.workunit.client.0.smithi031.stdout:2/333: chown d6/da/d4b/d4c/f5f 12 1 2024-09-16T08:57:19.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/413: symlink d2/d15/d1e/l98 0 2024-09-16T08:57:19.717 INFO:tasks.workunit.client.0.smithi031.stdout:4/396: dwrite d0/f39 [0,4194304] 0 2024-09-16T08:57:19.717 INFO:tasks.workunit.client.0.smithi031.stdout:4/397: chown d0/d2/d16/c20 18319 1 2024-09-16T08:57:19.717 INFO:tasks.workunit.client.0.smithi031.stdout:4/398: truncate d0/d2/d16/d59/f76 758218 0 2024-09-16T08:57:19.718 INFO:tasks.workunit.client.1.smithi097.stdout:2/908: link d3/d4/d9/d78/dcc/f109 d3/d4/d68/d8b/d7c/d7d/d53/d114/f11d 0 2024-09-16T08:57:19.721 INFO:tasks.workunit.client.0.smithi031.stdout:8/373: symlink d4/d77/l80 0 2024-09-16T08:57:19.723 INFO:tasks.workunit.client.0.smithi031.stdout:2/334: dwrite d6/d2e/f2f [0,4194304] 0 2024-09-16T08:57:19.727 INFO:tasks.workunit.client.1.smithi097.stdout:9/871: mknod d2/d5/db0/dc8/d74/d8c/d94/c115 0 2024-09-16T08:57:19.727 INFO:tasks.workunit.client.1.smithi097.stdout:9/872: dread - d2/d5/d72/dc1/ddd/dee/fc3 zero size 2024-09-16T08:57:19.729 INFO:tasks.workunit.client.1.smithi097.stdout:0/921: creat da/d7e/dc4/dfc/d47/db3/dbf/dcc/f143 x:0 0 0 2024-09-16T08:57:19.731 INFO:tasks.workunit.client.1.smithi097.stdout:1/973: symlink d5/db/dc/d19/l13a 0 2024-09-16T08:57:19.732 INFO:tasks.workunit.client.0.smithi031.stdout:4/399: rename d0/d2/d16/d1e/f32 to d0/d71/f81 0 2024-09-16T08:57:19.732 INFO:tasks.workunit.client.0.smithi031.stdout:4/400: chown d0/d5b/c70 15150 1 2024-09-16T08:57:19.733 INFO:tasks.workunit.client.0.smithi031.stdout:1/414: dwrite d2/de/f2a [0,4194304] 0 2024-09-16T08:57:19.734 INFO:tasks.workunit.client.1.smithi097.stdout:7/815: link d4/d9/dc/dc6/cfb d4/dfa/c117 0 2024-09-16T08:57:19.735 INFO:tasks.workunit.client.1.smithi097.stdout:2/909: symlink d3/d4/d68/d8b/d7c/d7d/l11e 0 2024-09-16T08:57:19.738 INFO:tasks.workunit.client.0.smithi031.stdout:2/335: creat d6/da/d5e/f70 x:0 0 0 2024-09-16T08:57:19.739 INFO:tasks.workunit.client.0.smithi031.stdout:2/336: readlink d6/d2e/d40/d6c/d2c/l65 0 2024-09-16T08:57:19.739 INFO:tasks.workunit.client.0.smithi031.stdout:2/337: fsync d6/d2e/d40/d6c/f3b 0 2024-09-16T08:57:19.740 INFO:tasks.workunit.client.0.smithi031.stdout:8/374: rename d4/d2f/f72 to d4/d77/f81 0 2024-09-16T08:57:19.741 INFO:tasks.workunit.client.1.smithi097.stdout:9/873: dread d2/d5/fa [0,4194304] 0 2024-09-16T08:57:19.742 INFO:tasks.workunit.client.0.smithi031.stdout:4/401: truncate d0/d2/d16/d1e/f52 869614 0 2024-09-16T08:57:19.742 INFO:tasks.workunit.client.0.smithi031.stdout:4/402: stat d0/d2 0 2024-09-16T08:57:19.742 INFO:tasks.workunit.client.0.smithi031.stdout:4/403: truncate d0/d3c/f62 820202 0 2024-09-16T08:57:19.743 INFO:tasks.workunit.client.0.smithi031.stdout:4/404: fsync d0/d2/d16/f30 0 2024-09-16T08:57:19.743 INFO:tasks.workunit.client.0.smithi031.stdout:1/415: symlink d2/d15/d1f/l99 0 2024-09-16T08:57:19.743 INFO:tasks.workunit.client.1.smithi097.stdout:9/874: dread d2/d5/d3a/d3c/f85 [0,4194304] 0 2024-09-16T08:57:19.744 INFO:tasks.workunit.client.1.smithi097.stdout:0/922: mknod da/d24/d4b/c144 0 2024-09-16T08:57:19.747 INFO:tasks.workunit.client.0.smithi031.stdout:2/338: link d6/fc d6/da/d37/d42/f71 0 2024-09-16T08:57:19.747 INFO:tasks.workunit.client.0.smithi031.stdout:2/339: chown d6/da/d37/l6b 239 1 2024-09-16T08:57:19.747 INFO:tasks.workunit.client.0.smithi031.stdout:2/340: stat d6/d20/d51 0 2024-09-16T08:57:19.747 INFO:tasks.workunit.client.0.smithi031.stdout:2/341: readlink d6/da/l55 0 2024-09-16T08:57:19.748 INFO:tasks.workunit.client.1.smithi097.stdout:1/974: mkdir d5/d2c/d3b/d7f/d13b 0 2024-09-16T08:57:19.748 INFO:tasks.workunit.client.1.smithi097.stdout:1/975: truncate d5/db/dc/d11d/fac 282400 0 2024-09-16T08:57:19.748 INFO:tasks.workunit.client.1.smithi097.stdout:1/976: write d5/d70/fa3 [4096324,121468] 0 2024-09-16T08:57:19.748 INFO:tasks.workunit.client.0.smithi031.stdout:8/375: creat d4/d7/d21/f82 x:0 0 0 2024-09-16T08:57:19.750 INFO:tasks.workunit.client.0.smithi031.stdout:4/405: symlink d0/d2/d6d/l82 0 2024-09-16T08:57:19.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/816: creat d4/d12/d1f/d22/f118 x:0 0 0 2024-09-16T08:57:19.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/817: write d4/ff6 [802534,39357] 0 2024-09-16T08:57:19.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/342: mkdir d6/da/d5e/d72 0 2024-09-16T08:57:19.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/343: truncate d6/d2e/f2f 5016629 0 2024-09-16T08:57:19.756 INFO:tasks.workunit.client.1.smithi097.stdout:9/875: rename d2/d5/d3a/d70/da6 to d2/d5/d56/d116 0 2024-09-16T08:57:19.757 INFO:tasks.workunit.client.0.smithi031.stdout:4/406: mkdir d0/d2/d6d/d75/d83 0 2024-09-16T08:57:19.758 INFO:tasks.workunit.client.1.smithi097.stdout:1/977: write d5/db/dc/d19/ff1 [1166396,82931] 0 2024-09-16T08:57:19.759 INFO:tasks.workunit.client.0.smithi031.stdout:2/344: mkdir d6/da/d4b/d4c/d73 0 2024-09-16T08:57:19.760 INFO:tasks.workunit.client.1.smithi097.stdout:9/876: symlink d2/d5/db0/dc8/d63/da8/l117 0 2024-09-16T08:57:19.762 INFO:tasks.workunit.client.0.smithi031.stdout:4/407: rmdir d0/d2/d6d/d75/d83 0 2024-09-16T08:57:19.762 INFO:tasks.workunit.client.1.smithi097.stdout:2/910: dwrite d3/d4/d68/d8b/d7c/d7d/d53/d47/ff7 [0,4194304] 0 2024-09-16T08:57:19.763 INFO:tasks.workunit.client.1.smithi097.stdout:1/978: write d5/db/dc/d19/f88 [2174331,10227] 0 2024-09-16T08:57:19.764 INFO:tasks.workunit.client.1.smithi097.stdout:9/877: symlink d2/d5/dd5/l118 0 2024-09-16T08:57:19.764 INFO:tasks.workunit.client.1.smithi097.stdout:7/818: dread d4/f9d [4194304,4194304] 0 2024-09-16T08:57:19.768 INFO:tasks.workunit.client.1.smithi097.stdout:1/979: unlink d5/d2c/d3b/c81 0 2024-09-16T08:57:19.768 INFO:tasks.workunit.client.1.smithi097.stdout:1/980: fsync d5/d70/fc4 0 2024-09-16T08:57:19.768 INFO:tasks.workunit.client.1.smithi097.stdout:7/819: getdents d4/d7/d77/da8 0 2024-09-16T08:57:19.768 INFO:tasks.workunit.client.1.smithi097.stdout:7/820: rename d4 to d4/d7/dab/d119 22 2024-09-16T08:57:19.768 INFO:tasks.workunit.client.1.smithi097.stdout:0/923: dwrite da/d1a/d41/db2/ff2 [0,4194304] 0 2024-09-16T08:57:19.769 INFO:tasks.workunit.client.1.smithi097.stdout:0/924: chown da/df/d8a/fa9 419643716 1 2024-09-16T08:57:19.769 INFO:tasks.workunit.client.1.smithi097.stdout:0/925: fdatasync da/d24/f33 0 2024-09-16T08:57:19.769 INFO:tasks.workunit.client.1.smithi097.stdout:0/926: write da/d7e/dc4/dfc/d47/db3/f7b [3524589,58080] 0 2024-09-16T08:57:19.769 INFO:tasks.workunit.client.1.smithi097.stdout:1/981: chown d5/db/dc/d19/l68 6562470 1 2024-09-16T08:57:19.770 INFO:tasks.workunit.client.1.smithi097.stdout:9/878: dread d2/d5/d3a/d3c/f6d [0,4194304] 0 2024-09-16T08:57:19.770 INFO:tasks.workunit.client.0.smithi031.stdout:1/416: dwrite d2/d4/d1b/d5c/f66 [0,4194304] 0 2024-09-16T08:57:19.770 INFO:tasks.workunit.client.0.smithi031.stdout:1/417: read d2/d4/d1b/f50 [169672,6241] 0 2024-09-16T08:57:19.770 INFO:tasks.workunit.client.0.smithi031.stdout:1/418: dread - d2/d4/f82 zero size 2024-09-16T08:57:19.770 INFO:tasks.workunit.client.0.smithi031.stdout:1/419: fsync d2/d4/f28 0 2024-09-16T08:57:19.773 INFO:tasks.workunit.client.1.smithi097.stdout:0/927: unlink da/df/c30 0 2024-09-16T08:57:19.775 INFO:tasks.workunit.client.1.smithi097.stdout:9/879: link d2/d5/db0/dc8/d63/da8/l117 d2/d5/l119 0 2024-09-16T08:57:19.775 INFO:tasks.workunit.client.1.smithi097.stdout:9/880: stat d2/d5/db0/dc8/d74/d8c/feb 0 2024-09-16T08:57:19.779 INFO:tasks.workunit.client.1.smithi097.stdout:0/928: creat da/d7e/dc4/dfc/d47/db3/f145 x:0 0 0 2024-09-16T08:57:19.782 INFO:tasks.workunit.client.0.smithi031.stdout:6/521: sync 2024-09-16T08:57:19.782 INFO:tasks.workunit.client.0.smithi031.stdout:6/522: getdents d0/d8e 0 2024-09-16T08:57:19.783 INFO:tasks.workunit.client.1.smithi097.stdout:1/982: dread d5/d70/f72 [0,4194304] 0 2024-09-16T08:57:19.785 INFO:tasks.workunit.client.1.smithi097.stdout:7/821: write d4/d7/f32 [976421,125096] 0 2024-09-16T08:57:19.786 INFO:tasks.workunit.client.0.smithi031.stdout:1/420: dread d2/f3c [0,4194304] 0 2024-09-16T08:57:19.788 INFO:tasks.workunit.client.1.smithi097.stdout:9/881: write d2/f25 [883327,40242] 0 2024-09-16T08:57:19.788 INFO:tasks.workunit.client.1.smithi097.stdout:9/882: chown d2/d5/d3a/dfb 16522885 1 2024-09-16T08:57:19.789 INFO:tasks.workunit.client.1.smithi097.stdout:9/883: write d2/d5/db0/dc8/d74/d8c/d94/d96/dac/f81 [1901140,109870] 0 2024-09-16T08:57:19.789 INFO:tasks.workunit.client.1.smithi097.stdout:9/884: chown d2/d5/db0/dc8/d74/d8c/d94/d96/dac/l68 0 1 2024-09-16T08:57:19.789 INFO:tasks.workunit.client.1.smithi097.stdout:9/885: readlink d2/d5/db0/dc8/d74/d8c/d94/d96/dac/le3 0 2024-09-16T08:57:19.789 INFO:tasks.workunit.client.0.smithi031.stdout:1/421: link d2/d4/d36/f8b d2/d15/d38/f9a 0 2024-09-16T08:57:19.791 INFO:tasks.workunit.client.1.smithi097.stdout:9/886: getdents d2/d5/db0/dc8/d74/de0 0 2024-09-16T08:57:19.791 INFO:tasks.workunit.client.1.smithi097.stdout:9/887: fdatasync d2/f13 0 2024-09-16T08:57:19.795 INFO:tasks.workunit.client.1.smithi097.stdout:3/888: sync 2024-09-16T08:57:19.795 INFO:tasks.workunit.client.1.smithi097.stdout:6/899: sync 2024-09-16T08:57:19.795 INFO:tasks.workunit.client.1.smithi097.stdout:5/951: sync 2024-09-16T08:57:19.795 INFO:tasks.workunit.client.1.smithi097.stdout:4/807: sync 2024-09-16T08:57:19.796 INFO:tasks.workunit.client.1.smithi097.stdout:8/806: sync 2024-09-16T08:57:19.796 INFO:tasks.workunit.client.1.smithi097.stdout:5/952: truncate db/d6e/d79/dba/f121 51412 0 2024-09-16T08:57:19.796 INFO:tasks.workunit.client.1.smithi097.stdout:9/888: symlink d2/d5/d56/d116/l11a 0 2024-09-16T08:57:19.803 INFO:tasks.workunit.client.1.smithi097.stdout:4/808: rmdir d0/d7/df/d83/df8 39 2024-09-16T08:57:19.803 INFO:tasks.workunit.client.1.smithi097.stdout:4/809: fsync d0/d36/d4c/fd7 0 2024-09-16T08:57:19.804 INFO:tasks.workunit.client.0.smithi031.stdout:6/523: dread d0/dd/d74/d19/f34 [0,4194304] 0 2024-09-16T08:57:19.804 INFO:tasks.workunit.client.1.smithi097.stdout:5/953: mknod db/d6e/d95/c13a 0 2024-09-16T08:57:19.804 INFO:tasks.workunit.client.1.smithi097.stdout:1/983: dread d5/db/dc/d19/f3f [0,4194304] 0 2024-09-16T08:57:19.805 INFO:tasks.workunit.client.0.smithi031.stdout:2/345: dwrite d6/da/f6a [0,4194304] 0 2024-09-16T08:57:19.806 INFO:tasks.workunit.client.1.smithi097.stdout:8/807: mknod d8/dd/d15/d2c/d48/cfe 0 2024-09-16T08:57:19.807 INFO:tasks.workunit.client.0.smithi031.stdout:6/524: getdents d0/dd/d74/d19/d9a 0 2024-09-16T08:57:19.808 INFO:tasks.workunit.client.1.smithi097.stdout:1/984: creat d5/d2c/d3b/d7f/f13c x:0 0 0 2024-09-16T08:57:19.810 INFO:tasks.workunit.client.1.smithi097.stdout:9/889: dread d2/d5/d3a/d3c/d73/f7a [0,4194304] 0 2024-09-16T08:57:19.810 INFO:tasks.workunit.client.1.smithi097.stdout:9/890: dread - d2/d5/d72/dc1/f10a zero size 2024-09-16T08:57:19.810 INFO:tasks.workunit.client.1.smithi097.stdout:8/808: rename d8/dd/lce to d8/dd/d15/d2c/d4a/df3/lff 0 2024-09-16T08:57:19.810 INFO:tasks.workunit.client.1.smithi097.stdout:8/809: chown d8/dd/d71/d9c 585347103 1 2024-09-16T08:57:19.810 INFO:tasks.workunit.client.1.smithi097.stdout:8/810: chown d8/dd/d71/d9c/fca 3148453 1 2024-09-16T08:57:19.810 INFO:tasks.workunit.client.1.smithi097.stdout:8/811: write d8/dd/f19 [9353061,26611] 0 2024-09-16T08:57:19.811 INFO:tasks.workunit.client.1.smithi097.stdout:1/985: creat d5/d17/d54/d6b/da8/dc5/dc7/df3/d139/f13d x:0 0 0 2024-09-16T08:57:19.814 INFO:tasks.workunit.client.0.smithi031.stdout:2/346: creat d6/f74 x:0 0 0 2024-09-16T08:57:19.815 INFO:tasks.workunit.client.1.smithi097.stdout:2/911: dwrite d3/f23 [0,4194304] 0 2024-09-16T08:57:19.815 INFO:tasks.workunit.client.1.smithi097.stdout:2/912: creat d3/d4/d8/f11f x:0 0 0 2024-09-16T08:57:19.816 INFO:tasks.workunit.client.1.smithi097.stdout:9/891: symlink d2/d5/db0/dc8/d63/l11b 0 2024-09-16T08:57:19.816 INFO:tasks.workunit.client.0.smithi031.stdout:6/525: stat d0/dd/ce 0 2024-09-16T08:57:19.816 INFO:tasks.workunit.client.0.smithi031.stdout:6/526: chown d0/dd/c50 88 1 2024-09-16T08:57:19.817 INFO:tasks.workunit.client.0.smithi031.stdout:4/408: dwrite d0/d2/d16/d2b/f74 [0,4194304] 0 2024-09-16T08:57:19.818 INFO:tasks.workunit.client.0.smithi031.stdout:2/347: symlink d6/d20/l75 0 2024-09-16T08:57:19.818 INFO:tasks.workunit.client.1.smithi097.stdout:8/812: creat d8/dd/d15/d2c/d48/f100 x:0 0 0 2024-09-16T08:57:19.819 INFO:tasks.workunit.client.1.smithi097.stdout:6/900: read d1/dd3/d9e/dca/d16/d4f/d9f/fa4 [724220,75941] 0 2024-09-16T08:57:19.819 INFO:tasks.workunit.client.1.smithi097.stdout:1/986: rename d5/d2c/f11c to d5/d2c/d3b/d7f/d13b/f13e 0 2024-09-16T08:57:19.824 INFO:tasks.workunit.client.1.smithi097.stdout:2/913: rmdir d3/d1b/dbf 39 2024-09-16T08:57:19.825 INFO:tasks.workunit.client.1.smithi097.stdout:9/892: unlink d2/f28 0 2024-09-16T08:57:19.825 INFO:tasks.workunit.client.1.smithi097.stdout:9/893: truncate d2/d5/d8d/db1/fff 719058 0 2024-09-16T08:57:19.825 INFO:tasks.workunit.client.1.smithi097.stdout:9/894: dread - d2/d5/d72/dc1/de6/f10c zero size 2024-09-16T08:57:19.826 INFO:tasks.workunit.client.1.smithi097.stdout:8/813: stat d8/dd/d15/d2c/d4a/d4d/d3d/db9/ld2 0 2024-09-16T08:57:19.827 INFO:tasks.workunit.client.1.smithi097.stdout:6/901: symlink d1/dd3/d9e/dca/d81/dc4/l12f 0 2024-09-16T08:57:19.829 INFO:tasks.workunit.client.1.smithi097.stdout:1/987: rename d5/l8f to d5/d70/d85/l13f 0 2024-09-16T08:57:19.831 INFO:tasks.workunit.client.0.smithi031.stdout:4/409: write d0/d2/d16/d1e/f34 [824751,105996] 0 2024-09-16T08:57:19.831 INFO:tasks.workunit.client.1.smithi097.stdout:2/914: symlink d3/d4/d68/d8b/d7c/d7d/l120 0 2024-09-16T08:57:19.832 INFO:tasks.workunit.client.1.smithi097.stdout:9/895: mknod d2/d5/d3a/c11c 0 2024-09-16T08:57:19.833 INFO:tasks.workunit.client.1.smithi097.stdout:6/902: stat d1/dd3/db0/lef 0 2024-09-16T08:57:19.833 INFO:tasks.workunit.client.0.smithi031.stdout:4/410: mkdir d0/d2/d16/d59/d84 0 2024-09-16T08:57:19.836 INFO:tasks.workunit.client.1.smithi097.stdout:8/814: rename d8/dd/d15/d2c/d4a/l52 to d8/dd/d15/d2c/d4a/d4e/l101 0 2024-09-16T08:57:19.837 INFO:tasks.workunit.client.1.smithi097.stdout:8/815: stat d8/dd/d15/ca0 0 2024-09-16T08:57:19.838 INFO:tasks.workunit.client.1.smithi097.stdout:8/816: stat d8/dd/d15/d2c/d4a/d4e/ccf 0 2024-09-16T08:57:19.846 INFO:tasks.workunit.client.0.smithi031.stdout:6/527: write d0/f15 [1046509,11787] 0 2024-09-16T08:57:19.846 INFO:tasks.workunit.client.0.smithi031.stdout:6/528: fsync d0/dd/f2f 0 2024-09-16T08:57:19.852 INFO:tasks.workunit.client.0.smithi031.stdout:2/348: dread d6/f2d [0,4194304] 0 2024-09-16T08:57:19.853 INFO:tasks.workunit.client.0.smithi031.stdout:2/349: link d6/d20/c49 d6/da/d4b/d4c/d68/c76 0 2024-09-16T08:57:19.854 INFO:tasks.workunit.client.0.smithi031.stdout:2/350: symlink d6/d2e/d40/d6c/d2c/d53/d60/l77 0 2024-09-16T08:57:19.856 INFO:tasks.workunit.client.0.smithi031.stdout:2/351: creat d6/d2e/d40/f78 x:0 0 0 2024-09-16T08:57:19.856 INFO:tasks.workunit.client.0.smithi031.stdout:2/352: chown d6/d20/l75 42201867 1 2024-09-16T08:57:19.856 INFO:tasks.workunit.client.1.smithi097.stdout:6/903: read d1/f37 [3005133,16457] 0 2024-09-16T08:57:19.856 INFO:tasks.workunit.client.1.smithi097.stdout:6/904: write d1/dd3/d9e/dca/d16/d4f/ff4 [63915,7313] 0 2024-09-16T08:57:19.857 INFO:tasks.workunit.client.1.smithi097.stdout:6/905: symlink d1/dd3/d9e/dca/d16/d6a/l130 0 2024-09-16T08:57:19.858 INFO:tasks.workunit.client.1.smithi097.stdout:6/906: truncate d1/dd3/d9e/dca/d16/ff1 2264154 0 2024-09-16T08:57:19.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/411: write d0/d2/d16/d1e/f23 [1582245,126992] 0 2024-09-16T08:57:19.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/907: mknod d1/dd8/c131 0 2024-09-16T08:57:19.860 INFO:tasks.workunit.client.1.smithi097.stdout:6/908: write d1/dd3/d9e/dca/d81/dc4/fcc [305019,54033] 0 2024-09-16T08:57:19.862 INFO:tasks.workunit.client.1.smithi097.stdout:6/909: unlink d1/dd3/d9e/dca/db6/l117 0 2024-09-16T08:57:19.865 INFO:tasks.workunit.client.0.smithi031.stdout:4/412: dread d0/d2/f13 [0,4194304] 0 2024-09-16T08:57:19.868 INFO:tasks.workunit.client.1.smithi097.stdout:0/929: dwrite da/d1a/f2b [4194304,4194304] 0 2024-09-16T08:57:19.869 INFO:tasks.workunit.client.1.smithi097.stdout:0/930: write da/d7e/dc4/dfc/d47/db3/f145 [678582,122217] 0 2024-09-16T08:57:19.877 INFO:tasks.workunit.client.1.smithi097.stdout:6/910: read d1/dd3/d9e/dca/dbb/fd1 [2664979,27709] 0 2024-09-16T08:57:19.879 INFO:tasks.workunit.client.1.smithi097.stdout:6/911: mkdir d1/dc/d3c/d132 0 2024-09-16T08:57:19.880 INFO:tasks.workunit.client.1.smithi097.stdout:6/912: creat d1/dd3/d9e/f133 x:0 0 0 2024-09-16T08:57:19.883 INFO:tasks.workunit.client.1.smithi097.stdout:6/913: rename d1/dd3/d9e/dca/d16/d6a/l8e to d1/dd3/d9e/l134 0 2024-09-16T08:57:19.886 INFO:tasks.workunit.client.1.smithi097.stdout:7/822: dwrite d4/d9/d2b/f4d [4194304,4194304] 0 2024-09-16T08:57:19.887 INFO:tasks.workunit.client.1.smithi097.stdout:4/810: dwrite d0/d7/df/f37 [0,4194304] 0 2024-09-16T08:57:19.888 INFO:tasks.workunit.client.1.smithi097.stdout:5/954: dwrite db/d18/d1c/d42/d46/d7e/d8f/ff5 [0,4194304] 0 2024-09-16T08:57:19.888 INFO:tasks.workunit.client.1.smithi097.stdout:9/896: dwrite d2/d5/d56/f8a [4194304,4194304] 0 2024-09-16T08:57:19.889 INFO:tasks.workunit.client.0.smithi031.stdout:1/422: dwrite d2/de/f79 [4194304,4194304] 0 2024-09-16T08:57:19.892 INFO:tasks.workunit.client.0.smithi031.stdout:2/353: dwrite d6/d2e/f2f [0,4194304] 0 2024-09-16T08:57:19.892 INFO:tasks.workunit.client.1.smithi097.stdout:1/988: dwrite d5/d70/f72 [0,4194304] 0 2024-09-16T08:57:19.892 INFO:tasks.workunit.client.1.smithi097.stdout:8/817: dwrite d8/dd/d15/d2c/ff9 [0,4194304] 0 2024-09-16T08:57:19.895 INFO:tasks.workunit.client.1.smithi097.stdout:9/897: dread d2/d5/d72/dc1/de6/fea [0,4194304] 0 2024-09-16T08:57:19.896 INFO:tasks.workunit.client.1.smithi097.stdout:4/811: rename d0/d23/d5f/c113 to d0/d23/dda/c118 0 2024-09-16T08:57:19.896 INFO:tasks.workunit.client.1.smithi097.stdout:4/812: write d0/d7/df/f13 [861182,130484] 0 2024-09-16T08:57:19.899 INFO:tasks.workunit.client.1.smithi097.stdout:0/931: dwrite da/d7e/dc4/dfc/d47/db3/dbf/f13e [0,4194304] 0 2024-09-16T08:57:19.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/423: rmdir d2/d4/d67/d8c 0 2024-09-16T08:57:19.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/424: write d2/d15/d1f/d43/f4c [2959230,77185] 0 2024-09-16T08:57:19.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/425: stat d2/d15/d5d/f3d 0 2024-09-16T08:57:19.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/426: write d2/de/f2f [2432055,98595] 0 2024-09-16T08:57:19.902 INFO:tasks.workunit.client.0.smithi031.stdout:2/354: dread d6/da/d37/d42/f71 [0,4194304] 0 2024-09-16T08:57:19.903 INFO:tasks.workunit.client.0.smithi031.stdout:6/529: dwrite d0/dd/d74/f18 [4194304,4194304] 0 2024-09-16T08:57:19.907 INFO:tasks.workunit.client.0.smithi031.stdout:6/530: symlink d0/dd/d2b/la7 0 2024-09-16T08:57:19.909 INFO:tasks.workunit.client.1.smithi097.stdout:9/898: rmdir d2/d5/db0/dc8/d74/d8c/dad/d10d 39 2024-09-16T08:57:19.910 INFO:tasks.workunit.client.1.smithi097.stdout:9/899: creat d2/d5/d72/dc1/f11d x:0 0 0 2024-09-16T08:57:19.911 INFO:tasks.workunit.client.1.smithi097.stdout:6/914: dwrite d1/dc/d3c/f76 [0,4194304] 0 2024-09-16T08:57:19.911 INFO:tasks.workunit.client.0.smithi031.stdout:6/531: mkdir d0/dd/d74/da6/da8 0 2024-09-16T08:57:19.911 INFO:tasks.workunit.client.0.smithi031.stdout:6/532: write d0/f15 [1331280,66731] 0 2024-09-16T08:57:19.914 INFO:tasks.workunit.client.1.smithi097.stdout:5/955: rename db/d18/d1c/c30 to db/d1a/d6d/dce/d105/c13b 0 2024-09-16T08:57:19.914 INFO:tasks.workunit.client.1.smithi097.stdout:5/956: truncate db/d18/d1c/d42/d46/d7e/d76/dca/f11f 880828 0 2024-09-16T08:57:19.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/355: write d6/da/f17 [3603269,26961] 0 2024-09-16T08:57:19.918 INFO:tasks.workunit.client.0.smithi031.stdout:4/413: dwrite d0/d2/d6d/f77 [0,4194304] 0 2024-09-16T08:57:19.924 INFO:tasks.workunit.client.1.smithi097.stdout:1/989: dwrite d5/d17/d54/d6b/d71/d94/fbb [0,4194304] 0 2024-09-16T08:57:19.925 INFO:tasks.workunit.client.1.smithi097.stdout:1/990: read f4 [2785280,20534] 0 2024-09-16T08:57:19.926 INFO:tasks.workunit.client.1.smithi097.stdout:9/900: mkdir d2/d5/db0/dc8/d74/d8c/d94/d96/d11e 0 2024-09-16T08:57:19.926 INFO:tasks.workunit.client.1.smithi097.stdout:9/901: chown d2/d5/d3a/d3c/fc6 665973 1 2024-09-16T08:57:19.929 INFO:tasks.workunit.client.1.smithi097.stdout:6/915: truncate d1/dd3/d9e/dca/d16/f4d 2100461 0 2024-09-16T08:57:19.930 INFO:tasks.workunit.client.1.smithi097.stdout:0/932: dwrite da/d7e/dc4/dfc/d47/db3/fd4 [0,4194304] 0 2024-09-16T08:57:19.937 INFO:tasks.workunit.client.1.smithi097.stdout:1/991: rename d5/d17/d54/d6b/d71/cfc to d5/d70/d9d/d12c/d131/c140 0 2024-09-16T08:57:19.938 INFO:tasks.workunit.client.1.smithi097.stdout:9/902: stat d2/l15 0 2024-09-16T08:57:19.939 INFO:tasks.workunit.client.1.smithi097.stdout:1/992: dread d5/d17/f90 [0,4194304] 0 2024-09-16T08:57:19.940 INFO:tasks.workunit.client.1.smithi097.stdout:6/916: symlink d1/dd3/d9e/dca/d16/l135 0 2024-09-16T08:57:19.940 INFO:tasks.workunit.client.1.smithi097.stdout:6/917: fdatasync d1/dd3/d9e/dca/d81/dc4/dc6/ff0 0 2024-09-16T08:57:19.943 INFO:tasks.workunit.client.1.smithi097.stdout:0/933: unlink da/d1a/d41/ld9 0 2024-09-16T08:57:19.943 INFO:tasks.workunit.client.1.smithi097.stdout:0/934: fdatasync da/d24/d4c/d80/f2a 0 2024-09-16T08:57:19.944 INFO:tasks.workunit.client.1.smithi097.stdout:9/903: rmdir d2/d5/db0/dc8/d74/de0 39 2024-09-16T08:57:19.946 INFO:tasks.workunit.client.1.smithi097.stdout:1/993: rename d5/d2c/d3b/la2 to d5/d2c/d3b/d7f/l141 0 2024-09-16T08:57:19.947 INFO:tasks.workunit.client.1.smithi097.stdout:6/918: mkdir d1/dd3/d9e/d136 0 2024-09-16T08:57:19.952 INFO:tasks.workunit.client.1.smithi097.stdout:0/935: rename da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/fc6 to da/d24/d4b/f146 0 2024-09-16T08:57:19.952 INFO:tasks.workunit.client.1.smithi097.stdout:0/936: dread - da/d7e/dc4/dfc/d47/db3/dbf/dcc/f143 zero size 2024-09-16T08:57:19.954 INFO:tasks.workunit.client.1.smithi097.stdout:6/919: symlink d1/dd3/d9e/dca/d81/dc4/ddc/l137 0 2024-09-16T08:57:19.954 INFO:tasks.workunit.client.1.smithi097.stdout:6/920: chown d1/dd3/d9e/dca/d81/l103 1 1 2024-09-16T08:57:19.954 INFO:tasks.workunit.client.1.smithi097.stdout:9/904: rmdir d2/d5/d8d/db1 39 2024-09-16T08:57:19.954 INFO:tasks.workunit.client.1.smithi097.stdout:9/905: chown d2/c3 501465 1 2024-09-16T08:57:19.955 INFO:tasks.workunit.client.0.smithi031.stdout:0/427: sync 2024-09-16T08:57:19.955 INFO:tasks.workunit.client.0.smithi031.stdout:7/471: sync 2024-09-16T08:57:19.956 INFO:tasks.workunit.client.1.smithi097.stdout:6/921: unlink d1/dd3/d9e/db3/de3/l106 0 2024-09-16T08:57:19.957 INFO:tasks.workunit.client.1.smithi097.stdout:9/906: rmdir d2/d5/db0/dc8/d74/d8c/d94/d96/d11e 0 2024-09-16T08:57:19.958 INFO:tasks.workunit.client.0.smithi031.stdout:0/428: mknod d6/d12/d47/c8b 0 2024-09-16T08:57:19.960 INFO:tasks.workunit.client.1.smithi097.stdout:6/922: creat d1/dd3/d9e/d136/f138 x:0 0 0 2024-09-16T08:57:19.960 INFO:tasks.workunit.client.1.smithi097.stdout:6/923: chown d1/dd3/d9e/dca/d16/d4f/df8/c127 1627 1 2024-09-16T08:57:19.960 INFO:tasks.workunit.client.1.smithi097.stdout:5/957: dwrite db/d18/d1c/d42/d46/d7e/f51 [4194304,4194304] 0 2024-09-16T08:57:19.961 INFO:tasks.workunit.client.0.smithi031.stdout:7/472: rename d2/f92 to d2/d8/d69/d82/fa0 0 2024-09-16T08:57:19.962 INFO:tasks.workunit.client.0.smithi031.stdout:0/429: mknod d6/d12/d13/d36/c8c 0 2024-09-16T08:57:19.964 INFO:tasks.workunit.client.1.smithi097.stdout:6/924: link d1/dc/f80 d1/dd3/d9e/dca/db6/f139 0 2024-09-16T08:57:19.965 INFO:tasks.workunit.client.1.smithi097.stdout:5/958: symlink db/l13c 0 2024-09-16T08:57:19.965 INFO:tasks.workunit.client.1.smithi097.stdout:5/959: chown db/d18/d20/fe6 197 1 2024-09-16T08:57:19.965 INFO:tasks.workunit.client.1.smithi097.stdout:5/960: readlink db/d1a/d6d/l103 0 2024-09-16T08:57:19.965 INFO:tasks.workunit.client.1.smithi097.stdout:5/961: creat db/d18/d20/d2b/f13d x:0 0 0 2024-09-16T08:57:19.965 INFO:tasks.workunit.client.1.smithi097.stdout:5/962: truncate db/d6e/d79/dd5/f12c 1010375 0 2024-09-16T08:57:19.966 INFO:tasks.workunit.client.1.smithi097.stdout:5/963: readlink db/d18/d1c/d42/d46/l93 0 2024-09-16T08:57:19.966 INFO:tasks.workunit.client.0.smithi031.stdout:0/430: unlink d6/d12/f6a 0 2024-09-16T08:57:19.969 INFO:tasks.workunit.client.0.smithi031.stdout:4/414: dwrite d0/d2/f13 [0,4194304] 0 2024-09-16T08:57:19.970 INFO:tasks.workunit.client.1.smithi097.stdout:9/907: dread d2/d5/d6e/fd2 [0,4194304] 0 2024-09-16T08:57:19.970 INFO:tasks.workunit.client.1.smithi097.stdout:1/994: dwrite d5/d2c/f5e [0,4194304] 0 2024-09-16T08:57:19.971 INFO:tasks.workunit.client.0.smithi031.stdout:4/415: rename d0/d2/d16/c20 to d0/d2/d3a/d55/c85 0 2024-09-16T08:57:19.971 INFO:tasks.workunit.client.0.smithi031.stdout:4/416: stat d0/d2/d6d/f77 0 2024-09-16T08:57:19.972 INFO:tasks.workunit.client.1.smithi097.stdout:0/937: dwrite da/d24/d4c/d80/d3b/d86/faa [0,4194304] 0 2024-09-16T08:57:19.973 INFO:tasks.workunit.client.1.smithi097.stdout:0/938: creat da/d7e/dc4/dfc/d47/db3/f147 x:0 0 0 2024-09-16T08:57:19.973 INFO:tasks.workunit.client.0.smithi031.stdout:4/417: mkdir d0/d2/d16/d59/d84/d86 0 2024-09-16T08:57:19.975 INFO:tasks.workunit.client.1.smithi097.stdout:1/995: rename d5/d70/d85/cd2 to d5/d17/d54/d6b/da8/dc5/dc7/c142 0 2024-09-16T08:57:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:6/533: dwrite d0/dd/d74/f7e [0,4194304] 0 2024-09-16T08:57:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:6/534: chown d0/dd/d74/d19/d54 1 1 2024-09-16T08:57:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:4/418: link d0/d2/cf d0/d3c/c87 0 2024-09-16T08:57:19.977 INFO:tasks.workunit.client.0.smithi031.stdout:4/419: dread - d0/d71/f81 zero size 2024-09-16T08:57:19.978 INFO:tasks.workunit.client.1.smithi097.stdout:8/818: dwrite d8/dd/d15/d2c/d4a/d4d/d62/fc2 [0,4194304] 0 2024-09-16T08:57:19.978 INFO:tasks.workunit.client.0.smithi031.stdout:1/427: dwrite d2/d15/d5d/f23 [0,4194304] 0 2024-09-16T08:57:19.978 INFO:tasks.workunit.client.1.smithi097.stdout:9/908: dread d2/f25 [0,4194304] 0 2024-09-16T08:57:19.989 INFO:tasks.workunit.client.1.smithi097.stdout:0/939: creat da/d7e/dc4/dfc/d47/db3/d60/d73/d85/f148 x:0 0 0 2024-09-16T08:57:19.991 INFO:tasks.workunit.client.1.smithi097.stdout:1/996: mkdir d5/d70/d9d/d12c/d143 0 2024-09-16T08:57:19.991 INFO:tasks.workunit.client.1.smithi097.stdout:1/997: chown d5/ddd/ce9 1052470122 1 2024-09-16T08:57:19.991 INFO:tasks.workunit.client.1.smithi097.stdout:1/998: read - d5/db/dc/fc0 zero size 2024-09-16T08:57:19.992 INFO:tasks.workunit.client.0.smithi031.stdout:6/535: getdents d0/dd/d74/d19 0 2024-09-16T08:57:19.994 INFO:tasks.workunit.client.0.smithi031.stdout:4/420: symlink d0/d2/d6d/d75/l88 0 2024-09-16T08:57:19.994 INFO:tasks.workunit.client.1.smithi097.stdout:9/909: mkdir d2/d5/db0/d11f 0 2024-09-16T08:57:19.995 INFO:tasks.workunit.client.1.smithi097.stdout:0/940: creat da/d1a/f149 x:0 0 0 2024-09-16T08:57:19.997 INFO:tasks.workunit.client.1.smithi097.stdout:1/999: rename d5/d70/d9d/cab to d5/d17/d54/d6b/c144 0 2024-09-16T08:57:19.997 INFO:tasks.workunit.client.0.smithi031.stdout:6/536: symlink d0/dd/d35/la9 0 2024-09-16T08:57:19.998 INFO:tasks.workunit.client.0.smithi031.stdout:6/537: unlink d0/dd/d74/d5e/d7c/d3f/f63 0 2024-09-16T08:57:20.000 INFO:tasks.workunit.client.1.smithi097.stdout:0/941: rename da/d7e/dc4/dfc/d47/db3/d60/c99 to da/d24/d4c/d80/d3b/c14a 0 2024-09-16T08:57:20.000 INFO:tasks.workunit.client.1.smithi097.stdout:0/942: creat da/d7e/dc4/dfc/d47/db3/d60/d73/d85/f14b x:0 0 0 2024-09-16T08:57:20.001 INFO:tasks.workunit.client.0.smithi031.stdout:6/538: rmdir d0/dd/d74/da6/da8 0 2024-09-16T08:57:20.001 INFO:tasks.workunit.client.0.smithi031.stdout:6/539: stat d0/dd/d74/d5e/d7c/d3f/d65 0 2024-09-16T08:57:20.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/421: dread d0/d2/d16/f4f [0,4194304] 0 2024-09-16T08:57:20.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/422: creat d0/d5b/f89 x:0 0 0 2024-09-16T08:57:20.041 INFO:tasks.workunit.client.0.smithi031.stdout:2/356: dwrite d6/da/f5d [0,4194304] 0 2024-09-16T08:57:20.041 INFO:tasks.workunit.client.0.smithi031.stdout:1/428: dwrite d2/de/f13 [4194304,4194304] 0 2024-09-16T08:57:20.041 INFO:tasks.workunit.client.0.smithi031.stdout:1/429: fdatasync d2/d15/d32/f55 0 2024-09-16T08:57:20.041 INFO:tasks.workunit.client.0.smithi031.stdout:1/430: chown d2/c34 7 1 2024-09-16T08:57:20.041 INFO:tasks.workunit.client.0.smithi031.stdout:1/431: fdatasync d2/d15/d32/f55 0 2024-09-16T08:57:20.041 INFO:tasks.workunit.client.0.smithi031.stdout:7/473: dwrite d2/fb [0,4194304] 0 2024-09-16T08:57:20.041 INFO:tasks.workunit.client.0.smithi031.stdout:7/474: write d2/d8/f8c [170206,18552] 0 2024-09-16T08:57:20.042 INFO:tasks.workunit.client.1.smithi097.stdout:8/819: dwrite d8/dd/d15/d2c/ff9 [0,4194304] 0 2024-09-16T08:57:20.042 INFO:tasks.workunit.client.1.smithi097.stdout:5/964: dwrite db/f84 [0,4194304] 0 2024-09-16T08:57:20.042 INFO:tasks.workunit.client.1.smithi097.stdout:6/925: dwrite d1/dd3/d9e/dca/f1c [0,4194304] 0 2024-09-16T08:57:20.042 INFO:tasks.workunit.client.0.smithi031.stdout:6/540: dwrite d0/dd/d2b/f31 [0,4194304] 0 2024-09-16T08:57:20.042 INFO:tasks.workunit.client.0.smithi031.stdout:0/431: dwrite d6/d12/f77 [0,4194304] 0 2024-09-16T08:57:20.043 INFO:tasks.workunit.client.1.smithi097.stdout:0/943: dwrite da/d7e/dc4/dfc/d94/dd5/f140 [0,4194304] 0 2024-09-16T08:57:20.043 INFO:tasks.workunit.client.0.smithi031.stdout:6/541: dread d0/dd/d2b/f45 [0,4194304] 0 2024-09-16T08:57:20.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/357: mknod d6/da/d37/c79 0 2024-09-16T08:57:20.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/358: dread - d6/d2e/d40/d6c/f3b zero size 2024-09-16T08:57:20.045 INFO:tasks.workunit.client.0.smithi031.stdout:2/359: creat d6/d2e/d40/f7a x:0 0 0 2024-09-16T08:57:20.047 INFO:tasks.workunit.client.0.smithi031.stdout:0/432: dread d6/d12/d47/d5b/f5e [0,4194304] 0 2024-09-16T08:57:20.047 INFO:tasks.workunit.client.0.smithi031.stdout:0/433: chown d6/d61/c75 87938 1 2024-09-16T08:57:20.047 INFO:tasks.workunit.client.0.smithi031.stdout:1/432: rename d2/d15/d1e to d2/d9b 0 2024-09-16T08:57:20.048 INFO:tasks.workunit.client.1.smithi097.stdout:5/965: creat db/d1a/d6d/dce/d105/f13e x:0 0 0 2024-09-16T08:57:20.048 INFO:tasks.workunit.client.1.smithi097.stdout:5/966: chown db/d18/d20/fc9 5 1 2024-09-16T08:57:20.048 INFO:tasks.workunit.client.1.smithi097.stdout:5/967: write db/d18/f113 [148251,31053] 0 2024-09-16T08:57:20.049 INFO:tasks.workunit.client.1.smithi097.stdout:6/926: getdents d1/dd3/d9e/dca/d16/d6a 0 2024-09-16T08:57:20.050 INFO:tasks.workunit.client.1.smithi097.stdout:6/927: chown d1/dc/l119 32120 1 2024-09-16T08:57:20.050 INFO:tasks.workunit.client.1.smithi097.stdout:6/928: dread - d1/dd3/d9e/dca/d81/dc4/fc8 zero size 2024-09-16T08:57:20.050 INFO:tasks.workunit.client.0.smithi031.stdout:7/475: mkdir d2/d8/d86/da1 0 2024-09-16T08:57:20.055 INFO:tasks.workunit.client.1.smithi097.stdout:0/944: mkdir da/d24/d4c/d6d/d14c 0 2024-09-16T08:57:20.056 INFO:tasks.workunit.client.1.smithi097.stdout:0/945: creat da/d7e/dc4/dfc/d47/f14d x:0 0 0 2024-09-16T08:57:20.056 INFO:tasks.workunit.client.1.smithi097.stdout:0/946: readlink da/d7e/dc4/dfc/l92 0 2024-09-16T08:57:20.056 INFO:tasks.workunit.client.1.smithi097.stdout:0/947: fsync da/d7e/dc4/dfc/d47/db3/d60/d73/d85/fc8 0 2024-09-16T08:57:20.056 INFO:tasks.workunit.client.0.smithi031.stdout:6/542: symlink d0/laa 0 2024-09-16T08:57:20.057 INFO:tasks.workunit.client.1.smithi097.stdout:5/968: dread db/d1a/f57 [0,4194304] 0 2024-09-16T08:57:20.057 INFO:tasks.workunit.client.1.smithi097.stdout:8/820: rmdir d8/dd/d15/d2c/d4a/d4d/ddd 39 2024-09-16T08:57:20.059 INFO:tasks.workunit.client.1.smithi097.stdout:6/929: rmdir d1 39 2024-09-16T08:57:20.060 INFO:tasks.workunit.client.0.smithi031.stdout:0/434: link d6/d12/d13/c88 d6/d12/d47/c8d 0 2024-09-16T08:57:20.063 INFO:tasks.workunit.client.0.smithi031.stdout:2/360: rename d6/da/c12 to d6/d2e/d40/d6c/d2c/d53/d60/c7b 0 2024-09-16T08:57:20.064 INFO:tasks.workunit.client.1.smithi097.stdout:0/948: symlink da/df/d13/db0/l14e 0 2024-09-16T08:57:20.066 INFO:tasks.workunit.client.0.smithi031.stdout:1/433: mkdir d2/d15/d9c 0 2024-09-16T08:57:20.066 INFO:tasks.workunit.client.0.smithi031.stdout:1/434: fdatasync d2/d15/d32/f97 0 2024-09-16T08:57:20.068 INFO:tasks.workunit.client.1.smithi097.stdout:5/969: rename db/d1a/d6d/dc6/c123 to db/d18/d1c/d42/d46/d7e/d64/d7f/d89/c13f 0 2024-09-16T08:57:20.068 INFO:tasks.workunit.client.1.smithi097.stdout:5/970: chown db/d18/d1c/d42/c120 147 1 2024-09-16T08:57:20.068 INFO:tasks.workunit.client.1.smithi097.stdout:5/971: readlink db/d18/d1c/l2a 0 2024-09-16T08:57:20.068 INFO:tasks.workunit.client.1.smithi097.stdout:5/972: write db/d1a/d6d/dc6/fe7 [711759,55367] 0 2024-09-16T08:57:20.069 INFO:tasks.workunit.client.0.smithi031.stdout:7/476: chown d2/d8/d11/c64 2 1 2024-09-16T08:57:20.072 INFO:tasks.workunit.client.1.smithi097.stdout:8/821: link d8/dd/c27 d8/dd/d15/d2c/d4a/d4e/db2/c102 0 2024-09-16T08:57:20.073 INFO:tasks.workunit.client.1.smithi097.stdout:8/822: readlink d8/dd/d15/d2c/d4a/d4d/d62/l64 0 2024-09-16T08:57:20.073 INFO:tasks.workunit.client.1.smithi097.stdout:8/823: creat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f103 x:0 0 0 2024-09-16T08:57:20.074 INFO:tasks.workunit.client.0.smithi031.stdout:6/543: symlink d0/dd/d74/d5e/d7c/lab 0 2024-09-16T08:57:20.074 INFO:tasks.workunit.client.1.smithi097.stdout:0/949: rmdir da/d7e/dc4/dfc/d94/dc9/dcb 39 2024-09-16T08:57:20.075 INFO:tasks.workunit.client.0.smithi031.stdout:2/361: symlink d6/d2e/d40/d58/l7c 0 2024-09-16T08:57:20.077 INFO:tasks.workunit.client.0.smithi031.stdout:1/435: mkdir d2/d4/d52/d9d 0 2024-09-16T08:57:20.079 INFO:tasks.workunit.client.1.smithi097.stdout:8/824: symlink d8/dd/d15/d2c/d4a/d4e/d85/l104 0 2024-09-16T08:57:20.080 INFO:tasks.workunit.client.0.smithi031.stdout:6/544: mkdir d0/dd/d35/d6c/dac 0 2024-09-16T08:57:20.082 INFO:tasks.workunit.client.1.smithi097.stdout:0/950: symlink da/d24/d4b/dfb/l14f 0 2024-09-16T08:57:20.083 INFO:tasks.workunit.client.1.smithi097.stdout:0/951: dread da/d7e/dc4/dfc/d47/db3/dbf/d12b/f8f [0,4194304] 0 2024-09-16T08:57:20.083 INFO:tasks.workunit.client.1.smithi097.stdout:0/952: write da/d7e/dc4/dfc/d94/ff6 [76072,27672] 0 2024-09-16T08:57:20.083 INFO:tasks.workunit.client.0.smithi031.stdout:2/362: symlink d6/da/d5e/d72/l7d 0 2024-09-16T08:57:20.083 INFO:tasks.workunit.client.0.smithi031.stdout:2/363: chown d6/da/fe 2605131 1 2024-09-16T08:57:20.085 INFO:tasks.workunit.client.0.smithi031.stdout:1/436: rename d2/d15/l42 to d2/d15/d9c/l9e 0 2024-09-16T08:57:20.086 INFO:tasks.workunit.client.1.smithi097.stdout:8/825: mkdir d8/dd/d15/d2c/d4a/d4e/db2/d105 0 2024-09-16T08:57:20.087 INFO:tasks.workunit.client.0.smithi031.stdout:6/545: mkdir d0/dd/d74/dad 0 2024-09-16T08:57:20.089 INFO:tasks.workunit.client.0.smithi031.stdout:1/437: dread d2/d15/d38/f76 [0,4194304] 0 2024-09-16T08:57:20.089 INFO:tasks.workunit.client.1.smithi097.stdout:6/930: dwrite d1/dd3/d9e/dca/d81/dc4/fc8 [0,4194304] 0 2024-09-16T08:57:20.090 INFO:tasks.workunit.client.0.smithi031.stdout:6/546: symlink d0/dd/d74/d5e/d7c/lae 0 2024-09-16T08:57:20.092 INFO:tasks.workunit.client.0.smithi031.stdout:1/438: mknod d2/d15/d1f/d43/c9f 0 2024-09-16T08:57:20.094 INFO:tasks.workunit.client.0.smithi031.stdout:2/364: write d6/da/f6a [3920538,44029] 0 2024-09-16T08:57:20.095 INFO:tasks.workunit.client.0.smithi031.stdout:6/547: mknod d0/dd/d74/d5e/d7c/d3f/caf 0 2024-09-16T08:57:20.095 INFO:tasks.workunit.client.0.smithi031.stdout:0/435: dwrite d6/d12/f77 [0,4194304] 0 2024-09-16T08:57:20.095 INFO:tasks.workunit.client.0.smithi031.stdout:6/548: chown d0/dd/c50 0 1 2024-09-16T08:57:20.095 INFO:tasks.workunit.client.0.smithi031.stdout:7/477: dwrite d2/f5b [0,4194304] 0 2024-09-16T08:57:20.095 INFO:tasks.workunit.client.0.smithi031.stdout:7/478: chown d2/d2d/c7c 6 1 2024-09-16T08:57:20.100 INFO:tasks.workunit.client.0.smithi031.stdout:1/439: link d2/de/f2a d2/d15/d32/d87/fa0 0 2024-09-16T08:57:20.101 INFO:tasks.workunit.client.0.smithi031.stdout:1/440: fsync d2/d4/d1b/d5c/f8e 0 2024-09-16T08:57:20.103 INFO:tasks.workunit.client.0.smithi031.stdout:0/436: dread d6/f9 [0,4194304] 0 2024-09-16T08:57:20.103 INFO:tasks.workunit.client.0.smithi031.stdout:0/437: fsync d6/d12/f6c 0 2024-09-16T08:57:20.108 INFO:tasks.workunit.client.0.smithi031.stdout:2/365: unlink d6/d20/f47 0 2024-09-16T08:57:20.108 INFO:tasks.workunit.client.0.smithi031.stdout:2/366: write d6/d2e/d40/d6c/d2c/d53/d60/f35 [5375146,121753] 0 2024-09-16T08:57:20.108 INFO:tasks.workunit.client.0.smithi031.stdout:2/367: write d6/da/f1d [683887,53617] 0 2024-09-16T08:57:20.108 INFO:tasks.workunit.client.0.smithi031.stdout:9/395: sync 2024-09-16T08:57:20.108 INFO:tasks.workunit.client.0.smithi031.stdout:9/396: dread - d3/d5/da/f89 zero size 2024-09-16T08:57:20.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/549: creat d0/dd/d74/d67/fb0 x:0 0 0 2024-09-16T08:57:20.112 INFO:tasks.workunit.client.1.smithi097.stdout:6/931: mknod d1/dc/c13a 0 2024-09-16T08:57:20.112 INFO:tasks.workunit.client.1.smithi097.stdout:6/932: fdatasync d1/dd3/d9e/dca/d16/d4f/d9f/fa4 0 2024-09-16T08:57:20.115 INFO:tasks.workunit.client.1.smithi097.stdout:0/953: rmdir da/df/dd8 0 2024-09-16T08:57:20.115 INFO:tasks.workunit.client.1.smithi097.stdout:0/954: dread - da/d7e/dc4/dfc/d47/f13b zero size 2024-09-16T08:57:20.116 INFO:tasks.workunit.client.0.smithi031.stdout:1/441: mknod d2/ca1 0 2024-09-16T08:57:20.116 INFO:tasks.workunit.client.0.smithi031.stdout:1/442: truncate d2/d4/d1b/f65 352223 0 2024-09-16T08:57:20.117 INFO:tasks.workunit.client.1.smithi097.stdout:6/933: link d1/dd3/d9e/dca/d16/c74 d1/dd3/d9e/dca/d16/d4f/df8/c13b 0 2024-09-16T08:57:20.118 INFO:tasks.workunit.client.0.smithi031.stdout:9/397: mkdir d3/d5/da/db/d21/d3b/d8b 0 2024-09-16T08:57:20.121 INFO:tasks.workunit.client.0.smithi031.stdout:6/550: creat d0/dd/d74/d5e/fb1 x:0 0 0 2024-09-16T08:57:20.122 INFO:tasks.workunit.client.0.smithi031.stdout:7/479: dwrite d2/d8/f1d [0,4194304] 0 2024-09-16T08:57:20.122 INFO:tasks.workunit.client.0.smithi031.stdout:2/368: write d6/d2e/d40/d6c/f2b [1202085,87811] 0 2024-09-16T08:57:20.123 INFO:tasks.workunit.client.0.smithi031.stdout:0/438: dwrite d6/d12/d47/d4d/f7a [0,4194304] 0 2024-09-16T08:57:20.123 INFO:tasks.workunit.client.1.smithi097.stdout:6/934: unlink d1/db7/dfd/f83 0 2024-09-16T08:57:20.123 INFO:tasks.workunit.client.1.smithi097.stdout:6/935: write d1/dd3/d9e/dca/d16/d4f/f94 [727055,87736] 0 2024-09-16T08:57:20.124 INFO:tasks.workunit.client.0.smithi031.stdout:9/398: rmdir d3/d5/da/d34/d3c 39 2024-09-16T08:57:20.125 INFO:tasks.workunit.client.0.smithi031.stdout:7/480: mkdir d2/d2d/d68/da2 0 2024-09-16T08:57:20.127 INFO:tasks.workunit.client.0.smithi031.stdout:2/369: symlink d6/d2e/l7e 0 2024-09-16T08:57:20.133 INFO:tasks.workunit.client.1.smithi097.stdout:6/936: dread d1/dd3/d9e/dca/d16/d4f/f55 [0,4194304] 0 2024-09-16T08:57:20.134 INFO:tasks.workunit.client.0.smithi031.stdout:6/551: dread d0/dd/d2b/f31 [0,4194304] 0 2024-09-16T08:57:20.134 INFO:tasks.workunit.client.0.smithi031.stdout:6/552: write d0/dd/d74/d67/fb0 [153660,48411] 0 2024-09-16T08:57:20.134 INFO:tasks.workunit.client.0.smithi031.stdout:6/553: readlink d0/dd/d74/d5e/d7c/d3f/l82 0 2024-09-16T08:57:20.134 INFO:tasks.workunit.client.0.smithi031.stdout:6/554: write d0/dd/d74/d19/f29 [3844545,21737] 0 2024-09-16T08:57:20.134 INFO:tasks.workunit.client.0.smithi031.stdout:1/443: dwrite d2/d4/f5 [0,4194304] 0 2024-09-16T08:57:20.134 INFO:tasks.workunit.client.0.smithi031.stdout:1/444: chown d2/d9b 1469310 1 2024-09-16T08:57:20.135 INFO:tasks.workunit.client.0.smithi031.stdout:1/445: truncate d2/d4/d1b/f65 754312 0 2024-09-16T08:57:20.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/481: symlink d2/d2d/la3 0 2024-09-16T08:57:20.135 INFO:tasks.workunit.client.0.smithi031.stdout:7/482: truncate d2/d8/f60 910236 0 2024-09-16T08:57:20.136 INFO:tasks.workunit.client.0.smithi031.stdout:0/439: read d6/d12/d47/f4e [2307218,41513] 0 2024-09-16T08:57:20.136 INFO:tasks.workunit.client.0.smithi031.stdout:9/399: rmdir d3/d48 39 2024-09-16T08:57:20.136 INFO:tasks.workunit.client.0.smithi031.stdout:9/400: write d3/d48/f49 [1168885,71741] 0 2024-09-16T08:57:20.136 INFO:tasks.workunit.client.0.smithi031.stdout:9/401: dread - d3/d5/da/d34/d3c/d5e/d62/f84 zero size 2024-09-16T08:57:20.138 INFO:tasks.workunit.client.0.smithi031.stdout:6/555: truncate d0/dd/f3e 3438412 0 2024-09-16T08:57:20.141 INFO:tasks.workunit.client.0.smithi031.stdout:0/440: rmdir d6/d12/d47/d7c 39 2024-09-16T08:57:20.143 INFO:tasks.workunit.client.0.smithi031.stdout:9/402: link d3/d5/c6c d3/d5/d73/d81/c8c 0 2024-09-16T08:57:20.144 INFO:tasks.workunit.client.0.smithi031.stdout:1/446: getdents d2/d4/d1b/d5c/d6b 0 2024-09-16T08:57:20.146 INFO:tasks.workunit.client.0.smithi031.stdout:9/403: symlink d3/d5/da/db/d21/d67/l8d 0 2024-09-16T08:57:20.147 INFO:tasks.workunit.client.0.smithi031.stdout:9/404: chown d3/l30 337 1 2024-09-16T08:57:20.147 INFO:tasks.workunit.client.0.smithi031.stdout:9/405: write d3/d5/da/d34/d4d/f66 [4985766,24530] 0 2024-09-16T08:57:20.148 INFO:tasks.workunit.client.0.smithi031.stdout:1/447: rmdir d2/d4/d52/d9d 0 2024-09-16T08:57:20.148 INFO:tasks.workunit.client.0.smithi031.stdout:1/448: dread - d2/d4/f82 zero size 2024-09-16T08:57:20.149 INFO:tasks.workunit.client.0.smithi031.stdout:9/406: creat d3/d5/da/d34/d4d/f8e x:0 0 0 2024-09-16T08:57:20.150 INFO:tasks.workunit.client.0.smithi031.stdout:9/407: unlink d3/c23 0 2024-09-16T08:57:20.151 INFO:tasks.workunit.client.0.smithi031.stdout:2/370: dwrite d6/d20/f4f [4194304,4194304] 0 2024-09-16T08:57:20.153 INFO:tasks.workunit.client.0.smithi031.stdout:2/371: mknod d6/d20/c7f 0 2024-09-16T08:57:20.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/372: fsync d6/da/f1e 0 2024-09-16T08:57:20.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/373: write d6/d2e/d40/d6c/f3b [568558,122101] 0 2024-09-16T08:57:20.155 INFO:tasks.workunit.client.0.smithi031.stdout:2/374: symlink d6/da/d5e/d72/l80 0 2024-09-16T08:57:20.156 INFO:tasks.workunit.client.0.smithi031.stdout:2/375: chown d6/da/fe 53440 1 2024-09-16T08:57:20.156 INFO:tasks.workunit.client.0.smithi031.stdout:2/376: chown d6/c10 2319238 1 2024-09-16T08:57:20.157 INFO:tasks.workunit.client.1.smithi097.stdout:6/937: dwrite d1/dd3/d9e/dca/d16/fdd [0,4194304] 0 2024-09-16T08:57:20.157 INFO:tasks.workunit.client.1.smithi097.stdout:6/938: write d1/dd3/d9e/dca/d81/dad/f9a [2309891,74183] 0 2024-09-16T08:57:20.157 INFO:tasks.workunit.client.0.smithi031.stdout:2/377: truncate d6/d20/f26 1118307 0 2024-09-16T08:57:20.160 INFO:tasks.workunit.client.1.smithi097.stdout:6/939: mkdir d1/dd3/d9e/dca/d13c 0 2024-09-16T08:57:20.169 INFO:tasks.workunit.client.1.smithi097.stdout:0/955: write da/d7e/dc4/dfc/d94/d101/f44 [3210880,116761] 0 2024-09-16T08:57:20.170 INFO:tasks.workunit.client.1.smithi097.stdout:3/889: sync 2024-09-16T08:57:20.170 INFO:tasks.workunit.client.1.smithi097.stdout:2/915: sync 2024-09-16T08:57:20.170 INFO:tasks.workunit.client.1.smithi097.stdout:0/956: symlink da/d24/d4c/d80/dab/l150 0 2024-09-16T08:57:20.172 INFO:tasks.workunit.client.1.smithi097.stdout:2/916: rmdir d3/d4/d68/d8b/d7c/d7d/d53/da3/de8 39 2024-09-16T08:57:20.173 INFO:tasks.workunit.client.0.smithi031.stdout:7/483: dwrite d2/d2d/f2e [4194304,4194304] 0 2024-09-16T08:57:20.174 INFO:tasks.workunit.client.1.smithi097.stdout:0/957: mknod da/d24/d4c/d80/d42/c151 0 2024-09-16T08:57:20.174 INFO:tasks.workunit.client.1.smithi097.stdout:0/958: read - da/d7e/dc4/dfc/d47/db3/dbf/dcc/f143 zero size 2024-09-16T08:57:20.177 INFO:tasks.workunit.client.0.smithi031.stdout:7/484: creat d2/d8/d11/fa4 x:0 0 0 2024-09-16T08:57:20.178 INFO:tasks.workunit.client.1.smithi097.stdout:2/917: creat d3/d4/d68/d8b/d61/dac/dd9/de6/f121 x:0 0 0 2024-09-16T08:57:20.180 INFO:tasks.workunit.client.1.smithi097.stdout:0/959: link da/d7e/f11c da/df/d8a/dac/f152 0 2024-09-16T08:57:20.202 INFO:tasks.workunit.client.0.smithi031.stdout:6/556: dwrite d0/dd/d74/f96 [0,4194304] 0 2024-09-16T08:57:20.204 INFO:tasks.workunit.client.0.smithi031.stdout:6/557: creat d0/d8e/fb2 x:0 0 0 2024-09-16T08:57:20.205 INFO:tasks.workunit.client.0.smithi031.stdout:6/558: mknod d0/dd/d74/cb3 0 2024-09-16T08:57:20.217 INFO:tasks.workunit.client.1.smithi097.stdout:8/826: dwrite d8/f1e [0,4194304] 0 2024-09-16T08:57:20.218 INFO:tasks.workunit.client.1.smithi097.stdout:8/827: fsync d8/f24 0 2024-09-16T08:57:20.218 INFO:tasks.workunit.client.1.smithi097.stdout:3/890: dwrite d7/f21 [0,4194304] 0 2024-09-16T08:57:20.218 INFO:tasks.workunit.client.1.smithi097.stdout:0/960: dwrite da/d1a/f1c [0,4194304] 0 2024-09-16T08:57:20.218 INFO:tasks.workunit.client.0.smithi031.stdout:2/378: dwrite d6/f5b [0,4194304] 0 2024-09-16T08:57:20.219 INFO:tasks.workunit.client.0.smithi031.stdout:2/379: readlink d6/da/d4b/l67 0 2024-09-16T08:57:20.219 INFO:tasks.workunit.client.0.smithi031.stdout:9/408: dwrite d3/d5/da/d34/f46 [0,4194304] 0 2024-09-16T08:57:20.219 INFO:tasks.workunit.client.0.smithi031.stdout:1/449: dwrite d2/d15/d5d/f23 [0,4194304] 0 2024-09-16T08:57:20.219 INFO:tasks.workunit.client.0.smithi031.stdout:6/559: write d0/f4 [5443989,68440] 0 2024-09-16T08:57:20.219 INFO:tasks.workunit.client.0.smithi031.stdout:1/450: chown d2/d4/d1b/c6e 67692 1 2024-09-16T08:57:20.219 INFO:tasks.workunit.client.1.smithi097.stdout:2/918: dwrite d3/d4/d68/d8b/d7c/d7d/d53/d47/f98 [0,4194304] 0 2024-09-16T08:57:20.219 INFO:tasks.workunit.client.1.smithi097.stdout:2/919: write d3/d4/d68/d8b/d61/dac/f106 [122474,10731] 0 2024-09-16T08:57:20.227 INFO:tasks.workunit.client.0.smithi031.stdout:9/409: dread d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:20.235 INFO:tasks.workunit.client.1.smithi097.stdout:2/920: rmdir d3/d4/d68/d8b/d61/dac/dd9/de6 39 2024-09-16T08:57:20.236 INFO:tasks.workunit.client.1.smithi097.stdout:2/921: write d3/d4/d68/d8b/d7c/d7d/d53/f50 [582003,52567] 0 2024-09-16T08:57:20.236 INFO:tasks.workunit.client.0.smithi031.stdout:9/410: mknod d3/d5/da/db/d21/d3b/d8b/c8f 0 2024-09-16T08:57:20.239 INFO:tasks.workunit.client.0.smithi031.stdout:9/411: rename d3/d5/da/db/d21/d67/f71 to d3/d5/da/db/d21/d3b/d80/f90 0 2024-09-16T08:57:20.242 INFO:tasks.workunit.client.0.smithi031.stdout:9/412: creat d3/d85/f91 x:0 0 0 2024-09-16T08:57:20.244 INFO:tasks.workunit.client.0.smithi031.stdout:9/413: mkdir d3/d5/da/d34/d3c/d5e/d62/d92 0 2024-09-16T08:57:20.246 INFO:tasks.workunit.client.0.smithi031.stdout:9/414: rename d3/d5/da/d34/d4d/f8e to d3/d5/da/d34/d3c/d5e/d62/d65/f93 0 2024-09-16T08:57:20.246 INFO:tasks.workunit.client.0.smithi031.stdout:9/415: readlink d3/d5/da/db/d21/d67/l8d 0 2024-09-16T08:57:20.246 INFO:tasks.workunit.client.0.smithi031.stdout:9/416: chown d3/d5/da/db/c25 2 1 2024-09-16T08:57:20.247 INFO:tasks.workunit.client.1.smithi097.stdout:2/922: rename d3/d4/d68/d8b to d3/d4/d9/d67/d122 0 2024-09-16T08:57:20.247 INFO:tasks.workunit.client.0.smithi031.stdout:9/417: truncate d3/d5/da/f2b 5346026 0 2024-09-16T08:57:20.248 INFO:tasks.workunit.client.1.smithi097.stdout:2/923: creat d3/d4/d9/d67/d122/d7c/d7d/d53/d47/f123 x:0 0 0 2024-09-16T08:57:20.249 INFO:tasks.workunit.client.0.smithi031.stdout:9/418: creat d3/d5/da/d34/d3c/d5e/d62/d65/f94 x:0 0 0 2024-09-16T08:57:20.250 INFO:tasks.workunit.client.1.smithi097.stdout:2/924: mknod d3/d1b/c124 0 2024-09-16T08:57:20.251 INFO:tasks.workunit.client.0.smithi031.stdout:9/419: rename d3/d5/da/d34/d3c/d42 to d3/d5/da/db/d21/d3b/d80/d95 0 2024-09-16T08:57:20.252 INFO:tasks.workunit.client.1.smithi097.stdout:2/925: mknod d3/d4/d9/d67/d122/d61/c125 0 2024-09-16T08:57:20.252 INFO:tasks.workunit.client.1.smithi097.stdout:2/926: getdents d3/d4/d9/d67/d122/d7c/d7d/d53/da3/de8/df8/d11a 0 2024-09-16T08:57:20.252 INFO:tasks.workunit.client.1.smithi097.stdout:2/927: write d3/d4/d9/d67/d122/d61/dac/dd9/de6/f121 [933705,5045] 0 2024-09-16T08:57:20.252 INFO:tasks.workunit.client.0.smithi031.stdout:9/420: symlink d3/d5/da/db/d21/d3b/d80/d95/l96 0 2024-09-16T08:57:20.252 INFO:tasks.workunit.client.0.smithi031.stdout:9/421: dread - d3/d5/da/f4f zero size 2024-09-16T08:57:20.254 INFO:tasks.workunit.client.1.smithi097.stdout:2/928: write d3/d1b/dbf/fec [193597,80728] 0 2024-09-16T08:57:20.254 INFO:tasks.workunit.client.1.smithi097.stdout:2/929: write d3/f32 [4884848,5527] 0 2024-09-16T08:57:20.254 INFO:tasks.workunit.client.0.smithi031.stdout:9/422: rename d3/d5/da/db/d21/d3b/d80/d95/l64 to d3/d5/da/d34/d3c/d5e/d62/d65/l97 0 2024-09-16T08:57:20.254 INFO:tasks.workunit.client.0.smithi031.stdout:9/423: creat d3/d85/f98 x:0 0 0 2024-09-16T08:57:20.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:19 smithi097 ceph-mon[35478]: pgmap v11: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 201 MiB/s rd, 265 MiB/s wr, 444 op/s 2024-09-16T08:57:20.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:19 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:20.283 INFO:tasks.workunit.client.0.smithi031.stdout:6/560: write d0/dd/d74/f96 [905585,95655] 0 2024-09-16T08:57:20.284 INFO:tasks.workunit.client.0.smithi031.stdout:6/561: stat d0/dd/d74/d5e/d7c/l1b 0 2024-09-16T08:57:20.291 INFO:tasks.workunit.client.1.smithi097.stdout:7/823: sync 2024-09-16T08:57:20.298 INFO:tasks.workunit.client.1.smithi097.stdout:4/813: sync 2024-09-16T08:57:20.298 INFO:tasks.workunit.client.1.smithi097.stdout:4/814: dread - d0/d7/df/d15/d1f/fef zero size 2024-09-16T08:57:20.300 INFO:tasks.workunit.client.1.smithi097.stdout:4/815: unlink d0/d7/df/d83/df8/dbc/cd4 0 2024-09-16T08:57:20.301 INFO:tasks.workunit.client.1.smithi097.stdout:4/816: fdatasync d0/d36/da7/f112 0 2024-09-16T08:57:20.301 INFO:tasks.workunit.client.0.smithi031.stdout:2/380: dwrite d6/d2e/d40/f54 [0,4194304] 0 2024-09-16T08:57:20.304 INFO:tasks.workunit.client.0.smithi031.stdout:2/381: rmdir d6/d20/d51 39 2024-09-16T08:57:20.325 INFO:tasks.workunit.client.1.smithi097.stdout:0/961: dwrite da/d24/d4c/fe7 [0,4194304] 0 2024-09-16T08:57:20.327 INFO:tasks.workunit.client.0.smithi031.stdout:5/438: sync 2024-09-16T08:57:20.327 INFO:tasks.workunit.client.0.smithi031.stdout:3/501: sync 2024-09-16T08:57:20.327 INFO:tasks.workunit.client.0.smithi031.stdout:2/382: write d6/d20/f3f [3870834,83516] 0 2024-09-16T08:57:20.328 INFO:tasks.workunit.client.1.smithi097.stdout:0/962: mknod da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dfa/c153 0 2024-09-16T08:57:20.328 INFO:tasks.workunit.client.1.smithi097.stdout:0/963: stat da/d7e/dc4/dfc/d47/db3/d60/d68/d5e/f72 0 2024-09-16T08:57:20.328 INFO:tasks.workunit.client.1.smithi097.stdout:2/930: dwrite d3/d4/d9/d67/d122/d7c/d7d/f6d [8388608,4194304] 0 2024-09-16T08:57:20.329 INFO:tasks.workunit.client.1.smithi097.stdout:2/931: write d3/d4/d9/d67/d122/d61/fb1 [1404135,2241] 0 2024-09-16T08:57:20.329 INFO:tasks.workunit.client.1.smithi097.stdout:2/932: write d3/d4/d9/d67/d122/d7c/d7d/f1f [3348893,45102] 0 2024-09-16T08:57:20.330 INFO:tasks.workunit.client.1.smithi097.stdout:7/824: dwrite d4/d9/f24 [0,4194304] 0 2024-09-16T08:57:20.330 INFO:tasks.workunit.client.1.smithi097.stdout:4/817: dwrite d0/d7/df/f13 [0,4194304] 0 2024-09-16T08:57:20.330 INFO:tasks.workunit.client.1.smithi097.stdout:4/818: read d0/d36/d4c/f80 [322559,67106] 0 2024-09-16T08:57:20.330 INFO:tasks.workunit.client.0.smithi031.stdout:1/451: dwrite d2/d4/d1b/d5c/d6b/f7b [0,4194304] 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:1/452: creat d2/d4/fa2 x:0 0 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:6/562: dwrite d0/dd/d74/d5e/f77 [0,4194304] 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:9/424: dwrite f2 [0,4194304] 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:9/425: stat d3/c2e 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:9/426: creat d3/d5/da/d34/f99 x:0 0 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:6/563: write d0/dd/d74/d5e/f7b [993451,82687] 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:2/383: symlink d6/da/d4b/l81 0 2024-09-16T08:57:20.331 INFO:tasks.workunit.client.0.smithi031.stdout:2/384: fdatasync d6/fc 0 2024-09-16T08:57:20.333 INFO:tasks.workunit.client.1.smithi097.stdout:2/933: mkdir d3/d4/d77/d126 0 2024-09-16T08:57:20.333 INFO:tasks.workunit.client.1.smithi097.stdout:2/934: fdatasync d3/d4/d9/d67/d122/d7c/d7d/f8d 0 2024-09-16T08:57:20.334 INFO:tasks.workunit.client.0.smithi031.stdout:5/439: dread d2/dc/f3a [0,4194304] 0 2024-09-16T08:57:20.341 INFO:tasks.workunit.client.0.smithi031.stdout:1/453: mkdir d2/d9b/da3 0 2024-09-16T08:57:20.342 INFO:tasks.workunit.client.1.smithi097.stdout:7/825: dread d4/d1b/f91 [0,4194304] 0 2024-09-16T08:57:20.342 INFO:tasks.workunit.client.1.smithi097.stdout:7/826: chown d4/d7/d19/ddd/l103 14 1 2024-09-16T08:57:20.342 INFO:tasks.workunit.client.1.smithi097.stdout:7/827: chown d4/d9/d1c/d5f 2 1 2024-09-16T08:57:20.343 INFO:tasks.workunit.client.0.smithi031.stdout:3/502: mkdir d3/d14/d37/d1a/d1d/d3b/d94 0 2024-09-16T08:57:20.349 INFO:tasks.workunit.client.1.smithi097.stdout:0/964: dwrite da/d1a/f63 [0,4194304] 0 2024-09-16T08:57:20.353 INFO:tasks.workunit.client.0.smithi031.stdout:9/427: unlink d3/d5/da/db/l10 0 2024-09-16T08:57:20.354 INFO:tasks.workunit.client.0.smithi031.stdout:9/428: read d3/d5/da/db/f39 [13948,117290] 0 2024-09-16T08:57:20.355 INFO:tasks.workunit.client.1.smithi097.stdout:2/935: getdents d3/d4/d9/d67/d122/d7c/d7d/d53/da3/de8 0 2024-09-16T08:57:20.355 INFO:tasks.workunit.client.1.smithi097.stdout:2/936: fsync d3/d4/d8/d51/fd1 0 2024-09-16T08:57:20.355 INFO:tasks.workunit.client.1.smithi097.stdout:2/937: chown d3/d4/d9/d78/dcc 748989378 1 2024-09-16T08:57:20.355 INFO:tasks.workunit.client.0.smithi031.stdout:3/503: dread d3/f5 [0,4194304] 0 2024-09-16T08:57:20.356 INFO:tasks.workunit.client.0.smithi031.stdout:3/504: creat d3/d14/d37/d33/f95 x:0 0 0 2024-09-16T08:57:20.356 INFO:tasks.workunit.client.0.smithi031.stdout:6/564: mkdir d0/dd/d74/d19/db4 0 2024-09-16T08:57:20.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/385: mknod d6/da/d4b/d4c/d68/c82 0 2024-09-16T08:57:20.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/386: chown d6/da/d4b/l81 543111960 1 2024-09-16T08:57:20.357 INFO:tasks.workunit.client.1.smithi097.stdout:7/828: write d4/d7/d19/f71 [4172141,72459] 0 2024-09-16T08:57:20.357 INFO:tasks.workunit.client.1.smithi097.stdout:7/829: stat d4/d9/dc/f72 0 2024-09-16T08:57:20.360 INFO:tasks.workunit.client.1.smithi097.stdout:0/965: symlink da/d24/d4c/d80/d42/l154 0 2024-09-16T08:57:20.360 INFO:tasks.workunit.client.1.smithi097.stdout:0/966: chown da/d24/d4c/d80/d3b/f5a 44 1 2024-09-16T08:57:20.361 INFO:tasks.workunit.client.0.smithi031.stdout:9/429: dread d3/d5/da/f2c [0,4194304] 0 2024-09-16T08:57:20.363 INFO:tasks.workunit.client.1.smithi097.stdout:2/938: rmdir d3/d4/d9/d67/d122/d61/dac 39 2024-09-16T08:57:20.364 INFO:tasks.workunit.client.0.smithi031.stdout:5/440: rename d2/dc/d25/d21/f8e to d2/dc/d25/d21/d2e/f8f 0 2024-09-16T08:57:20.365 INFO:tasks.workunit.client.1.smithi097.stdout:7/830: creat d4/d7/d30/ddf/f11a x:0 0 0 2024-09-16T08:57:20.367 INFO:tasks.workunit.client.0.smithi031.stdout:1/454: symlink d2/d15/d5d/la4 0 2024-09-16T08:57:20.368 INFO:tasks.workunit.client.1.smithi097.stdout:9/910: sync 2024-09-16T08:57:20.368 INFO:tasks.workunit.client.1.smithi097.stdout:0/967: truncate da/d7e/dc4/dfc/d47/db3/fd4 2448562 0 2024-09-16T08:57:20.371 INFO:tasks.workunit.client.0.smithi031.stdout:3/505: mkdir d3/d14/d6f/d82/d96 0 2024-09-16T08:57:20.371 INFO:tasks.workunit.client.0.smithi031.stdout:3/506: read d3/d14/d37/d1a/f26 [1178196,82085] 0 2024-09-16T08:57:20.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:19 smithi031 ceph-mon[27844]: pgmap v11: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 201 MiB/s rd, 265 MiB/s wr, 444 op/s 2024-09-16T08:57:20.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:19 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:20.373 INFO:tasks.workunit.client.1.smithi097.stdout:9/911: creat d2/d5/d8d/db1/f120 x:0 0 0 2024-09-16T08:57:20.373 INFO:tasks.workunit.client.0.smithi031.stdout:6/565: mknod d0/dd/d35/cb5 0 2024-09-16T08:57:20.374 INFO:tasks.workunit.client.1.smithi097.stdout:5/973: sync 2024-09-16T08:57:20.374 INFO:tasks.workunit.client.1.smithi097.stdout:6/940: sync 2024-09-16T08:57:20.374 INFO:tasks.workunit.client.1.smithi097.stdout:8/828: sync 2024-09-16T08:57:20.374 INFO:tasks.workunit.client.1.smithi097.stdout:3/891: sync 2024-09-16T08:57:20.374 INFO:tasks.workunit.client.1.smithi097.stdout:6/941: truncate d1/dc/fcb 585092 0 2024-09-16T08:57:20.375 INFO:tasks.workunit.client.0.smithi031.stdout:2/387: creat d6/da/d4b/f83 x:0 0 0 2024-09-16T08:57:20.376 INFO:tasks.workunit.client.1.smithi097.stdout:2/939: write d3/d4/d9/d67/d122/d61/dac/fd6 [2322660,117556] 0 2024-09-16T08:57:20.376 INFO:tasks.workunit.client.1.smithi097.stdout:2/940: fdatasync d3/d4/d9/f9c 0 2024-09-16T08:57:20.376 INFO:tasks.workunit.client.1.smithi097.stdout:2/941: truncate d3/d4/d8/d51/fcb 454405 0 2024-09-16T08:57:20.376 INFO:tasks.workunit.client.1.smithi097.stdout:0/968: symlink da/d13f/l155 0 2024-09-16T08:57:20.377 INFO:tasks.workunit.client.1.smithi097.stdout:0/969: write da/d7e/dc4/dfc/f59 [727290,78172] 0 2024-09-16T08:57:20.377 INFO:tasks.workunit.client.1.smithi097.stdout:0/970: chown da/d24/ff0 129172967 1 2024-09-16T08:57:20.378 INFO:tasks.workunit.client.1.smithi097.stdout:4/819: rename d0/d7/df/d15/d1f to d0/d39/d119 0 2024-09-16T08:57:20.384 INFO:tasks.workunit.client.1.smithi097.stdout:9/912: write d2/d5/db0/dc8/f32 [2496962,2405] 0 2024-09-16T08:57:20.384 INFO:tasks.workunit.client.1.smithi097.stdout:5/974: symlink db/d18/d1c/d42/d46/d7e/d8f/d102/d11b/l140 0 2024-09-16T08:57:20.384 INFO:tasks.workunit.client.0.smithi031.stdout:9/430: symlink d3/d5/d73/l9a 0 2024-09-16T08:57:20.385 INFO:tasks.workunit.client.0.smithi031.stdout:5/441: mknod d2/dc/d36/d76/c90 0 2024-09-16T08:57:20.386 INFO:tasks.workunit.client.0.smithi031.stdout:3/507: mkdir d3/d14/d37/d33/d97 0 2024-09-16T08:57:20.387 INFO:tasks.workunit.client.1.smithi097.stdout:3/892: dread d7/de/d3c/d3f/d60/fe1 [0,4194304] 0 2024-09-16T08:57:20.387 INFO:tasks.workunit.client.1.smithi097.stdout:2/942: mkdir d3/d4/d68/d104/d127 0 2024-09-16T08:57:20.387 INFO:tasks.workunit.client.1.smithi097.stdout:2/943: write d3/d4/d9/d67/d122/d7c/d7d/fc6 [3819765,53209] 0 2024-09-16T08:57:20.392 INFO:tasks.workunit.client.1.smithi097.stdout:0/971: unlink da/d7e/dc4/dfc/d94/d101/c70 0 2024-09-16T08:57:20.392 INFO:tasks.workunit.client.1.smithi097.stdout:8/829: write d8/dd/d15/d2c/d4a/d4e/f70 [132208,110161] 0 2024-09-16T08:57:20.397 INFO:tasks.workunit.client.1.smithi097.stdout:4/820: rename d0/d7/led to d0/d35/dc0/l11a 0 2024-09-16T08:57:20.397 INFO:tasks.workunit.client.0.smithi031.stdout:9/431: rmdir d3/d5/da/d34/d3c/d5e 39 2024-09-16T08:57:20.399 INFO:tasks.workunit.client.1.smithi097.stdout:5/975: link db/d18/d1c/d42/d46/lf8 db/d18/d1c/d42/d46/d7e/d8f/d102/l141 0 2024-09-16T08:57:20.400 INFO:tasks.workunit.client.0.smithi031.stdout:5/442: mkdir d2/dc/d91 0 2024-09-16T08:57:20.400 INFO:tasks.workunit.client.0.smithi031.stdout:5/443: fdatasync d2/dc/d3d/f74 0 2024-09-16T08:57:20.401 INFO:tasks.workunit.client.1.smithi097.stdout:9/913: mkdir d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/d108/d121 0 2024-09-16T08:57:20.402 INFO:tasks.workunit.client.0.smithi031.stdout:2/388: getdents d6/da/d37 0 2024-09-16T08:57:20.404 INFO:tasks.workunit.client.1.smithi097.stdout:3/893: unlink d7/de/d3c/dcc/l128 0 2024-09-16T08:57:20.405 INFO:tasks.workunit.client.1.smithi097.stdout:3/894: write d7/de/d6c/fe7 [810475,15208] 0 2024-09-16T08:57:20.405 INFO:tasks.workunit.client.1.smithi097.stdout:8/830: dread d8/dd/d71/f73 [0,4194304] 0 2024-09-16T08:57:20.405 INFO:tasks.workunit.client.0.smithi031.stdout:2/389: truncate f1 18697 0 2024-09-16T08:57:20.407 INFO:tasks.workunit.client.1.smithi097.stdout:4/821: creat d0/d35/da4/f11b x:0 0 0 2024-09-16T08:57:20.408 INFO:tasks.workunit.client.1.smithi097.stdout:5/976: mknod db/d6e/c142 0 2024-09-16T08:57:20.411 INFO:tasks.workunit.client.1.smithi097.stdout:9/914: unlink d2/d5/d72/dc1/de6/f10c 0 2024-09-16T08:57:20.411 INFO:tasks.workunit.client.1.smithi097.stdout:3/895: symlink d7/de/d9c/dca/l12f 0 2024-09-16T08:57:20.412 INFO:tasks.workunit.client.0.smithi031.stdout:5/444: dread d2/f59 [0,4194304] 0 2024-09-16T08:57:20.412 INFO:tasks.workunit.client.1.smithi097.stdout:0/972: dread da/d7e/dc4/dfc/d47/db3/dbf/f13e [0,4194304] 0 2024-09-16T08:57:20.412 INFO:tasks.workunit.client.1.smithi097.stdout:9/915: symlink d2/d5/db0/dc8/d74/l122 0 2024-09-16T08:57:20.413 INFO:tasks.workunit.client.1.smithi097.stdout:9/916: chown d2/c42 247 1 2024-09-16T08:57:20.413 INFO:tasks.workunit.client.1.smithi097.stdout:0/973: chown da/d7e/dc4/dfc/d94/l116 50185 1 2024-09-16T08:57:20.413 INFO:tasks.workunit.client.1.smithi097.stdout:9/917: chown d2/d5/d72/dc1/ddd/dee/cce 38 1 2024-09-16T08:57:20.414 INFO:tasks.workunit.client.1.smithi097.stdout:3/896: creat d7/f130 x:0 0 0 2024-09-16T08:57:20.414 INFO:tasks.workunit.client.1.smithi097.stdout:3/897: truncate d7/de/d3c/d3f/d60/dcb/d35/d3d/f112 2306060 0 2024-09-16T08:57:20.414 INFO:tasks.workunit.client.1.smithi097.stdout:3/898: fdatasync d7/de/d3c/d3f/d11c/fdc 0 2024-09-16T08:57:20.414 INFO:tasks.workunit.client.1.smithi097.stdout:3/899: write d7/de/d3c/f79 [3743669,72504] 0 2024-09-16T08:57:20.414 INFO:tasks.workunit.client.0.smithi031.stdout:5/445: getdents d2/dc/d25/d68 0 2024-09-16T08:57:20.415 INFO:tasks.workunit.client.1.smithi097.stdout:9/918: mknod d2/d5/db0/dc8/d74/de0/d110/c123 0 2024-09-16T08:57:20.416 INFO:tasks.workunit.client.0.smithi031.stdout:5/446: unlink d2/dc/c10 0 2024-09-16T08:57:20.416 INFO:tasks.workunit.client.0.smithi031.stdout:5/447: readlink d2/l84 0 2024-09-16T08:57:20.417 INFO:tasks.workunit.client.0.smithi031.stdout:8/376: sync 2024-09-16T08:57:20.418 INFO:tasks.workunit.client.1.smithi097.stdout:8/831: write d8/dd/d15/d2c/ff9 [1178106,58258] 0 2024-09-16T08:57:20.418 INFO:tasks.workunit.client.1.smithi097.stdout:3/900: mkdir d7/d15/dee/d11f/d30/d88/db3/d131 0 2024-09-16T08:57:20.418 INFO:tasks.workunit.client.1.smithi097.stdout:0/974: getdents da/d7e/dc4/dfc/d47 0 2024-09-16T08:57:20.418 INFO:tasks.workunit.client.1.smithi097.stdout:0/975: dread - da/d7e/dc4/dfc/d47/db3/d60/d73/d85/f148 zero size 2024-09-16T08:57:20.419 INFO:tasks.workunit.client.1.smithi097.stdout:9/919: rename d2/d5/fb to d2/d5/d8d/db1/f124 0 2024-09-16T08:57:20.421 INFO:tasks.workunit.client.1.smithi097.stdout:3/901: mknod d7/d15/dee/c132 0 2024-09-16T08:57:20.421 INFO:tasks.workunit.client.1.smithi097.stdout:3/902: truncate d7/de/d3c/d4f/dfd/f102 595726 0 2024-09-16T08:57:20.424 INFO:tasks.workunit.client.1.smithi097.stdout:0/976: symlink da/df/d8a/dac/l156 0 2024-09-16T08:57:20.424 INFO:tasks.workunit.client.1.smithi097.stdout:9/920: unlink d2/d5/f35 0 2024-09-16T08:57:20.425 INFO:tasks.workunit.client.1.smithi097.stdout:9/921: truncate d2/d5/d3a/ff6 417112 0 2024-09-16T08:57:20.425 INFO:tasks.workunit.client.0.smithi031.stdout:4/423: sync 2024-09-16T08:57:20.426 INFO:tasks.workunit.client.1.smithi097.stdout:3/903: symlink d7/de/d3c/d3f/d11c/d101/l133 0 2024-09-16T08:57:20.426 INFO:tasks.workunit.client.1.smithi097.stdout:3/904: write d7/de/f1e [2121410,75318] 0 2024-09-16T08:57:20.426 INFO:tasks.workunit.client.1.smithi097.stdout:3/905: truncate d7/d15/dee/d11f/d30/d69/ffe 53963 0 2024-09-16T08:57:20.427 INFO:tasks.workunit.client.1.smithi097.stdout:4/822: dread d0/d7/d52/d26/d3d/f44 [0,4194304] 0 2024-09-16T08:57:20.427 INFO:tasks.workunit.client.1.smithi097.stdout:4/823: stat d0/d36/d4c/fd7 0 2024-09-16T08:57:20.427 INFO:tasks.workunit.client.1.smithi097.stdout:4/824: fsync d0/d23/d5f/fc9 0 2024-09-16T08:57:20.427 INFO:tasks.workunit.client.1.smithi097.stdout:4/825: stat d0/fd 0 2024-09-16T08:57:20.427 INFO:tasks.workunit.client.1.smithi097.stdout:4/826: read - d0/d23/ddc/de5/fe6 zero size 2024-09-16T08:57:20.428 INFO:tasks.workunit.client.1.smithi097.stdout:0/977: creat da/d7e/dc4/dfc/d47/f157 x:0 0 0 2024-09-16T08:57:20.430 INFO:tasks.workunit.client.0.smithi031.stdout:4/424: rename d0/d2/d16/f30 to d0/d3c/d6b/f8a 0 2024-09-16T08:57:20.431 INFO:tasks.workunit.client.1.smithi097.stdout:3/906: creat d7/d15/dee/d11f/d30/f134 x:0 0 0 2024-09-16T08:57:20.431 INFO:tasks.workunit.client.1.smithi097.stdout:3/907: stat d7/de/d3c/d3f/d60/d6b/fc5 0 2024-09-16T08:57:20.433 INFO:tasks.workunit.client.1.smithi097.stdout:4/827: rename d0/d7/df/d83/df8/fd6 to d0/d7/d52/d26/d8c/db0/f11c 0 2024-09-16T08:57:20.433 INFO:tasks.workunit.client.1.smithi097.stdout:8/832: dread d8/dd/d15/d2c/d4a/d4d/d62/fc2 [0,4194304] 0 2024-09-16T08:57:20.435 INFO:tasks.workunit.client.1.smithi097.stdout:0/978: rename da/d13f/l155 to da/d7e/l158 0 2024-09-16T08:57:20.437 INFO:tasks.workunit.client.1.smithi097.stdout:0/979: symlink da/d7e/dde/l159 0 2024-09-16T08:57:20.437 INFO:tasks.workunit.client.1.smithi097.stdout:9/922: read d2/d5/f71 [2117310,27912] 0 2024-09-16T08:57:20.439 INFO:tasks.workunit.client.1.smithi097.stdout:0/980: mknod da/df/d13/c15a 0 2024-09-16T08:57:20.439 INFO:tasks.workunit.client.1.smithi097.stdout:0/981: truncate da/d7e/dc4/dfc/d47/f157 648292 0 2024-09-16T08:57:20.442 INFO:tasks.workunit.client.1.smithi097.stdout:0/982: mkdir da/d24/d4c/d80/dab/d15b 0 2024-09-16T08:57:20.442 INFO:tasks.workunit.client.1.smithi097.stdout:0/983: readlink da/d7e/dc4/dfc/d47/db3/d60/d73/d85/le5 0 2024-09-16T08:57:20.442 INFO:tasks.workunit.client.1.smithi097.stdout:0/984: chown da/df/d13/c17 2 1 2024-09-16T08:57:20.442 INFO:tasks.workunit.client.1.smithi097.stdout:0/985: write da/d24/d4b/dfb/f122 [723747,59130] 0 2024-09-16T08:57:20.442 INFO:tasks.workunit.client.1.smithi097.stdout:0/986: creat da/d7e/dc4/dfc/d47/f15c x:0 0 0 2024-09-16T08:57:20.442 INFO:tasks.workunit.client.1.smithi097.stdout:0/987: truncate da/d7e/f11c 202196 0 2024-09-16T08:57:20.442 INFO:tasks.workunit.client.1.smithi097.stdout:0/988: chown da/d7e/dc4/dfc/d47/db3/dbf/d12b/d84 936936 1 2024-09-16T08:57:20.443 INFO:tasks.workunit.client.1.smithi097.stdout:6/942: dwrite d1/dd3/d9e/dca/d16/fdd [0,4194304] 0 2024-09-16T08:57:20.443 INFO:tasks.workunit.client.0.smithi031.stdout:9/432: dwrite d3/d5/da/d34/d3c/d5e/d62/d65/f93 [0,4194304] 0 2024-09-16T08:57:20.443 INFO:tasks.workunit.client.0.smithi031.stdout:9/433: write d3/d5/f6 [4397279,26337] 0 2024-09-16T08:57:20.443 INFO:tasks.workunit.client.0.smithi031.stdout:1/455: dwrite d2/d15/d32/f88 [0,4194304] 0 2024-09-16T08:57:20.448 INFO:tasks.workunit.client.1.smithi097.stdout:8/833: write d8/dd/d15/d16/f3c [3334236,12457] 0 2024-09-16T08:57:20.449 INFO:tasks.workunit.client.1.smithi097.stdout:4/828: write d0/d23/f9f [3851592,84286] 0 2024-09-16T08:57:20.449 INFO:tasks.workunit.client.1.smithi097.stdout:4/829: readlink d0/d35/d6c/lc7 0 2024-09-16T08:57:20.449 INFO:tasks.workunit.client.1.smithi097.stdout:6/943: symlink d1/dd3/d9e/dca/d81/dc4/df7/l13d 0 2024-09-16T08:57:20.451 INFO:tasks.workunit.client.1.smithi097.stdout:8/834: symlink d8/dd/d15/d2c/d4a/d4e/d85/l106 0 2024-09-16T08:57:20.453 INFO:tasks.workunit.client.1.smithi097.stdout:4/830: truncate d0/fd 4044512 0 2024-09-16T08:57:20.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/944: rename d1/dd3/d9e/dca/db6/f139 to d1/dc/f13e 0 2024-09-16T08:57:20.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/945: chown d1/dd3 4075 1 2024-09-16T08:57:20.456 INFO:tasks.workunit.client.1.smithi097.stdout:6/946: chown d1/dd3/d9e/dca/d81/dc4/dc6/f11f 1759529 1 2024-09-16T08:57:20.458 INFO:tasks.workunit.client.1.smithi097.stdout:8/835: mkdir d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/df5/d107 0 2024-09-16T08:57:20.458 INFO:tasks.workunit.client.0.smithi031.stdout:3/508: dwrite d3/d14/d37/d1a/d1d/d3b/f4f [0,4194304] 0 2024-09-16T08:57:20.458 INFO:tasks.workunit.client.0.smithi031.stdout:9/434: write d3/d5/da/db/d21/d3b/f79 [840628,106489] 0 2024-09-16T08:57:20.458 INFO:tasks.workunit.client.0.smithi031.stdout:1/456: truncate d2/d4/f5 1913512 0 2024-09-16T08:57:20.458 INFO:tasks.workunit.client.0.smithi031.stdout:1/457: write d2/de/f2f [4497990,1345] 0 2024-09-16T08:57:20.459 INFO:tasks.workunit.client.1.smithi097.stdout:2/944: dwrite d3/d4/d9/d67/d122/fb7 [0,4194304] 0 2024-09-16T08:57:20.459 INFO:tasks.workunit.client.1.smithi097.stdout:6/947: symlink d1/dc/l13f 0 2024-09-16T08:57:20.459 INFO:tasks.workunit.client.1.smithi097.stdout:6/948: chown d1/dd3/fb4 8916 1 2024-09-16T08:57:20.459 INFO:tasks.workunit.client.1.smithi097.stdout:6/949: dread - d1/dc/f80 zero size 2024-09-16T08:57:20.459 INFO:tasks.workunit.client.1.smithi097.stdout:6/950: write d1/dc/f72 [710201,23507] 0 2024-09-16T08:57:20.461 INFO:tasks.workunit.client.1.smithi097.stdout:8/836: creat d8/dd/d71/d9c/f108 x:0 0 0 2024-09-16T08:57:20.462 INFO:tasks.workunit.client.0.smithi031.stdout:9/435: truncate d3/d5/da/db/f39 496500 0 2024-09-16T08:57:20.462 INFO:tasks.workunit.client.0.smithi031.stdout:9/436: dread - d3/d5/d6a/f76 zero size 2024-09-16T08:57:20.462 INFO:tasks.workunit.client.0.smithi031.stdout:9/437: creat d3/d5/d73/f9b x:0 0 0 2024-09-16T08:57:20.467 INFO:tasks.workunit.client.0.smithi031.stdout:9/438: rename d3/d5/da/db/d21/d3b/d80 to d3/d5/da/d34/d3c/d5e/d9c 0 2024-09-16T08:57:20.467 INFO:tasks.workunit.client.0.smithi031.stdout:2/390: read f1 [7826,126188] 0 2024-09-16T08:57:20.468 INFO:tasks.workunit.client.0.smithi031.stdout:2/391: creat d6/da/f84 x:0 0 0 2024-09-16T08:57:20.468 INFO:tasks.workunit.client.0.smithi031.stdout:2/392: creat d6/da/f85 x:0 0 0 2024-09-16T08:57:20.470 INFO:tasks.workunit.client.0.smithi031.stdout:3/509: write d3/d14/d37/d33/f6d [3408175,13578] 0 2024-09-16T08:57:20.471 INFO:tasks.workunit.client.1.smithi097.stdout:2/945: write d3/d4/d8/d51/f63 [2585549,122781] 0 2024-09-16T08:57:20.473 INFO:tasks.workunit.client.1.smithi097.stdout:6/951: link d1/dc/f7d d1/dc/d3c/f140 0 2024-09-16T08:57:20.474 INFO:tasks.workunit.client.1.smithi097.stdout:5/977: dwrite db/d18/d1c/d42/d46/d7e/d64/d7f/d89/fc3 [0,4194304] 0 2024-09-16T08:57:20.474 INFO:tasks.workunit.client.1.smithi097.stdout:3/908: dwrite d7/de/d3c/d3f/d60/dcb/d35/f67 [0,4194304] 0 2024-09-16T08:57:20.474 INFO:tasks.workunit.client.0.smithi031.stdout:8/377: dwrite d4/d7/d21/d35/f5d [0,4194304] 0 2024-09-16T08:57:20.474 INFO:tasks.workunit.client.0.smithi031.stdout:4/425: dwrite d0/d2/d16/d1e/f34 [0,4194304] 0 2024-09-16T08:57:20.475 INFO:tasks.workunit.client.0.smithi031.stdout:9/439: creat d3/d48/f9d x:0 0 0 2024-09-16T08:57:20.475 INFO:tasks.workunit.client.1.smithi097.stdout:0/989: dwrite da/d1a/d41/db2/f135 [0,4194304] 0 2024-09-16T08:57:20.475 INFO:tasks.workunit.client.1.smithi097.stdout:0/990: fdatasync da/d7e/dc4/fe0 0 2024-09-16T08:57:20.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/448: dwrite d2/d4b/f8a [0,4194304] 0 2024-09-16T08:57:20.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/449: chown d2/dc/d25/d21/d2e/c39 571 1 2024-09-16T08:57:20.476 INFO:tasks.workunit.client.0.smithi031.stdout:5/450: write d2/dc/f19 [5052868,36014] 0 2024-09-16T08:57:20.477 INFO:tasks.workunit.client.1.smithi097.stdout:8/837: unlink d8/dd/d15/d2c/d4a/d4d/d3d/db9/fed 0 2024-09-16T08:57:20.481 INFO:tasks.workunit.client.1.smithi097.stdout:2/946: mknod d3/d4/d9/d67/d122/d7c/d7d/d53/da3/de8/c128 0 2024-09-16T08:57:20.481 INFO:tasks.workunit.client.1.smithi097.stdout:2/947: fdatasync d3/d4/d9/d67/fd0 0 2024-09-16T08:57:20.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/393: rename d6/da/d5e/d72/l7d to d6/d2e/d40/d6c/l86 0 2024-09-16T08:57:20.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/394: getdents d6/da/d37 0 2024-09-16T08:57:20.486 INFO:tasks.workunit.client.0.smithi031.stdout:3/510: rmdir d3/d14/d37/d1a/d1d/d3b 39 2024-09-16T08:57:20.487 INFO:tasks.workunit.client.1.smithi097.stdout:6/952: dread d1/dc/f38 [0,4194304] 0 2024-09-16T08:57:20.487 INFO:tasks.workunit.client.1.smithi097.stdout:4/831: dwrite d0/d7/df/d83/df8/f97 [0,4194304] 0 2024-09-16T08:57:20.487 INFO:tasks.workunit.client.1.smithi097.stdout:4/832: creat d0/d36/da7/f11d x:0 0 0 2024-09-16T08:57:20.490 INFO:tasks.workunit.client.0.smithi031.stdout:8/378: dread d4/f17 [0,4194304] 0 2024-09-16T08:57:20.491 INFO:tasks.workunit.client.1.smithi097.stdout:0/991: dwrite da/d1a/d41/db2/f11d [0,4194304] 0 2024-09-16T08:57:20.491 INFO:tasks.workunit.client.1.smithi097.stdout:0/992: truncate da/d1a/f8e 873955 0 2024-09-16T08:57:20.492 INFO:tasks.workunit.client.0.smithi031.stdout:5/451: dread d2/fa [0,4194304] 0 2024-09-16T08:57:20.492 INFO:tasks.workunit.client.0.smithi031.stdout:5/452: chown d2/f2b 72145053 1 2024-09-16T08:57:20.493 INFO:tasks.workunit.client.1.smithi097.stdout:9/923: getdents d2/d5 0 2024-09-16T08:57:20.493 INFO:tasks.workunit.client.1.smithi097.stdout:9/924: write d2/d5/d3a/d3c/fc6 [1394326,53746] 0 2024-09-16T08:57:20.493 INFO:tasks.workunit.client.1.smithi097.stdout:9/925: dread - d2/d5/d72/dc1/f11d zero size 2024-09-16T08:57:20.493 INFO:tasks.workunit.client.1.smithi097.stdout:3/909: symlink d7/d15/dee/d11f/d30/d69/l135 0 2024-09-16T08:57:20.493 INFO:tasks.workunit.client.1.smithi097.stdout:3/910: write d7/de/d3c/d3f/d60/dcb/fd5 [714005,68886] 0 2024-09-16T08:57:20.494 INFO:tasks.workunit.client.0.smithi031.stdout:4/426: symlink d0/d2/l8b 0 2024-09-16T08:57:20.494 INFO:tasks.workunit.client.0.smithi031.stdout:4/427: chown d0/l5 3708896 1 2024-09-16T08:57:20.503 INFO:tasks.workunit.client.1.smithi097.stdout:3/911: dread d7/de/d3c/d3f/d11c/fdc [0,4194304] 0 2024-09-16T08:57:20.504 INFO:tasks.workunit.client.1.smithi097.stdout:8/838: getdents d8/dd/d15/d2c/d4a/d4d 0 2024-09-16T08:57:20.505 INFO:tasks.workunit.client.0.smithi031.stdout:2/395: creat d6/da/d37/f87 x:0 0 0 2024-09-16T08:57:20.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/441: sync 2024-09-16T08:57:20.506 INFO:tasks.workunit.client.0.smithi031.stdout:7/485: sync 2024-09-16T08:57:20.506 INFO:tasks.workunit.client.0.smithi031.stdout:2/396: dread - d6/da/d5e/f70 zero size 2024-09-16T08:57:20.506 INFO:tasks.workunit.client.0.smithi031.stdout:0/442: readlink d6/d12/d47/d5b/l5c 0 2024-09-16T08:57:20.506 INFO:tasks.workunit.client.0.smithi031.stdout:7/486: dread - d2/d2d/d68/f6b zero size 2024-09-16T08:57:20.506 INFO:tasks.workunit.client.0.smithi031.stdout:7/487: dread - d2/d8/d11/d81/f9f zero size 2024-09-16T08:57:20.514 INFO:tasks.workunit.client.0.smithi031.stdout:2/397: dread d6/d20/f4f [4194304,4194304] 0 2024-09-16T08:57:20.514 INFO:tasks.workunit.client.0.smithi031.stdout:2/398: write d6/da/f3d [1611718,2873] 0 2024-09-16T08:57:20.514 INFO:tasks.workunit.client.0.smithi031.stdout:2/399: fsync d6/f16 0 2024-09-16T08:57:20.518 INFO:tasks.workunit.client.0.smithi031.stdout:7/488: write d2/d2d/f77 [1746095,103073] 0 2024-09-16T08:57:20.518 INFO:tasks.workunit.client.0.smithi031.stdout:3/511: dwrite d3/d14/d37/f10 [0,4194304] 0 2024-09-16T08:57:20.518 INFO:tasks.workunit.client.0.smithi031.stdout:3/512: chown d3/d14/d37/l78 3925299 1 2024-09-16T08:57:20.518 INFO:tasks.workunit.client.0.smithi031.stdout:3/513: read d3/d14/d37/d1a/f26 [842757,110928] 0 2024-09-16T08:57:20.525 INFO:tasks.workunit.client.0.smithi031.stdout:3/514: read d3/d14/f2e [2990503,102560] 0 2024-09-16T08:57:20.525 INFO:tasks.workunit.client.0.smithi031.stdout:3/515: stat d3/d14/d37/d47/f5b 0 2024-09-16T08:57:20.538 INFO:tasks.workunit.client.0.smithi031.stdout:2/400: dwrite d6/d2e/d40/d6c/d2c/d53/d60/f35 [0,4194304] 0 2024-09-16T08:57:20.538 INFO:tasks.workunit.client.0.smithi031.stdout:7/489: dwrite d2/d8/f50 [4194304,4194304] 0 2024-09-16T08:57:20.538 INFO:tasks.workunit.client.0.smithi031.stdout:7/490: creat d2/d8/d11/d81/fa5 x:0 0 0 2024-09-16T08:57:20.576 INFO:tasks.workunit.client.0.smithi031.stdout:8/379: rename d4/d7/l6f to d4/d7/d21/d24/l83 0 2024-09-16T08:57:20.576 INFO:tasks.workunit.client.0.smithi031.stdout:8/380: chown d4/d7/d36/l4e 109 1 2024-09-16T08:57:20.578 INFO:tasks.workunit.client.0.smithi031.stdout:5/453: creat d2/dc/d1d/f92 x:0 0 0 2024-09-16T08:57:20.580 INFO:tasks.workunit.client.0.smithi031.stdout:4/428: link d0/d2/d16/c1d d0/d3c/c8c 0 2024-09-16T08:57:20.583 INFO:tasks.workunit.client.1.smithi097.stdout:5/978: getdents db/d18 0 2024-09-16T08:57:20.588 INFO:tasks.workunit.client.1.smithi097.stdout:6/953: mkdir d1/db7/dfd/d141 0 2024-09-16T08:57:20.592 INFO:tasks.workunit.client.0.smithi031.stdout:3/516: link d3/d14/d37/d1a/d1d/f75 d3/d14/d6f/d82/d96/f98 0 2024-09-16T08:57:20.599 INFO:tasks.workunit.client.1.smithi097.stdout:5/979: dwrite db/d18/d1c/f136 [0,4194304] 0 2024-09-16T08:57:20.601 INFO:tasks.workunit.client.0.smithi031.stdout:8/381: mknod d4/d7/d36/d5e/c84 0 2024-09-16T08:57:20.603 INFO:tasks.workunit.client.0.smithi031.stdout:4/429: mkdir d0/d2/d16/d59/d8d 0 2024-09-16T08:57:20.605 INFO:tasks.workunit.client.0.smithi031.stdout:0/443: symlink d6/d12/d13/l8e 0 2024-09-16T08:57:20.610 INFO:tasks.workunit.client.0.smithi031.stdout:3/517: creat d3/d14/d93/f99 x:0 0 0 2024-09-16T08:57:20.611 INFO:tasks.workunit.client.0.smithi031.stdout:8/382: symlink d4/d7/l85 0 2024-09-16T08:57:20.613 INFO:tasks.workunit.client.0.smithi031.stdout:0/444: symlink d6/d12/d13/d36/l8f 0 2024-09-16T08:57:20.613 INFO:tasks.workunit.client.0.smithi031.stdout:0/445: dread - d6/d12/d47/f83 zero size 2024-09-16T08:57:20.613 INFO:tasks.workunit.client.0.smithi031.stdout:0/446: fdatasync d6/f9 0 2024-09-16T08:57:20.613 INFO:tasks.workunit.client.0.smithi031.stdout:3/518: dread d3/d14/d37/d1a/d1d/d3b/d3f/f62 [4194304,4194304] 0 2024-09-16T08:57:20.614 INFO:tasks.workunit.client.0.smithi031.stdout:3/519: write d3/d14/d37/d1a/d1d/f92 [514235,89863] 0 2024-09-16T08:57:20.614 INFO:tasks.workunit.client.1.smithi097.stdout:4/833: mkdir d0/d7/d52/d26/d8c/d11e 0 2024-09-16T08:57:20.614 INFO:tasks.workunit.client.1.smithi097.stdout:4/834: chown d0/d39/d119/cb4 1708054 1 2024-09-16T08:57:20.615 INFO:tasks.workunit.client.0.smithi031.stdout:8/383: rename d4/d7/d21/d35/d40/f4c to d4/d7/d21/d24/f86 0 2024-09-16T08:57:20.618 INFO:tasks.workunit.client.0.smithi031.stdout:3/520: mkdir d3/d14/d37/d1a/d1d/d9a 0 2024-09-16T08:57:20.618 INFO:tasks.workunit.client.0.smithi031.stdout:3/521: getdents d3 0 2024-09-16T08:57:20.620 INFO:tasks.workunit.client.1.smithi097.stdout:0/993: creat da/d24/d4c/f15d x:0 0 0 2024-09-16T08:57:20.620 INFO:tasks.workunit.client.1.smithi097.stdout:0/994: dread - da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dad/fec zero size 2024-09-16T08:57:20.620 INFO:tasks.workunit.client.1.smithi097.stdout:0/995: truncate da/d24/d4b/dfb/f122 1002942 0 2024-09-16T08:57:20.621 INFO:tasks.workunit.client.0.smithi031.stdout:3/522: truncate d3/f4 3651983 0 2024-09-16T08:57:20.622 INFO:tasks.workunit.client.0.smithi031.stdout:7/491: dwrite d2/d8/d11/f24 [4194304,4194304] 0 2024-09-16T08:57:20.623 INFO:tasks.workunit.client.0.smithi031.stdout:4/430: dwrite d0/d2/d16/f4e [0,4194304] 0 2024-09-16T08:57:20.623 INFO:tasks.workunit.client.1.smithi097.stdout:9/926: creat d2/d5/db0/dc8/d74/d8c/d94/f125 x:0 0 0 2024-09-16T08:57:20.625 INFO:tasks.workunit.client.1.smithi097.stdout:3/912: rename d7/de/l24 to d7/de/d3c/d3f/d11c/d101/l136 0 2024-09-16T08:57:20.625 INFO:tasks.workunit.client.1.smithi097.stdout:6/954: rename d1/dd3 to d1/dd3/d9e/dca/d81/dc4/ddc/d142 22 2024-09-16T08:57:20.628 INFO:tasks.workunit.client.0.smithi031.stdout:3/523: creat d3/d14/d37/d1a/f9b x:0 0 0 2024-09-16T08:57:20.628 INFO:tasks.workunit.client.0.smithi031.stdout:3/524: fsync d3/f59 0 2024-09-16T08:57:20.628 INFO:tasks.workunit.client.0.smithi031.stdout:3/525: fsync d3/d14/d37/d1a/d1d/d3b/d3f/f74 0 2024-09-16T08:57:20.628 INFO:tasks.workunit.client.0.smithi031.stdout:3/526: chown d3/l64 13 1 2024-09-16T08:57:20.628 INFO:tasks.workunit.client.0.smithi031.stdout:3/527: readlink d3/d14/d37/d33/l4b 0 2024-09-16T08:57:20.630 INFO:tasks.workunit.client.0.smithi031.stdout:8/384: dread d4/f27 [0,4194304] 0 2024-09-16T08:57:20.631 INFO:tasks.workunit.client.1.smithi097.stdout:8/839: mkdir d8/dd/d15/d2c/d4a/df3/d109 0 2024-09-16T08:57:20.633 INFO:tasks.workunit.client.0.smithi031.stdout:4/431: truncate d0/d2/f3d 466432 0 2024-09-16T08:57:20.635 INFO:tasks.workunit.client.1.smithi097.stdout:9/927: dread d2/f4 [0,4194304] 0 2024-09-16T08:57:20.635 INFO:tasks.workunit.client.1.smithi097.stdout:9/928: write f0 [1251904,28159] 0 2024-09-16T08:57:20.636 INFO:tasks.workunit.client.0.smithi031.stdout:3/528: creat d3/d14/d37/d1a/d1d/d3b/d60/f9c x:0 0 0 2024-09-16T08:57:20.638 INFO:tasks.workunit.client.0.smithi031.stdout:0/447: dwrite d6/f57 [0,4194304] 0 2024-09-16T08:57:20.638 INFO:tasks.workunit.client.0.smithi031.stdout:0/448: write d6/d12/d13/d3c/f45 [3781484,126671] 0 2024-09-16T08:57:20.638 INFO:tasks.workunit.client.0.smithi031.stdout:8/385: creat d4/d7/d36/d38/f87 x:0 0 0 2024-09-16T08:57:20.640 INFO:tasks.workunit.client.0.smithi031.stdout:7/492: dwrite d2/d2d/d68/f6b [0,4194304] 0 2024-09-16T08:57:20.640 INFO:tasks.workunit.client.0.smithi031.stdout:7/493: chown d2/d2d/l5f 1934 1 2024-09-16T08:57:20.640 INFO:tasks.workunit.client.0.smithi031.stdout:7/494: chown d2/d8/f60 1503984 1 2024-09-16T08:57:20.640 INFO:tasks.workunit.client.0.smithi031.stdout:7/495: truncate d2/f6a 723587 0 2024-09-16T08:57:20.641 INFO:tasks.workunit.client.0.smithi031.stdout:4/432: rename d0/d2/d6d to d0/d8e 0 2024-09-16T08:57:20.643 INFO:tasks.workunit.client.1.smithi097.stdout:0/996: symlink da/d7e/dc4/dfc/d47/db3/d60/d73/d85/dad/l15e 0 2024-09-16T08:57:20.645 INFO:tasks.workunit.client.0.smithi031.stdout:8/386: unlink d4/d2d/l2e 0 2024-09-16T08:57:20.648 INFO:tasks.workunit.client.1.smithi097.stdout:5/980: rename db/d22/c73 to db/d18/d1c/d42/d46/d7e/d64/d8d/c143 0 2024-09-16T08:57:20.649 INFO:tasks.workunit.client.0.smithi031.stdout:3/529: write d3/d14/d37/d1a/d1d/d3b/d3f/f74 [1577640,111623] 0 2024-09-16T08:57:20.650 INFO:tasks.workunit.client.1.smithi097.stdout:3/913: mkdir d7/d15/d137 0 2024-09-16T08:57:20.651 INFO:tasks.workunit.client.1.smithi097.stdout:8/840: mknod d8/dd/d71/d9c/c10a 0 2024-09-16T08:57:20.653 INFO:tasks.workunit.client.1.smithi097.stdout:9/929: mkdir d2/d5/d72/dc1/ddd/d126 0 2024-09-16T08:57:20.654 INFO:tasks.workunit.client.0.smithi031.stdout:8/387: dread d4/d7/d36/d38/f3c [0,4194304] 0 2024-09-16T08:57:20.656 INFO:tasks.workunit.client.1.smithi097.stdout:9/930: write d2/d5/db0/dc8/d74/d8c/feb [1024459,57161] 0 2024-09-16T08:57:20.657 INFO:tasks.workunit.client.1.smithi097.stdout:0/997: mkdir da/df/d8a/d15f 0 2024-09-16T08:57:20.660 INFO:tasks.workunit.client.1.smithi097.stdout:6/955: rename d1/dd3/d9e/dca/d81/dc4/df7/cfb to d1/c143 0 2024-09-16T08:57:20.660 INFO:tasks.workunit.client.1.smithi097.stdout:5/981: mkdir db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/d144 0 2024-09-16T08:57:20.661 INFO:tasks.workunit.client.0.smithi031.stdout:0/449: dwrite d6/f1a [0,4194304] 0 2024-09-16T08:57:20.664 INFO:tasks.workunit.client.1.smithi097.stdout:3/914: symlink d7/d15/dee/d11f/d63/dd8/l138 0 2024-09-16T08:57:20.664 INFO:tasks.workunit.client.1.smithi097.stdout:3/915: creat d7/d15/d7d/f139 x:0 0 0 2024-09-16T08:57:20.666 INFO:tasks.workunit.client.1.smithi097.stdout:9/931: mkdir d2/d3f/d127 0 2024-09-16T08:57:20.667 INFO:tasks.workunit.client.1.smithi097.stdout:9/932: chown d2/d5/d3a/d3c/f85 100055 1 2024-09-16T08:57:20.667 INFO:tasks.workunit.client.0.smithi031.stdout:7/496: symlink d2/d8/d11/d5d/la6 0 2024-09-16T08:57:20.671 INFO:tasks.workunit.client.1.smithi097.stdout:5/982: truncate db/d18/d20/d2b/f87 3571372 0 2024-09-16T08:57:20.673 INFO:tasks.workunit.client.0.smithi031.stdout:4/433: creat d0/d2/d16/d59/d84/d86/f8f x:0 0 0 2024-09-16T08:57:20.673 INFO:tasks.workunit.client.0.smithi031.stdout:4/434: write d0/d3c/d6b/f5e [4060280,86759] 0 2024-09-16T08:57:20.673 INFO:tasks.workunit.client.0.smithi031.stdout:4/435: write d0/d3c/f7f [730313,60468] 0 2024-09-16T08:57:20.678 INFO:tasks.workunit.client.0.smithi031.stdout:8/388: mkdir d4/d7/d21/d37/d46/d88 0 2024-09-16T08:57:20.681 INFO:tasks.workunit.client.0.smithi031.stdout:7/497: symlink d2/d8/d11/d81/la7 0 2024-09-16T08:57:20.681 INFO:tasks.workunit.client.0.smithi031.stdout:7/498: write d2/fb [4493559,40582] 0 2024-09-16T08:57:20.681 INFO:tasks.workunit.client.1.smithi097.stdout:3/916: creat d7/d15/dee/d11f/d30/d88/db3/f13a x:0 0 0 2024-09-16T08:57:20.683 INFO:tasks.workunit.client.0.smithi031.stdout:8/389: rename d4/d7/d21/d35/d40/l55 to d4/d7/d66/l89 0 2024-09-16T08:57:20.685 INFO:tasks.workunit.client.1.smithi097.stdout:9/933: mkdir d2/d5/db0/dc8/d63/da8/dca/d128 0 2024-09-16T08:57:20.685 INFO:tasks.workunit.client.1.smithi097.stdout:8/841: dwrite d8/dd/d42/f9e [0,4194304] 0 2024-09-16T08:57:20.685 INFO:tasks.workunit.client.1.smithi097.stdout:8/842: readlink d8/dd/d15/d2c/d4a/d4d/d3f/lee 0 2024-09-16T08:57:20.685 INFO:tasks.workunit.client.1.smithi097.stdout:8/843: write d8/dd/d15/d2c/f74 [1371470,40990] 0 2024-09-16T08:57:20.686 INFO:tasks.workunit.client.0.smithi031.stdout:3/530: dwrite d3/d14/d37/d1a/d1d/d3b/f4f [0,4194304] 0 2024-09-16T08:57:20.686 INFO:tasks.workunit.client.1.smithi097.stdout:5/983: symlink db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90/l145 0 2024-09-16T08:57:20.689 INFO:tasks.workunit.client.0.smithi031.stdout:7/499: creat d2/d2d/d6d/fa8 x:0 0 0 2024-09-16T08:57:20.691 INFO:tasks.workunit.client.0.smithi031.stdout:8/390: rmdir d4/d5b 39 2024-09-16T08:57:20.695 INFO:tasks.workunit.client.1.smithi097.stdout:8/844: creat d8/dd/d42/f10b x:0 0 0 2024-09-16T08:57:20.696 INFO:tasks.workunit.client.1.smithi097.stdout:9/934: dread d2/d5/d72/f89 [0,4194304] 0 2024-09-16T08:57:20.696 INFO:tasks.workunit.client.1.smithi097.stdout:9/935: readlink d2/d5/l83 0 2024-09-16T08:57:20.696 INFO:tasks.workunit.client.1.smithi097.stdout:8/845: dread - d8/dd/d15/d16/d7c/da2/fdc zero size 2024-09-16T08:57:20.696 INFO:tasks.workunit.client.1.smithi097.stdout:8/846: fsync d8/fad 0 2024-09-16T08:57:20.696 INFO:tasks.workunit.client.0.smithi031.stdout:0/450: dwrite d6/d12/d47/f83 [0,4194304] 0 2024-09-16T08:57:20.697 INFO:tasks.workunit.client.1.smithi097.stdout:3/917: write d7/de/d6c/f37 [1785882,120981] 0 2024-09-16T08:57:20.702 INFO:tasks.workunit.client.1.smithi097.stdout:7/831: sync 2024-09-16T08:57:20.702 INFO:tasks.workunit.client.1.smithi097.stdout:7/832: readlink d4/d9/l73 0 2024-09-16T08:57:20.703 INFO:tasks.workunit.client.1.smithi097.stdout:9/936: mkdir d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d129 0 2024-09-16T08:57:20.704 INFO:tasks.workunit.client.1.smithi097.stdout:9/937: read d2/d5/f29 [55321,42943] 0 2024-09-16T08:57:20.704 INFO:tasks.workunit.client.1.smithi097.stdout:9/938: chown d2/d5/db0/d11f 232 1 2024-09-16T08:57:20.704 INFO:tasks.workunit.client.1.smithi097.stdout:2/948: sync 2024-09-16T08:57:20.706 INFO:tasks.workunit.client.1.smithi097.stdout:8/847: mknod d8/dd/d15/d2c/d4a/d4d/d3d/c10c 0 2024-09-16T08:57:20.711 INFO:tasks.workunit.client.1.smithi097.stdout:7/833: truncate d4/d1b/d41/f95 3219979 0 2024-09-16T08:57:20.712 INFO:tasks.workunit.client.1.smithi097.stdout:0/998: dwrite da/d7e/f96 [0,4194304] 0 2024-09-16T08:57:20.712 INFO:tasks.workunit.client.1.smithi097.stdout:0/999: chown da/d1a/d9c/le2 48497 1 2024-09-16T08:57:20.712 INFO:tasks.workunit.client.0.smithi031.stdout:7/500: dread d2/d8/f4a [0,4194304] 0 2024-09-16T08:57:20.712 INFO:tasks.workunit.client.0.smithi031.stdout:7/501: chown d2/d8/d11/fa4 100094 1 2024-09-16T08:57:20.715 INFO:tasks.workunit.client.0.smithi031.stdout:4/436: dwrite d0/f2e [0,4194304] 0 2024-09-16T08:57:20.716 INFO:tasks.workunit.client.0.smithi031.stdout:8/391: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:20.716 INFO:tasks.workunit.client.0.smithi031.stdout:4/437: chown d0/d8e/d75 0 1 2024-09-16T08:57:20.716 INFO:tasks.workunit.client.1.smithi097.stdout:2/949: dread d3/d4/d9/d67/d122/d61/fb1 [0,4194304] 0 2024-09-16T08:57:20.717 INFO:tasks.workunit.client.0.smithi031.stdout:4/438: read d0/d3c/f62 [447770,11679] 0 2024-09-16T08:57:20.717 INFO:tasks.workunit.client.0.smithi031.stdout:4/439: truncate d0/d5b/f7d 649406 0 2024-09-16T08:57:20.717 INFO:tasks.workunit.client.1.smithi097.stdout:6/956: dwrite d1/f122 [0,4194304] 0 2024-09-16T08:57:20.723 INFO:tasks.workunit.client.1.smithi097.stdout:8/848: dread d8/dd/d15/d2c/d4a/d4d/d3d/d7b/f84 [0,4194304] 0 2024-09-16T08:57:20.724 INFO:tasks.workunit.client.1.smithi097.stdout:2/950: unlink d3/d1b/dbf/l110 0 2024-09-16T08:57:20.724 INFO:tasks.workunit.client.1.smithi097.stdout:2/951: write d3/d4/d9/d67/f92 [807374,57104] 0 2024-09-16T08:57:20.727 INFO:tasks.workunit.client.0.smithi031.stdout:8/392: dread d4/d7/d21/d24/f86 [0,4194304] 0 2024-09-16T08:57:20.727 INFO:tasks.workunit.client.0.smithi031.stdout:3/531: dwrite d3/d14/d37/d33/f6d [0,4194304] 0 2024-09-16T08:57:20.727 INFO:tasks.workunit.client.0.smithi031.stdout:3/532: write d3/d14/d37/d47/f66 [1478262,68474] 0 2024-09-16T08:57:20.729 INFO:tasks.workunit.client.0.smithi031.stdout:8/393: truncate d4/d7/d21/d37/f76 995801 0 2024-09-16T08:57:20.731 INFO:tasks.workunit.client.1.smithi097.stdout:6/957: mkdir d1/dd3/d9e/dca/db6/d144 0 2024-09-16T08:57:20.731 INFO:tasks.workunit.client.1.smithi097.stdout:6/958: chown d1/dd3/d9e/dca/d81/dad/f104 8597842 1 2024-09-16T08:57:20.734 INFO:tasks.workunit.client.1.smithi097.stdout:7/834: rename d4/d1b/da4/de0 to d4/d9/d2b/d11b 0 2024-09-16T08:57:20.734 INFO:tasks.workunit.client.1.smithi097.stdout:7/835: write d4/d9/dc/d8e/fa2 [444258,9905] 0 2024-09-16T08:57:20.734 INFO:tasks.workunit.client.1.smithi097.stdout:7/836: read - d4/d7/d7b/d43/daf/f108 zero size 2024-09-16T08:57:20.737 INFO:tasks.workunit.client.1.smithi097.stdout:2/952: symlink d3/d4/d9/d67/d122/d7c/d7d/d53/da3/l129 0 2024-09-16T08:57:20.737 INFO:tasks.workunit.client.1.smithi097.stdout:2/953: chown d3/d4/fc8 124876529 1 2024-09-16T08:57:20.737 INFO:tasks.workunit.client.1.smithi097.stdout:2/954: read - d3/d4/d9/d67/d122/f10d zero size 2024-09-16T08:57:20.737 INFO:tasks.workunit.client.1.smithi097.stdout:2/955: truncate d3/d4/d8/fdd 576340 0 2024-09-16T08:57:20.737 INFO:tasks.workunit.client.1.smithi097.stdout:2/956: truncate d3/d4/d9/d67/d122/f10d 4862 0 2024-09-16T08:57:20.738 INFO:tasks.workunit.client.0.smithi031.stdout:7/502: getdents d2/d8/d11/d81 0 2024-09-16T08:57:20.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/394: creat d4/d7/d21/d35/d79/f8a x:0 0 0 2024-09-16T08:57:20.740 INFO:tasks.workunit.client.1.smithi097.stdout:7/837: rename d4/d12/c6e to d4/d9/d51/c11c 0 2024-09-16T08:57:20.741 INFO:tasks.workunit.client.0.smithi031.stdout:7/503: mkdir d2/d8/d86/da1/da9 0 2024-09-16T08:57:20.743 INFO:tasks.workunit.client.1.smithi097.stdout:2/957: symlink d3/d4/d8/d51/l12a 0 2024-09-16T08:57:20.744 INFO:tasks.workunit.client.1.smithi097.stdout:3/918: dwrite d7/de/d9c/daf/f117 [0,4194304] 0 2024-09-16T08:57:20.745 INFO:tasks.workunit.client.0.smithi031.stdout:4/440: dread d0/d2/d16/d2b/f74 [0,4194304] 0 2024-09-16T08:57:20.745 INFO:tasks.workunit.client.0.smithi031.stdout:4/441: chown d0/f4c 542 1 2024-09-16T08:57:20.745 INFO:tasks.workunit.client.0.smithi031.stdout:4/442: write d0/f38 [5436294,1198] 0 2024-09-16T08:57:20.745 INFO:tasks.workunit.client.0.smithi031.stdout:4/443: read d0/d2/d16/d59/f63 [55662,104754] 0 2024-09-16T08:57:20.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/395: truncate d4/d7/d21/d35/d40/f58 3206435 0 2024-09-16T08:57:20.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/396: read d4/d5b/f62 [443779,87186] 0 2024-09-16T08:57:20.747 INFO:tasks.workunit.client.1.smithi097.stdout:7/838: rename d4/d7/d7b/f7c to d4/d7/d19/f11d 0 2024-09-16T08:57:20.748 INFO:tasks.workunit.client.0.smithi031.stdout:7/504: mknod d2/d8/d11/d5d/d6e/caa 0 2024-09-16T08:57:20.748 INFO:tasks.workunit.client.0.smithi031.stdout:7/505: chown d2/d8/f63 135 1 2024-09-16T08:57:20.749 INFO:tasks.workunit.client.0.smithi031.stdout:7/506: creat d2/fab x:0 0 0 2024-09-16T08:57:20.749 INFO:tasks.workunit.client.1.smithi097.stdout:2/958: mknod d3/d4/d9/d67/d122/c12b 0 2024-09-16T08:57:20.753 INFO:tasks.workunit.client.1.smithi097.stdout:7/839: mknod d4/daa/c11e 0 2024-09-16T08:57:20.755 INFO:tasks.workunit.client.1.smithi097.stdout:2/959: truncate d3/d4/d9/d78/f94 1274028 0 2024-09-16T08:57:20.756 INFO:tasks.workunit.client.1.smithi097.stdout:2/960: mknod d3/d4/d9/d67/c12c 0 2024-09-16T08:57:20.756 INFO:tasks.workunit.client.1.smithi097.stdout:2/961: chown d3/d4/d68/c116 3470748 1 2024-09-16T08:57:20.761 INFO:tasks.workunit.client.0.smithi031.stdout:4/444: write d0/d2/d16/d1e/f3b [415329,90220] 0 2024-09-16T08:57:20.761 INFO:tasks.workunit.client.0.smithi031.stdout:4/445: readlink d0/d2/d16/d2b/l40 0 2024-09-16T08:57:20.761 INFO:tasks.workunit.client.0.smithi031.stdout:4/446: creat d0/d2/f90 x:0 0 0 2024-09-16T08:57:20.761 INFO:tasks.workunit.client.0.smithi031.stdout:4/447: truncate d0/d2/f90 765324 0 2024-09-16T08:57:20.761 INFO:tasks.workunit.client.0.smithi031.stdout:4/448: creat d0/d5b/f91 x:0 0 0 2024-09-16T08:57:20.766 INFO:tasks.workunit.client.0.smithi031.stdout:4/449: rename d0/d2/d3a/d55/l58 to d0/d2/d16/l92 0 2024-09-16T08:57:20.766 INFO:tasks.workunit.client.1.smithi097.stdout:5/984: dwrite db/d18/d1c/d42/d46/d7e/f9e [0,4194304] 0 2024-09-16T08:57:20.767 INFO:tasks.workunit.client.0.smithi031.stdout:0/451: dwrite d6/d12/d47/d5b/f71 [0,4194304] 0 2024-09-16T08:57:20.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/840: write d4/d7/d7b/d43/fa7 [268996,105818] 0 2024-09-16T08:57:20.769 INFO:tasks.workunit.client.1.smithi097.stdout:7/841: write d4/d9/d1c/d5f/fdb [94809,71289] 0 2024-09-16T08:57:20.770 INFO:tasks.workunit.client.0.smithi031.stdout:0/452: dread d6/d12/d13/f6f [0,4194304] 0 2024-09-16T08:57:20.770 INFO:tasks.workunit.client.1.smithi097.stdout:9/939: dwrite d2/d5/f71 [0,4194304] 0 2024-09-16T08:57:20.770 INFO:tasks.workunit.client.1.smithi097.stdout:9/940: chown d2/d5/d56/d116/cf3 38 1 2024-09-16T08:57:20.771 INFO:tasks.workunit.client.1.smithi097.stdout:5/985: mkdir db/d18/d146 0 2024-09-16T08:57:20.774 INFO:tasks.workunit.client.0.smithi031.stdout:4/450: write d0/d2/d16/d2b/f74 [2902948,110917] 0 2024-09-16T08:57:20.774 INFO:tasks.workunit.client.0.smithi031.stdout:4/451: chown d0/d5b/f7d 7845362 1 2024-09-16T08:57:20.774 INFO:tasks.workunit.client.0.smithi031.stdout:4/452: creat d0/d2/d3a/f93 x:0 0 0 2024-09-16T08:57:20.778 INFO:tasks.workunit.client.1.smithi097.stdout:8/849: dwrite d8/dd/d15/d2c/d4a/d4d/d3d/db9/fda [0,4194304] 0 2024-09-16T08:57:20.779 INFO:tasks.workunit.client.0.smithi031.stdout:3/533: dwrite d3/fb [0,4194304] 0 2024-09-16T08:57:20.784 INFO:tasks.workunit.client.1.smithi097.stdout:6/959: dwrite d1/dd3/d9e/dca/d81/dc4/dc6/ff0 [0,4194304] 0 2024-09-16T08:57:20.784 INFO:tasks.workunit.client.1.smithi097.stdout:6/960: creat d1/dd3/d9e/db3/f145 x:0 0 0 2024-09-16T08:57:20.785 INFO:tasks.workunit.client.1.smithi097.stdout:5/986: link db/d18/d1c/d42/d46/ff0 db/d18/d20/f147 0 2024-09-16T08:57:20.787 INFO:tasks.workunit.client.1.smithi097.stdout:7/842: getdents d4/d7/d30/ddf 0 2024-09-16T08:57:20.789 INFO:tasks.workunit.client.0.smithi031.stdout:3/534: symlink d3/d14/d6f/d82/l9d 0 2024-09-16T08:57:20.789 INFO:tasks.workunit.client.0.smithi031.stdout:3/535: write d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 [1194541,110334] 0 2024-09-16T08:57:20.791 INFO:tasks.workunit.client.0.smithi031.stdout:7/507: dwrite d2/f94 [0,4194304] 0 2024-09-16T08:57:20.791 INFO:tasks.workunit.client.0.smithi031.stdout:7/508: chown d2/d8/f63 97709 1 2024-09-16T08:57:20.792 INFO:tasks.workunit.client.1.smithi097.stdout:8/850: symlink d8/dd/d15/l10d 0 2024-09-16T08:57:20.794 INFO:tasks.workunit.client.0.smithi031.stdout:3/536: creat d3/d14/d37/d1a/d1d/d1e/d67/d7f/f9e x:0 0 0 2024-09-16T08:57:20.798 INFO:tasks.workunit.client.1.smithi097.stdout:6/961: mkdir d1/dd3/d9e/dca/d16/d4f/df8/d146 0 2024-09-16T08:57:20.800 INFO:tasks.workunit.client.0.smithi031.stdout:0/453: dwrite d6/f26 [0,4194304] 0 2024-09-16T08:57:20.800 INFO:tasks.workunit.client.1.smithi097.stdout:2/962: dwrite d3/d1b/f112 [0,4194304] 0 2024-09-16T08:57:20.800 INFO:tasks.workunit.client.1.smithi097.stdout:3/919: dwrite d7/de/d3c/f73 [0,4194304] 0 2024-09-16T08:57:20.801 INFO:tasks.workunit.client.0.smithi031.stdout:6/566: sync 2024-09-16T08:57:20.802 INFO:tasks.workunit.client.1.smithi097.stdout:7/843: creat d4/d7/d7b/d43/daf/f11f x:0 0 0 2024-09-16T08:57:20.802 INFO:tasks.workunit.client.1.smithi097.stdout:7/844: truncate d4/d9/dc/d8e/fde 570663 0 2024-09-16T08:57:20.805 INFO:tasks.workunit.client.1.smithi097.stdout:5/987: write db/fe [3669233,104688] 0 2024-09-16T08:57:20.806 INFO:tasks.workunit.client.1.smithi097.stdout:8/851: dread d8/fc [4194304,4194304] 0 2024-09-16T08:57:20.806 INFO:tasks.workunit.client.0.smithi031.stdout:3/537: creat d3/d14/d6f/d7d/f9f x:0 0 0 2024-09-16T08:57:20.806 INFO:tasks.workunit.client.0.smithi031.stdout:3/538: chown d3/d14/d37/f10 511856 1 2024-09-16T08:57:20.806 INFO:tasks.workunit.client.0.smithi031.stdout:3/539: getdents d3/d14/d37/d1a/d1d/d3b/d3f/d79/d7c 0 2024-09-16T08:57:20.806 INFO:tasks.workunit.client.0.smithi031.stdout:3/540: chown d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 4053 1 2024-09-16T08:57:20.808 INFO:tasks.workunit.client.1.smithi097.stdout:3/920: rmdir d7/de/d3c/d3f/d60/d11e 0 2024-09-16T08:57:20.812 INFO:tasks.workunit.client.1.smithi097.stdout:2/963: creat d3/d4/d9/d67/d122/d61/dac/dd9/de6/f12d x:0 0 0 2024-09-16T08:57:20.812 INFO:tasks.workunit.client.1.smithi097.stdout:2/964: readlink d3/d4/d9/d67/d122/d7c/d7d/d53/da3/de8/df8/dfc/lfe 0 2024-09-16T08:57:20.814 INFO:tasks.workunit.client.1.smithi097.stdout:9/941: dwrite d2/d5/d56/faf [0,4194304] 0 2024-09-16T08:57:20.815 INFO:tasks.workunit.client.0.smithi031.stdout:0/454: symlink d6/d61/l90 0 2024-09-16T08:57:20.817 INFO:tasks.workunit.client.1.smithi097.stdout:5/988: symlink db/l148 0 2024-09-16T08:57:20.819 INFO:tasks.workunit.client.0.smithi031.stdout:6/567: symlink d0/dd/d74/da6/lb6 0 2024-09-16T08:57:20.821 INFO:tasks.workunit.client.1.smithi097.stdout:8/852: mkdir d8/dd/d15/d2c/d4a/df3/d109/d10e 0 2024-09-16T08:57:20.821 INFO:tasks.workunit.client.0.smithi031.stdout:3/541: write d3/d14/d37/d33/f6d [1047547,31559] 0 2024-09-16T08:57:20.821 INFO:tasks.workunit.client.0.smithi031.stdout:3/542: fsync d3/f5 0 2024-09-16T08:57:20.822 INFO:tasks.workunit.client.0.smithi031.stdout:4/453: dwrite d0/d2/d16/d59/f5d [0,4194304] 0 2024-09-16T08:57:20.824 INFO:tasks.workunit.client.0.smithi031.stdout:0/455: mkdir d6/d61/d91 0 2024-09-16T08:57:20.824 INFO:tasks.workunit.client.0.smithi031.stdout:4/454: dread d0/f18 [0,4194304] 0 2024-09-16T08:57:20.824 INFO:tasks.workunit.client.1.smithi097.stdout:2/965: mknod d3/d4/d68/d104/d127/c12e 0 2024-09-16T08:57:20.830 INFO:tasks.workunit.client.0.smithi031.stdout:4/455: creat d0/f94 x:0 0 0 2024-09-16T08:57:20.833 INFO:tasks.workunit.client.0.smithi031.stdout:6/568: rmdir d0/d8e 39 2024-09-16T08:57:20.836 INFO:tasks.workunit.client.0.smithi031.stdout:6/569: symlink d0/dd/d74/dad/lb7 0 2024-09-16T08:57:20.839 INFO:tasks.workunit.client.0.smithi031.stdout:6/570: rmdir d0/dd/d74/d19 39 2024-09-16T08:57:20.839 INFO:tasks.workunit.client.0.smithi031.stdout:6/571: read d0/dd/d74/f30 [2416447,76439] 0 2024-09-16T08:57:20.842 INFO:tasks.workunit.client.0.smithi031.stdout:6/572: rmdir d0/dd/d74/d5e/d7c/d56 39 2024-09-16T08:57:20.843 INFO:tasks.workunit.client.0.smithi031.stdout:4/456: dread d0/d2/d16/f35 [0,4194304] 0 2024-09-16T08:57:20.845 INFO:tasks.workunit.client.1.smithi097.stdout:9/942: rename d2/l23 to d2/d5/l12a 0 2024-09-16T08:57:20.845 INFO:tasks.workunit.client.1.smithi097.stdout:9/943: stat d2/d5/c106 0 2024-09-16T08:57:20.845 INFO:tasks.workunit.client.1.smithi097.stdout:9/944: chown d2/f13 49113051 1 2024-09-16T08:57:20.846 INFO:tasks.workunit.client.0.smithi031.stdout:8/397: dwrite d4/d7/d21/d37/f76 [0,4194304] 0 2024-09-16T08:57:20.846 INFO:tasks.workunit.client.1.smithi097.stdout:5/989: mknod db/d1a/d6d/dc6/c149 0 2024-09-16T08:57:20.847 INFO:tasks.workunit.client.0.smithi031.stdout:8/398: truncate d4/d7/d21/d35/d40/f58 492554 0 2024-09-16T08:57:20.848 INFO:tasks.workunit.client.1.smithi097.stdout:8/853: mknod d8/dd/d15/d2c/d4a/d4d/d3d/d7b/c10f 0 2024-09-16T08:57:20.849 INFO:tasks.workunit.client.1.smithi097.stdout:9/945: getdents d2/d5/db0 0 2024-09-16T08:57:20.850 INFO:tasks.workunit.client.1.smithi097.stdout:9/946: fdatasync d2/d5/db0/dc8/d63/da8/dca/fd3 0 2024-09-16T08:57:20.850 INFO:tasks.workunit.client.1.smithi097.stdout:5/990: symlink db/d18/d1c/d42/l14a 0 2024-09-16T08:57:20.851 INFO:tasks.workunit.client.0.smithi031.stdout:1/458: sync 2024-09-16T08:57:20.852 INFO:tasks.workunit.client.0.smithi031.stdout:9/440: sync 2024-09-16T08:57:20.852 INFO:tasks.workunit.client.0.smithi031.stdout:9/441: readlink d3/l4a 0 2024-09-16T08:57:20.852 INFO:tasks.workunit.client.1.smithi097.stdout:8/854: link d8/dd/f82 d8/dd/d71/d9c/f110 0 2024-09-16T08:57:20.853 INFO:tasks.workunit.client.1.smithi097.stdout:9/947: mkdir d2/d5/db0/dc8/d63/d12b 0 2024-09-16T08:57:20.854 INFO:tasks.workunit.client.0.smithi031.stdout:1/459: truncate d2/f3e 424584 0 2024-09-16T08:57:20.854 INFO:tasks.workunit.client.1.smithi097.stdout:5/991: stat db/d18/d1c/l2a 0 2024-09-16T08:57:20.855 INFO:tasks.workunit.client.0.smithi031.stdout:9/442: truncate d3/d5/da/f2c 1259602 0 2024-09-16T08:57:20.855 INFO:tasks.workunit.client.0.smithi031.stdout:9/443: truncate d3/d5/d73/f86 852911 0 2024-09-16T08:57:20.855 INFO:tasks.workunit.client.1.smithi097.stdout:8/855: getdents d8/dd/d15/d2c/d4a/d4e 0 2024-09-16T08:57:20.856 INFO:tasks.workunit.client.1.smithi097.stdout:9/948: creat d2/d5/f12c x:0 0 0 2024-09-16T08:57:20.857 INFO:tasks.workunit.client.1.smithi097.stdout:5/992: truncate db/d18/d1c/d42/d46/d7e/d8f/ff5 1472772 0 2024-09-16T08:57:20.857 INFO:tasks.workunit.client.1.smithi097.stdout:5/993: dread - db/d18/d1c/d42/d46/d7e/d64/d8d/f12a zero size 2024-09-16T08:57:20.858 INFO:tasks.workunit.client.1.smithi097.stdout:8/856: mknod d8/dd/d71/d9c/dac/c111 0 2024-09-16T08:57:20.858 INFO:tasks.workunit.client.1.smithi097.stdout:8/857: readlink d8/dd/d15/d2c/d4a/d4e/l101 0 2024-09-16T08:57:20.858 INFO:tasks.workunit.client.1.smithi097.stdout:8/858: write d8/dd/d15/d16/d7c/feb [884697,13230] 0 2024-09-16T08:57:20.858 INFO:tasks.workunit.client.1.smithi097.stdout:8/859: chown d8/dd/d15/d2c/d4a/df3/d109 335 1 2024-09-16T08:57:20.858 INFO:tasks.workunit.client.1.smithi097.stdout:5/994: rename db/d6e/d81/cf4 to db/d22/d58/c14b 0 2024-09-16T08:57:20.859 INFO:tasks.workunit.client.1.smithi097.stdout:8/860: rename d8/dd/d15/d2c/d4a/d4e/d85/fd6 to d8/dd/d15/d2c/d4a/d4d/d3f/d99/f112 0 2024-09-16T08:57:20.860 INFO:tasks.workunit.client.1.smithi097.stdout:5/995: mkdir db/d1a/d14c 0 2024-09-16T08:57:20.860 INFO:tasks.workunit.client.1.smithi097.stdout:5/996: stat db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d90 0 2024-09-16T08:57:20.860 INFO:tasks.workunit.client.1.smithi097.stdout:5/997: dread - db/d18/d1c/d42/d46/d7e/d64/d7f/d89/daf/d9f/fcc zero size 2024-09-16T08:57:20.861 INFO:tasks.workunit.client.1.smithi097.stdout:8/861: creat d8/dd/d15/d2c/d4a/df3/d109/d10e/f113 x:0 0 0 2024-09-16T08:57:20.861 INFO:tasks.workunit.client.1.smithi097.stdout:8/862: fdatasync d8/dd/d15/d2c/d4a/f87 0 2024-09-16T08:57:20.862 INFO:tasks.workunit.client.1.smithi097.stdout:5/998: symlink db/d18/d1c/d42/d46/d7e/d64/d7f/d89/l14d 0 2024-09-16T08:57:20.862 INFO:tasks.workunit.client.1.smithi097.stdout:5/999: fsync db/d18/d1c/d42/d46/d7e/d76/f8e 0 2024-09-16T08:57:20.862 INFO:tasks.workunit.client.1.smithi097.stdout:8/863: mknod d8/dd/d71/d9c/c114 0 2024-09-16T08:57:20.866 INFO:tasks.workunit.client.0.smithi031.stdout:9/444: write d3/d5/da/d34/f35 [3400434,93560] 0 2024-09-16T08:57:20.868 INFO:tasks.workunit.client.0.smithi031.stdout:7/509: dwrite d2/d8/d11/d81/fa5 [0,4194304] 0 2024-09-16T08:57:20.869 INFO:tasks.workunit.client.0.smithi031.stdout:6/573: dread d0/dd/d74/d19/f1d [4194304,4194304] 0 2024-09-16T08:57:20.869 INFO:tasks.workunit.client.0.smithi031.stdout:9/445: rmdir d3/d5/da/d34/d3c/d5e/d62/d92 0 2024-09-16T08:57:20.870 INFO:tasks.workunit.client.0.smithi031.stdout:3/543: dwrite d3/fb [0,4194304] 0 2024-09-16T08:57:20.870 INFO:tasks.workunit.client.1.smithi097.stdout:6/962: dwrite d1/dd3/d9e/dca/d16/d4f/ff4 [0,4194304] 0 2024-09-16T08:57:20.870 INFO:tasks.workunit.client.1.smithi097.stdout:7/845: dwrite d4/d9/d1c/f2d [0,4194304] 0 2024-09-16T08:57:20.871 INFO:tasks.workunit.client.1.smithi097.stdout:2/966: dwrite d3/d1b/da7/ffb [0,4194304] 0 2024-09-16T08:57:20.872 INFO:tasks.workunit.client.0.smithi031.stdout:8/399: dread d4/d7/d21/d37/d46/f75 [0,4194304] 0 2024-09-16T08:57:20.872 INFO:tasks.workunit.client.0.smithi031.stdout:8/400: readlink d4/d7/d21/d24/l60 0 2024-09-16T08:57:20.872 INFO:tasks.workunit.client.0.smithi031.stdout:4/457: dread d0/f24 [0,4194304] 0 2024-09-16T08:57:20.872 INFO:tasks.workunit.client.0.smithi031.stdout:0/456: dwrite d6/d12/d47/d5b/f71 [0,4194304] 0 2024-09-16T08:57:20.873 INFO:tasks.workunit.client.0.smithi031.stdout:0/457: creat d6/d12/d47/d5b/f92 x:0 0 0 2024-09-16T08:57:20.874 INFO:tasks.workunit.client.0.smithi031.stdout:7/510: unlink d2/d8/d86/f9e 0 2024-09-16T08:57:20.874 INFO:tasks.workunit.client.0.smithi031.stdout:7/511: readlink d2/d8/d11/d5d/la6 0 2024-09-16T08:57:20.876 INFO:tasks.workunit.client.0.smithi031.stdout:6/574: mknod d0/dd/d35/d6c/d81/cb8 0 2024-09-16T08:57:20.878 INFO:tasks.workunit.client.1.smithi097.stdout:6/963: creat d1/dd3/f147 x:0 0 0 2024-09-16T08:57:20.879 INFO:tasks.workunit.client.0.smithi031.stdout:9/446: symlink d3/l9e 0 2024-09-16T08:57:20.879 INFO:tasks.workunit.client.0.smithi031.stdout:1/460: dread d2/d4/f1c [0,4194304] 0 2024-09-16T08:57:20.880 INFO:tasks.workunit.client.1.smithi097.stdout:7/846: truncate d4/d9/dc/f49 1994722 0 2024-09-16T08:57:20.882 INFO:tasks.workunit.client.1.smithi097.stdout:8/864: dread d8/f1e [0,4194304] 0 2024-09-16T08:57:20.884 INFO:tasks.workunit.client.0.smithi031.stdout:4/458: creat d0/d8e/f95 x:0 0 0 2024-09-16T08:57:20.884 INFO:tasks.workunit.client.0.smithi031.stdout:4/459: fsync d0/f2e 0 2024-09-16T08:57:20.890 INFO:tasks.workunit.client.0.smithi031.stdout:0/458: creat d6/d12/d47/d4d/f93 x:0 0 0 2024-09-16T08:57:20.893 INFO:tasks.workunit.client.1.smithi097.stdout:6/964: unlink d1/dd3/d9e/la1 0 2024-09-16T08:57:20.895 INFO:tasks.workunit.client.1.smithi097.stdout:7/847: mknod d4/d9/d51/d8b/c120 0 2024-09-16T08:57:20.896 INFO:tasks.workunit.client.1.smithi097.stdout:8/865: dread d8/dd/d71/d9c/f110 [0,4194304] 0 2024-09-16T08:57:20.898 INFO:tasks.workunit.client.1.smithi097.stdout:6/965: rename d1/dd3/d9e/dca/d16/f4d to d1/dd3/d9e/dca/d16/d4f/d9f/f148 0 2024-09-16T08:57:20.898 INFO:tasks.workunit.client.1.smithi097.stdout:6/966: getdents d1/dd3/d9e/dca/d16/d6a 0 2024-09-16T08:57:20.898 INFO:tasks.workunit.client.1.smithi097.stdout:6/967: read - d1/dd3/d9e/db3/f145 zero size 2024-09-16T08:57:20.899 INFO:tasks.workunit.client.1.smithi097.stdout:2/967: dwrite d3/d4/d8/f7a [0,4194304] 0 2024-09-16T08:57:20.900 INFO:tasks.workunit.client.0.smithi031.stdout:6/575: dwrite d0/dd/d35/d6c/d83/f93 [0,4194304] 0 2024-09-16T08:57:20.900 INFO:tasks.workunit.client.1.smithi097.stdout:8/866: mkdir d8/dd/d15/d2c/d4a/d4e/db2/d115 0 2024-09-16T08:57:20.902 INFO:tasks.workunit.client.1.smithi097.stdout:4/835: sync 2024-09-16T08:57:20.902 INFO:tasks.workunit.client.1.smithi097.stdout:4/836: chown d0/d36/da7/faa 7 1 2024-09-16T08:57:20.902 INFO:tasks.workunit.client.1.smithi097.stdout:6/968: mkdir d1/dd3/d9e/dca/d16/d6a/d149 0 2024-09-16T08:57:20.903 INFO:tasks.workunit.client.1.smithi097.stdout:6/969: write d1/dd3/d9e/dca/d81/dc4/fcc [143195,128557] 0 2024-09-16T08:57:20.903 INFO:tasks.workunit.client.1.smithi097.stdout:6/970: fsync d1/dc/d3c/f123 0 2024-09-16T08:57:20.906 INFO:tasks.workunit.client.1.smithi097.stdout:4/837: dread d0/d35/da4/dd1/fe1 [0,4194304] 0 2024-09-16T08:57:20.907 INFO:tasks.workunit.client.1.smithi097.stdout:2/968: creat d3/d4/f12f x:0 0 0 2024-09-16T08:57:20.909 INFO:tasks.workunit.client.0.smithi031.stdout:4/460: dwrite d0/f24 [0,4194304] 0 2024-09-16T08:57:20.909 INFO:tasks.workunit.client.0.smithi031.stdout:9/447: rename d3/d5/da/db/d21/d3b/f79 to d3/d5/da/d34/d3c/d5e/d9c/f9f 0 2024-09-16T08:57:20.909 INFO:tasks.workunit.client.0.smithi031.stdout:9/448: readlink d3/d5/da/d34/d3c/d5e/d62/d65/l97 0 2024-09-16T08:57:20.910 INFO:tasks.workunit.client.0.smithi031.stdout:9/449: stat d3/d5/da/db/d21/d3b/c70 0 2024-09-16T08:57:20.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/867: creat d8/dd/d15/d2c/d4a/d4e/d97/f116 x:0 0 0 2024-09-16T08:57:20.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/868: stat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb 0 2024-09-16T08:57:20.911 INFO:tasks.workunit.client.1.smithi097.stdout:8/869: write d8/dd/d15/d2c/f74 [825440,110211] 0 2024-09-16T08:57:20.912 INFO:tasks.workunit.client.0.smithi031.stdout:1/461: mkdir d2/d15/d32/d87/da5 0 2024-09-16T08:57:20.912 INFO:tasks.workunit.client.0.smithi031.stdout:1/462: fsync d2/f14 0 2024-09-16T08:57:20.913 INFO:tasks.workunit.client.0.smithi031.stdout:9/450: dread d3/d48/f49 [0,4194304] 0 2024-09-16T08:57:20.913 INFO:tasks.workunit.client.0.smithi031.stdout:9/451: fsync d3/d5/da/d34/d3c/d5e/d62/d65/f94 0 2024-09-16T08:57:20.913 INFO:tasks.workunit.client.1.smithi097.stdout:2/969: dread d3/d4/d8/f16 [0,4194304] 0 2024-09-16T08:57:20.913 INFO:tasks.workunit.client.1.smithi097.stdout:6/971: symlink d1/dd3/d9e/db3/l14a 0 2024-09-16T08:57:20.914 INFO:tasks.workunit.client.0.smithi031.stdout:3/544: symlink d3/d14/d37/d1a/d1d/d3b/la0 0 2024-09-16T08:57:20.915 INFO:tasks.workunit.client.0.smithi031.stdout:0/459: mknod d6/d12/d47/d7c/c94 0 2024-09-16T08:57:20.917 INFO:tasks.workunit.client.1.smithi097.stdout:4/838: write d0/d7/f10d [727105,76043] 0 2024-09-16T08:57:20.917 INFO:tasks.workunit.client.1.smithi097.stdout:4/839: readlink d0/d39/d119/l109 0 2024-09-16T08:57:20.917 INFO:tasks.workunit.client.1.smithi097.stdout:4/840: write d0/d7/f10d [4023693,39159] 0 2024-09-16T08:57:20.917 INFO:tasks.workunit.client.1.smithi097.stdout:6/972: mknod d1/dd3/d9e/dca/db6/c14b 0 2024-09-16T08:57:20.924 INFO:tasks.workunit.client.0.smithi031.stdout:6/576: link d0/dd/d35/l49 d0/lb9 0 2024-09-16T08:57:20.925 INFO:tasks.workunit.client.1.smithi097.stdout:6/973: dread d1/dd3/d9e/dca/d16/f2e [0,4194304] 0 2024-09-16T08:57:20.925 INFO:tasks.workunit.client.1.smithi097.stdout:4/841: dread d0/d39/d71/f77 [0,4194304] 0 2024-09-16T08:57:20.925 INFO:tasks.workunit.client.0.smithi031.stdout:4/461: mkdir d0/d71/d96 0 2024-09-16T08:57:20.927 INFO:tasks.workunit.client.0.smithi031.stdout:7/512: rename d2/d2d/la3 to d2/d8/d11/d66/lac 0 2024-09-16T08:57:20.929 INFO:tasks.workunit.client.0.smithi031.stdout:1/463: unlink d2/d15/c5a 0 2024-09-16T08:57:20.930 INFO:tasks.workunit.client.0.smithi031.stdout:7/513: dread d2/d8/f8c [0,4194304] 0 2024-09-16T08:57:20.931 INFO:tasks.workunit.client.1.smithi097.stdout:6/974: unlink d1/f122 0 2024-09-16T08:57:20.933 INFO:tasks.workunit.client.0.smithi031.stdout:9/452: truncate d3/d5/da/f6b 92103 0 2024-09-16T08:57:20.934 INFO:tasks.workunit.client.1.smithi097.stdout:4/842: creat d0/d39/d119/f11f x:0 0 0 2024-09-16T08:57:20.935 INFO:tasks.workunit.client.1.smithi097.stdout:4/843: dread d0/d7/df/d83/df8/fcf [0,4194304] 0 2024-09-16T08:57:20.936 INFO:tasks.workunit.client.0.smithi031.stdout:3/545: rmdir d3/d14 39 2024-09-16T08:57:20.938 INFO:tasks.workunit.client.1.smithi097.stdout:6/975: mkdir d1/dd3/d9e/d14c 0 2024-09-16T08:57:20.938 INFO:tasks.workunit.client.1.smithi097.stdout:6/976: truncate d1/dd3/d9e/dca/d81/dad/f104 416613 0 2024-09-16T08:57:20.938 INFO:tasks.workunit.client.1.smithi097.stdout:6/977: chown d1/dd3/d9e/dca/d16/d4f/c7e 186005 1 2024-09-16T08:57:20.938 INFO:tasks.workunit.client.1.smithi097.stdout:6/978: chown d1/lf9 27847 1 2024-09-16T08:57:20.939 INFO:tasks.workunit.client.0.smithi031.stdout:6/577: mkdir d0/dd/d35/d6c/dac/dba 0 2024-09-16T08:57:20.939 INFO:tasks.workunit.client.0.smithi031.stdout:6/578: write d0/dd/d74/d5e/d7c/d3f/f4b [4668348,48568] 0 2024-09-16T08:57:20.940 INFO:tasks.workunit.client.1.smithi097.stdout:4/844: rmdir d0/d7 39 2024-09-16T08:57:20.941 INFO:tasks.workunit.client.0.smithi031.stdout:4/462: truncate d0/d2/d16/f19 893611 0 2024-09-16T08:57:20.941 INFO:tasks.workunit.client.0.smithi031.stdout:4/463: truncate d0/d2/d16/d59/f5d 4444047 0 2024-09-16T08:57:20.941 INFO:tasks.workunit.client.0.smithi031.stdout:4/464: truncate d0/d3c/d6b/f8a 1235939 0 2024-09-16T08:57:20.943 INFO:tasks.workunit.client.1.smithi097.stdout:6/979: mknod d1/dd3/d9e/dca/dbb/c14d 0 2024-09-16T08:57:20.943 INFO:tasks.workunit.client.1.smithi097.stdout:6/980: write d1/dd3/d9e/dca/d16/d4f/d9f/fba [726713,82228] 0 2024-09-16T08:57:20.944 INFO:tasks.workunit.client.0.smithi031.stdout:3/546: symlink d3/d14/d37/d1a/d1d/d3b/d94/la1 0 2024-09-16T08:57:20.944 INFO:tasks.workunit.client.0.smithi031.stdout:3/547: write d3/d14/d6f/d7d/f9f [425038,73580] 0 2024-09-16T08:57:20.945 INFO:tasks.workunit.client.1.smithi097.stdout:4/845: truncate d0/d23/f3a 4733277 0 2024-09-16T08:57:20.945 INFO:tasks.workunit.client.1.smithi097.stdout:4/846: chown d0/d35/dc0 2112 1 2024-09-16T08:57:20.945 INFO:tasks.workunit.client.1.smithi097.stdout:4/847: fsync d0/d7/d52/f94 0 2024-09-16T08:57:20.946 INFO:tasks.workunit.client.0.smithi031.stdout:0/460: dwrite d6/f4f [0,4194304] 0 2024-09-16T08:57:20.946 INFO:tasks.workunit.client.0.smithi031.stdout:0/461: truncate d6/d12/d47/f7e 425052 0 2024-09-16T08:57:20.946 INFO:tasks.workunit.client.1.smithi097.stdout:2/970: dwrite d3/d4/d9/d67/d122/d7c/d7d/fc6 [0,4194304] 0 2024-09-16T08:57:20.947 INFO:tasks.workunit.client.0.smithi031.stdout:4/465: truncate d0/fa 172631 0 2024-09-16T08:57:20.948 INFO:tasks.workunit.client.1.smithi097.stdout:4/848: stat d0/d39/l9b 0 2024-09-16T08:57:20.948 INFO:tasks.workunit.client.1.smithi097.stdout:4/849: write d0/fe [2781126,12727] 0 2024-09-16T08:57:20.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/548: creat d3/d14/d37/d1a/d1d/d9a/fa2 x:0 0 0 2024-09-16T08:57:20.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/549: chown c0 1701664 1 2024-09-16T08:57:20.950 INFO:tasks.workunit.client.0.smithi031.stdout:3/550: chown d3/c27 665 1 2024-09-16T08:57:20.951 INFO:tasks.workunit.client.1.smithi097.stdout:8/870: dwrite d8/dd/d15/d2c/d4a/d4d/d3d/d7b/fc0 [0,4194304] 0 2024-09-16T08:57:20.951 INFO:tasks.workunit.client.1.smithi097.stdout:8/871: read - d8/dd/d15/d2c/d4a/d4e/d85/f9d zero size 2024-09-16T08:57:20.951 INFO:tasks.workunit.client.1.smithi097.stdout:2/971: mkdir d3/d4/d9/d67/d122/d61/dac/d130 0 2024-09-16T08:57:20.953 INFO:tasks.workunit.client.1.smithi097.stdout:4/850: symlink d0/d7/df/d15/l120 0 2024-09-16T08:57:20.953 INFO:tasks.workunit.client.0.smithi031.stdout:6/579: dread d0/d10/f3b [0,4194304] 0 2024-09-16T08:57:20.957 INFO:tasks.workunit.client.0.smithi031.stdout:0/462: unlink d6/d12/d47/f4e 0 2024-09-16T08:57:20.957 INFO:tasks.workunit.client.0.smithi031.stdout:0/463: chown d6/d12/d13/c2f 6 1 2024-09-16T08:57:20.957 INFO:tasks.workunit.client.0.smithi031.stdout:0/464: chown d6/d61 316 1 2024-09-16T08:57:20.957 INFO:tasks.workunit.client.0.smithi031.stdout:0/465: creat d6/d12/f95 x:0 0 0 2024-09-16T08:57:20.957 INFO:tasks.workunit.client.0.smithi031.stdout:0/466: stat d6/d12/d47/d5b/c79 0 2024-09-16T08:57:20.957 INFO:tasks.workunit.client.0.smithi031.stdout:9/453: write f2 [945102,14103] 0 2024-09-16T08:57:20.958 INFO:tasks.workunit.client.1.smithi097.stdout:8/872: unlink d8/dd/d71/d9c/fca 0 2024-09-16T08:57:20.961 INFO:tasks.workunit.client.1.smithi097.stdout:2/972: mkdir d3/d1b/dbf/d131 0 2024-09-16T08:57:20.961 INFO:tasks.workunit.client.1.smithi097.stdout:2/973: dread - d3/d4/d9/d67/d122/d7c/d7d/d53/d47/dd4/f108 zero size 2024-09-16T08:57:20.961 INFO:tasks.workunit.client.1.smithi097.stdout:2/974: write d3/d4/d9/d67/d122/d7c/d7d/d53/d114/f11d [528009,113655] 0 2024-09-16T08:57:20.962 INFO:tasks.workunit.client.0.smithi031.stdout:0/467: rmdir d6/d61 39 2024-09-16T08:57:20.962 INFO:tasks.workunit.client.0.smithi031.stdout:5/454: sync 2024-09-16T08:57:20.962 INFO:tasks.workunit.client.0.smithi031.stdout:2/401: sync 2024-09-16T08:57:20.962 INFO:tasks.workunit.client.1.smithi097.stdout:4/851: rmdir d0/d23/ddc/de5 39 2024-09-16T08:57:20.963 INFO:tasks.workunit.client.1.smithi097.stdout:4/852: truncate d0/d7/df/d83/ff5 115882 0 2024-09-16T08:57:20.963 INFO:tasks.workunit.client.1.smithi097.stdout:6/981: dread d1/dd3/d9e/db3/de3/f11e [0,4194304] 0 2024-09-16T08:57:20.964 INFO:tasks.workunit.client.1.smithi097.stdout:4/853: read d0/d7/df/d83/df8/fcf [192597,39695] 0 2024-09-16T08:57:20.967 INFO:tasks.workunit.client.0.smithi031.stdout:3/551: write d3/d14/d37/d1a/d1d/d3b/d3f/f52 [1301696,3433] 0 2024-09-16T08:57:20.967 INFO:tasks.workunit.client.1.smithi097.stdout:8/873: unlink d8/dd/d15/d2c/d4a/d4e/d97/cd9 0 2024-09-16T08:57:20.967 INFO:tasks.workunit.client.1.smithi097.stdout:8/874: chown d8/f13 0 1 2024-09-16T08:57:20.967 INFO:tasks.workunit.client.1.smithi097.stdout:8/875: fdatasync d8/dd/d15/d2c/d4a/d4d/d62/f76 0 2024-09-16T08:57:20.968 INFO:tasks.workunit.client.0.smithi031.stdout:1/464: dwrite d2/d4/fa2 [0,4194304] 0 2024-09-16T08:57:20.970 INFO:tasks.workunit.client.0.smithi031.stdout:9/454: rename d3/d5/da/db/d21/d3b/f58 to d3/d5/d73/fa0 0 2024-09-16T08:57:20.970 INFO:tasks.workunit.client.0.smithi031.stdout:2/402: rename d6 to d6/da/d5e/d72/d88 22 2024-09-16T08:57:20.970 INFO:tasks.workunit.client.0.smithi031.stdout:2/403: chown d6/da/fd 46771 1 2024-09-16T08:57:20.970 INFO:tasks.workunit.client.0.smithi031.stdout:7/514: dwrite d2/d8/f4a [0,4194304] 0 2024-09-16T08:57:20.971 INFO:tasks.workunit.client.0.smithi031.stdout:7/515: chown d2/d8/d11/d81/f9f 990953502 1 2024-09-16T08:57:20.972 INFO:tasks.workunit.client.1.smithi097.stdout:2/975: creat d3/d4/d9/d67/d122/d61/dac/dd9/d10f/f132 x:0 0 0 2024-09-16T08:57:20.972 INFO:tasks.workunit.client.1.smithi097.stdout:2/976: creat d3/d4/f133 x:0 0 0 2024-09-16T08:57:20.974 INFO:tasks.workunit.client.1.smithi097.stdout:4/854: stat d0/d7/l7a 0 2024-09-16T08:57:20.975 INFO:tasks.workunit.client.0.smithi031.stdout:0/468: dread d6/d12/d47/d4d/f7a [0,4194304] 0 2024-09-16T08:57:20.975 INFO:tasks.workunit.client.0.smithi031.stdout:0/469: chown d6/f7 1688473 1 2024-09-16T08:57:20.977 INFO:tasks.workunit.client.0.smithi031.stdout:5/455: dread d2/dc/d25/d21/f45 [0,4194304] 0 2024-09-16T08:57:20.977 INFO:tasks.workunit.client.0.smithi031.stdout:5/456: write d2/dc/d3d/d4c/f7a [229423,13359] 0 2024-09-16T08:57:20.980 INFO:tasks.workunit.client.0.smithi031.stdout:2/404: dread d6/da/f6a [0,4194304] 0 2024-09-16T08:57:20.982 INFO:tasks.workunit.client.0.smithi031.stdout:6/580: dwrite d0/dd/d74/d19/f2e [4194304,4194304] 0 2024-09-16T08:57:20.982 INFO:tasks.workunit.client.0.smithi031.stdout:6/581: write d0/dd/d2b/f45 [701365,18978] 0 2024-09-16T08:57:20.982 INFO:tasks.workunit.client.0.smithi031.stdout:6/582: fdatasync d0/dd/d74/d19/f32 0 2024-09-16T08:57:20.982 INFO:tasks.workunit.client.0.smithi031.stdout:6/583: chown d0/dd/d74/d19/f85 2613 1 2024-09-16T08:57:20.983 INFO:tasks.workunit.client.1.smithi097.stdout:2/977: creat d3/d4/d9/d67/d122/d7c/d7d/d53/d114/f134 x:0 0 0 2024-09-16T08:57:20.983 INFO:tasks.workunit.client.0.smithi031.stdout:3/552: dread d3/d14/f2e [0,4194304] 0 2024-09-16T08:57:20.985 INFO:tasks.workunit.client.1.smithi097.stdout:4/855: rename d0/d36/d4c/c8a to d0/d7/d52/d26/d8c/db0/dfa/c121 0 2024-09-16T08:57:20.985 INFO:tasks.workunit.client.1.smithi097.stdout:4/856: chown d0/d7/df/d15 3426 1 2024-09-16T08:57:20.985 INFO:tasks.workunit.client.1.smithi097.stdout:4/857: write d0/d35/da4/dc5/de7/fec [823363,102009] 0 2024-09-16T08:57:20.985 INFO:tasks.workunit.client.0.smithi031.stdout:1/465: link d2/d15/d32/f97 d2/d4/d1b/fa6 0 2024-09-16T08:57:20.987 INFO:tasks.workunit.client.1.smithi097.stdout:2/978: symlink d3/d1b/l135 0 2024-09-16T08:57:20.990 INFO:tasks.workunit.client.1.smithi097.stdout:4/858: creat d0/d36/d4c/dea/f122 x:0 0 0 2024-09-16T08:57:20.992 INFO:tasks.workunit.client.0.smithi031.stdout:0/470: unlink d6/d61/f78 0 2024-09-16T08:57:20.993 INFO:tasks.workunit.client.1.smithi097.stdout:6/982: dwrite d1/dd3/d9e/d136/f138 [0,4194304] 0 2024-09-16T08:57:20.993 INFO:tasks.workunit.client.1.smithi097.stdout:6/983: write d1/dd3/d9e/dca/f11 [219119,95006] 0 2024-09-16T08:57:20.994 INFO:tasks.workunit.client.1.smithi097.stdout:2/979: rename d3/d4/d9/d67/d122/d7c/d7d/d53/d47/c65 to d3/d4/d9/d78/c136 0 2024-09-16T08:57:20.996 INFO:tasks.workunit.client.0.smithi031.stdout:5/457: creat d2/dc/f93 x:0 0 0 2024-09-16T08:57:20.997 INFO:tasks.workunit.client.0.smithi031.stdout:6/584: write d0/d10/f58 [1543717,2040] 0 2024-09-16T08:57:20.997 INFO:tasks.workunit.client.0.smithi031.stdout:6/585: readlink d0/dd/d74/da6/lb6 0 2024-09-16T08:57:20.998 INFO:tasks.workunit.client.1.smithi097.stdout:4/859: unlink d0/d7/df/d15/c96 0 2024-09-16T08:57:20.999 INFO:tasks.workunit.client.0.smithi031.stdout:9/455: dwrite d3/d5/da/f2b [0,4194304] 0 2024-09-16T08:57:21.000 INFO:tasks.workunit.client.1.smithi097.stdout:8/876: dwrite d8/dd/d15/d2c/d4a/d4e/db2/fb3 [0,4194304] 0 2024-09-16T08:57:21.000 INFO:tasks.workunit.client.1.smithi097.stdout:8/877: chown d8/dd/d15/d2c/ldb 76600812 1 2024-09-16T08:57:21.000 INFO:tasks.workunit.client.1.smithi097.stdout:4/860: dread d0/d39/d119/f117 [0,4194304] 0 2024-09-16T08:57:21.000 INFO:tasks.workunit.client.1.smithi097.stdout:4/861: chown d0/d7/df/d15/l17 115314818 1 2024-09-16T08:57:21.000 INFO:tasks.workunit.client.1.smithi097.stdout:4/862: dread - d0/d7/df/d83/fae zero size 2024-09-16T08:57:21.001 INFO:tasks.workunit.client.0.smithi031.stdout:2/405: rmdir d6/da/d37/d42/d69 0 2024-09-16T08:57:21.002 INFO:tasks.workunit.client.0.smithi031.stdout:9/456: dread d3/d5/da/f2b [4194304,4194304] 0 2024-09-16T08:57:21.008 INFO:tasks.workunit.client.0.smithi031.stdout:3/553: mknod d3/d14/d37/d1a/d1d/d3b/d3f/d49/ca3 0 2024-09-16T08:57:21.010 INFO:tasks.workunit.client.1.smithi097.stdout:6/984: creat d1/dd3/d9e/dca/db6/f14e x:0 0 0 2024-09-16T08:57:21.010 INFO:tasks.workunit.client.1.smithi097.stdout:6/985: fdatasync d1/dd3/fb4 0 2024-09-16T08:57:21.010 INFO:tasks.workunit.client.1.smithi097.stdout:6/986: read - d1/dd3/d9e/dca/db6/fd7 zero size 2024-09-16T08:57:21.011 INFO:tasks.workunit.client.0.smithi031.stdout:1/466: mknod d2/de/ca7 0 2024-09-16T08:57:21.013 INFO:tasks.workunit.client.1.smithi097.stdout:2/980: creat d3/d4/d9/d67/d122/d7c/d7d/f137 x:0 0 0 2024-09-16T08:57:21.013 INFO:tasks.workunit.client.1.smithi097.stdout:2/981: fdatasync d3/d4/d77/f70 0 2024-09-16T08:57:21.015 INFO:tasks.workunit.client.0.smithi031.stdout:4/466: dread d0/d2/d16/f19 [0,4194304] 0 2024-09-16T08:57:21.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/554: read d3/f5 [913711,98684] 0 2024-09-16T08:57:21.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/555: dread - d3/d14/d37/d33/f7a zero size 2024-09-16T08:57:21.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/556: chown d3/d14/d37/d1a/d1d/l83 317358 1 2024-09-16T08:57:21.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/557: write d3/d14/d37/d33/f4d [412550,101450] 0 2024-09-16T08:57:21.020 INFO:tasks.workunit.client.0.smithi031.stdout:3/558: chown d3/d14/d37/d1a/d1d/d3b/d60 212 1 2024-09-16T08:57:21.027 INFO:tasks.workunit.client.0.smithi031.stdout:2/406: rename d6/f4d to d6/d20/f89 0 2024-09-16T08:57:21.030 INFO:tasks.workunit.client.0.smithi031.stdout:3/559: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/la4 0 2024-09-16T08:57:21.049 INFO:tasks.workunit.client.1.smithi097.stdout:8/878: link d8/dd/d15/d16/d7c/da2/lc5 d8/dd/d15/d2c/d4a/df3/d109/l117 0 2024-09-16T08:57:21.051 INFO:tasks.workunit.client.1.smithi097.stdout:4/863: rename d0/d7/d52/d26/cbb to d0/d36/c123 0 2024-09-16T08:57:21.052 INFO:tasks.workunit.client.1.smithi097.stdout:6/987: mkdir d1/dd3/d9e/dca/d81/dc4/dc6/d14f 0 2024-09-16T08:57:21.052 INFO:tasks.workunit.client.1.smithi097.stdout:6/988: creat d1/dd3/d9e/dca/d16/d4f/d9f/f150 x:0 0 0 2024-09-16T08:57:21.053 INFO:tasks.workunit.client.1.smithi097.stdout:2/982: unlink d3/fae 0 2024-09-16T08:57:21.054 INFO:tasks.workunit.client.1.smithi097.stdout:2/983: stat d3/d4/d9/d67/d122/d61/dac/d130 0 2024-09-16T08:57:21.055 INFO:tasks.workunit.client.1.smithi097.stdout:8/879: unlink d8/dd/d42/f9e 0 2024-09-16T08:57:21.055 INFO:tasks.workunit.client.1.smithi097.stdout:8/880: fdatasync d8/f31 0 2024-09-16T08:57:21.055 INFO:tasks.workunit.client.1.smithi097.stdout:8/881: stat d8/dd/d15/d16/d7c 0 2024-09-16T08:57:21.058 INFO:tasks.workunit.client.1.smithi097.stdout:4/864: creat d0/d36/f124 x:0 0 0 2024-09-16T08:57:21.058 INFO:tasks.workunit.client.1.smithi097.stdout:4/865: write d0/d23/f3a [5343296,26421] 0 2024-09-16T08:57:21.061 INFO:tasks.workunit.client.1.smithi097.stdout:2/984: unlink d3/d4/d68/cb3 0 2024-09-16T08:57:21.062 INFO:tasks.workunit.client.1.smithi097.stdout:8/882: mknod d8/dd/d15/d2c/d4a/d4e/d85/c118 0 2024-09-16T08:57:21.063 INFO:tasks.workunit.client.1.smithi097.stdout:4/866: truncate d0/d7/d52/d26/d8c/db0/fca 82578 0 2024-09-16T08:57:21.064 INFO:tasks.workunit.client.1.smithi097.stdout:4/867: dread - d0/d36/da7/f112 zero size 2024-09-16T08:57:21.064 INFO:tasks.workunit.client.1.smithi097.stdout:4/868: chown d0/d35/da4/dc5/de7/l10a 609725330 1 2024-09-16T08:57:21.064 INFO:tasks.workunit.client.1.smithi097.stdout:6/989: getdents d1/db7 0 2024-09-16T08:57:21.065 INFO:tasks.workunit.client.1.smithi097.stdout:2/985: symlink d3/d4/d9/d67/d122/d7c/d7d/l138 0 2024-09-16T08:57:21.066 INFO:tasks.workunit.client.1.smithi097.stdout:6/990: symlink d1/dd3/d9e/dca/d16/d6a/d149/l151 0 2024-09-16T08:57:21.067 INFO:tasks.workunit.client.1.smithi097.stdout:8/883: dread d8/dd/d15/d2c/d4a/d4d/fa5 [0,4194304] 0 2024-09-16T08:57:21.067 INFO:tasks.workunit.client.1.smithi097.stdout:8/884: write d8/dd/d15/d2c/d4a/d4e/db2/fd1 [419293,147] 0 2024-09-16T08:57:21.067 INFO:tasks.workunit.client.1.smithi097.stdout:8/885: write d8/dd/f61 [4521973,108965] 0 2024-09-16T08:57:21.067 INFO:tasks.workunit.client.1.smithi097.stdout:2/986: unlink d3/d4/d9/d67/d122/d7c/d7d/d53/d114/f134 0 2024-09-16T08:57:21.068 INFO:tasks.workunit.client.0.smithi031.stdout:9/457: dwrite d3/d5/da/db/f39 [0,4194304] 0 2024-09-16T08:57:21.068 INFO:tasks.workunit.client.0.smithi031.stdout:5/458: dwrite d2/dc/f1f [0,4194304] 0 2024-09-16T08:57:21.068 INFO:tasks.workunit.client.0.smithi031.stdout:9/458: stat d3/d5/da/db/d21/d67/l8d 0 2024-09-16T08:57:21.068 INFO:tasks.workunit.client.1.smithi097.stdout:6/991: creat d1/dd3/d9e/dca/d16/f152 x:0 0 0 2024-09-16T08:57:21.069 INFO:tasks.workunit.client.1.smithi097.stdout:8/886: link d8/dd/d15/d2c/d4a/d4e/ccf d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/c119 0 2024-09-16T08:57:21.070 INFO:tasks.workunit.client.0.smithi031.stdout:6/586: dwrite d0/dd/d74/d19/f26 [0,4194304] 0 2024-09-16T08:57:21.070 INFO:tasks.workunit.client.0.smithi031.stdout:6/587: truncate d0/dd/d2b/f31 4425319 0 2024-09-16T08:57:21.071 INFO:tasks.workunit.client.1.smithi097.stdout:2/987: creat d3/d4/d68/f139 x:0 0 0 2024-09-16T08:57:21.072 INFO:tasks.workunit.client.1.smithi097.stdout:6/992: mkdir d1/dd3/d9e/dca/d81/dc4/dc6/d153 0 2024-09-16T08:57:21.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/887: creat d8/dd/d15/d2c/d4a/d4d/d3f/d99/f11a x:0 0 0 2024-09-16T08:57:21.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/888: write d8/dd/d15/d2c/f41 [1595462,69111] 0 2024-09-16T08:57:21.074 INFO:tasks.workunit.client.1.smithi097.stdout:8/889: stat d8/dd/d15/d2c/d4a/df3/d109 0 2024-09-16T08:57:21.074 INFO:tasks.workunit.client.0.smithi031.stdout:9/459: symlink d3/d5/la1 0 2024-09-16T08:57:21.074 INFO:tasks.workunit.client.0.smithi031.stdout:9/460: write d3/fc [4146731,6330] 0 2024-09-16T08:57:21.075 INFO:tasks.workunit.client.1.smithi097.stdout:2/988: creat d3/d4/d9/d67/d122/d7c/d7d/d53/da3/de8/df8/f13a x:0 0 0 2024-09-16T08:57:21.075 INFO:tasks.workunit.client.1.smithi097.stdout:2/989: creat d3/d4/d9/d67/d122/d61/dac/dd9/de6/f13b x:0 0 0 2024-09-16T08:57:21.076 INFO:tasks.workunit.client.1.smithi097.stdout:2/990: write d3/d4/d9/d67/d122/fda [4325172,4882] 0 2024-09-16T08:57:21.081 INFO:tasks.workunit.client.1.smithi097.stdout:2/991: symlink d3/d4/d9/d67/d122/d61/dac/dd9/de6/l13c 0 2024-09-16T08:57:21.081 INFO:tasks.workunit.client.1.smithi097.stdout:2/992: chown d3/d4/d9/d67/d122/d7c/d7d/d53/f6c 13 1 2024-09-16T08:57:21.082 INFO:tasks.workunit.client.0.smithi031.stdout:9/461: link d3/d5/da/db/l38 d3/la2 0 2024-09-16T08:57:21.083 INFO:tasks.workunit.client.0.smithi031.stdout:9/462: truncate d3/d5/f2d 958141 0 2024-09-16T08:57:21.084 INFO:tasks.workunit.client.1.smithi097.stdout:6/993: rmdir d1/dd3/d9e/db3 39 2024-09-16T08:57:21.084 INFO:tasks.workunit.client.0.smithi031.stdout:3/560: dwrite d3/d14/d37/d1a/d1d/d1e/f81 [0,4194304] 0 2024-09-16T08:57:21.085 INFO:tasks.workunit.client.0.smithi031.stdout:3/561: truncate d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 4501249 0 2024-09-16T08:57:21.085 INFO:tasks.workunit.client.0.smithi031.stdout:3/562: chown d3/d14/d37/d33/c3d 48 1 2024-09-16T08:57:21.085 INFO:tasks.workunit.client.0.smithi031.stdout:6/588: dread d0/d10/f60 [0,4194304] 0 2024-09-16T08:57:21.085 INFO:tasks.workunit.client.0.smithi031.stdout:4/467: dwrite d0/f2e [0,4194304] 0 2024-09-16T08:57:21.086 INFO:tasks.workunit.client.0.smithi031.stdout:6/589: readlink d0/dd/d74/l4f 0 2024-09-16T08:57:21.086 INFO:tasks.workunit.client.0.smithi031.stdout:6/590: write d0/d8e/fb2 [995621,34127] 0 2024-09-16T08:57:21.086 INFO:tasks.workunit.client.0.smithi031.stdout:4/468: creat d0/d2/d3a/f97 x:0 0 0 2024-09-16T08:57:21.086 INFO:tasks.workunit.client.0.smithi031.stdout:6/591: write d0/dd/d74/d67/fb0 [213414,87972] 0 2024-09-16T08:57:21.086 INFO:tasks.workunit.client.1.smithi097.stdout:8/890: dread d8/dd/d15/d2c/d4a/d4d/d3d/db9/fda [0,4194304] 0 2024-09-16T08:57:21.086 INFO:tasks.workunit.client.1.smithi097.stdout:6/994: rmdir d1/dd3/d9e/dca/d13c 0 2024-09-16T08:57:21.087 INFO:tasks.workunit.client.0.smithi031.stdout:9/463: creat d3/d5/da/db/d21/fa3 x:0 0 0 2024-09-16T08:57:21.088 INFO:tasks.workunit.client.0.smithi031.stdout:1/467: dwrite d2/d4/f28 [0,4194304] 0 2024-09-16T08:57:21.090 INFO:tasks.workunit.client.1.smithi097.stdout:8/891: rename d8/dd/d15/d2c/d4a/d4d/d3f/f65 to d8/dd/d15/d2c/d4a/d4e/db2/d115/f11b 0 2024-09-16T08:57:21.096 INFO:tasks.workunit.client.1.smithi097.stdout:8/892: rename d8/dd/d15/d2c/d48/d56/f6d to d8/dd/d71/d9c/f11c 0 2024-09-16T08:57:21.101 INFO:tasks.workunit.client.0.smithi031.stdout:7/516: dwrite d2/d8/f60 [0,4194304] 0 2024-09-16T08:57:21.101 INFO:tasks.workunit.client.0.smithi031.stdout:7/517: chown d2/l3d 193 1 2024-09-16T08:57:21.101 INFO:tasks.workunit.client.1.smithi097.stdout:8/893: mknod d8/dd/d71/d9c/dac/c11d 0 2024-09-16T08:57:21.101 INFO:tasks.workunit.client.1.smithi097.stdout:4/869: fdatasync d0/d7/f10d 0 2024-09-16T08:57:21.101 INFO:tasks.workunit.client.1.smithi097.stdout:8/894: chown d8/c11 8021 1 2024-09-16T08:57:21.102 INFO:tasks.workunit.client.0.smithi031.stdout:4/469: rename d0/d2/d16/d59/f76 to d0/d2/d16/d59/d84/f98 0 2024-09-16T08:57:21.102 INFO:tasks.workunit.client.0.smithi031.stdout:4/470: dread - d0/d2/f1a zero size 2024-09-16T08:57:21.103 INFO:tasks.workunit.client.0.smithi031.stdout:4/471: truncate d0/d2/f1a 243577 0 2024-09-16T08:57:21.104 INFO:tasks.workunit.client.0.smithi031.stdout:5/459: dwrite d2/dc/d25/d21/f4e [0,4194304] 0 2024-09-16T08:57:21.106 INFO:tasks.workunit.client.1.smithi097.stdout:4/870: rename d0/d7/l7a to d0/d23/dda/l125 0 2024-09-16T08:57:21.107 INFO:tasks.workunit.client.0.smithi031.stdout:6/592: fdatasync d0/dd/d74/d67/fb0 0 2024-09-16T08:57:21.107 INFO:tasks.workunit.client.0.smithi031.stdout:1/468: dread d2/d15/d32/f88 [0,4194304] 0 2024-09-16T08:57:21.108 INFO:tasks.workunit.client.0.smithi031.stdout:2/407: dwrite d6/da/fd [0,4194304] 0 2024-09-16T08:57:21.108 INFO:tasks.workunit.client.0.smithi031.stdout:6/593: write d0/dd/f23 [541789,112780] 0 2024-09-16T08:57:21.108 INFO:tasks.workunit.client.0.smithi031.stdout:6/594: readlink d0/dd/d2b/l95 0 2024-09-16T08:57:21.108 INFO:tasks.workunit.client.0.smithi031.stdout:6/595: fdatasync d0/d10/f58 0 2024-09-16T08:57:21.110 INFO:tasks.workunit.client.1.smithi097.stdout:2/993: dwrite d3/d4/d9/d67/d122/d61/fb1 [0,4194304] 0 2024-09-16T08:57:21.110 INFO:tasks.workunit.client.1.smithi097.stdout:8/895: creat d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/f11e x:0 0 0 2024-09-16T08:57:21.111 INFO:tasks.workunit.client.1.smithi097.stdout:6/995: dwrite d1/dd3/d9e/dca/dbb/f125 [0,4194304] 0 2024-09-16T08:57:21.111 INFO:tasks.workunit.client.1.smithi097.stdout:6/996: dread - d1/dc/f13e zero size 2024-09-16T08:57:21.111 INFO:tasks.workunit.client.1.smithi097.stdout:6/997: chown d1/dd3/d9e/dca/d16/f21 1135 1 2024-09-16T08:57:21.111 INFO:tasks.workunit.client.1.smithi097.stdout:6/998: write d1/dd3/d9e/f133 [761540,9907] 0 2024-09-16T08:57:21.111 INFO:tasks.workunit.client.1.smithi097.stdout:6/999: fsync d1/dd3/d9e/db3/f112 0 2024-09-16T08:57:21.116 INFO:tasks.workunit.client.0.smithi031.stdout:7/518: creat d2/d8/d86/da1/da9/fad x:0 0 0 2024-09-16T08:57:21.117 INFO:tasks.workunit.client.1.smithi097.stdout:2/994: truncate d3/d4/d9/d67/f84 153425 0 2024-09-16T08:57:21.119 INFO:tasks.workunit.client.1.smithi097.stdout:8/896: symlink d8/dd/d15/d2c/d4a/d4d/d62/l11f 0 2024-09-16T08:57:21.120 INFO:tasks.workunit.client.1.smithi097.stdout:8/897: read d8/dd/d42/f89 [136424,49860] 0 2024-09-16T08:57:21.123 INFO:tasks.workunit.client.0.smithi031.stdout:9/464: dwrite d3/d5/da/db/fe [0,4194304] 0 2024-09-16T08:57:21.127 INFO:tasks.workunit.client.1.smithi097.stdout:4/871: getdents d0/d7/d52/d26/d3d 0 2024-09-16T08:57:21.130 INFO:tasks.workunit.client.1.smithi097.stdout:2/995: dread d3/f71 [0,4194304] 0 2024-09-16T08:57:21.131 INFO:tasks.workunit.client.1.smithi097.stdout:8/898: mknod d8/dd/d15/d2c/d4a/d4d/d62/c120 0 2024-09-16T08:57:21.132 INFO:tasks.workunit.client.0.smithi031.stdout:4/472: creat d0/d2/d16/f99 x:0 0 0 2024-09-16T08:57:21.134 INFO:tasks.workunit.client.1.smithi097.stdout:4/872: creat d0/d7/f126 x:0 0 0 2024-09-16T08:57:21.136 INFO:tasks.workunit.client.1.smithi097.stdout:2/996: getdents d3/d4/d9/d67/d122/d61/dac/dd9 0 2024-09-16T08:57:21.137 INFO:tasks.workunit.client.0.smithi031.stdout:1/469: mknod d2/d4/d36/ca8 0 2024-09-16T08:57:21.139 INFO:tasks.workunit.client.1.smithi097.stdout:8/899: truncate d8/dd/d15/d2c/d4a/d4d/d3d/db9/fda 1545610 0 2024-09-16T08:57:21.140 INFO:tasks.workunit.client.0.smithi031.stdout:2/408: symlink d6/d2e/d40/d6c/d2c/l8a 0 2024-09-16T08:57:21.141 INFO:tasks.workunit.client.1.smithi097.stdout:9/949: sync 2024-09-16T08:57:21.141 INFO:tasks.workunit.client.1.smithi097.stdout:4/873: mknod d0/d7/df/d83/df8/c127 0 2024-09-16T08:57:21.141 INFO:tasks.workunit.client.1.smithi097.stdout:3/921: sync 2024-09-16T08:57:21.141 INFO:tasks.workunit.client.1.smithi097.stdout:7/848: sync 2024-09-16T08:57:21.143 INFO:tasks.workunit.client.0.smithi031.stdout:6/596: rmdir d0/dd/d74/d8b 39 2024-09-16T08:57:21.145 INFO:tasks.workunit.client.1.smithi097.stdout:2/997: creat d3/d4/d77/d126/f13d x:0 0 0 2024-09-16T08:57:21.145 INFO:tasks.workunit.client.1.smithi097.stdout:2/998: fsync d3/d4/d8/f25 0 2024-09-16T08:57:21.145 INFO:tasks.workunit.client.1.smithi097.stdout:2/999: readlink d3/d4/d9/d67/db2/lce 0 2024-09-16T08:57:21.145 INFO:tasks.workunit.client.0.smithi031.stdout:7/519: rename d2/d2d/f89 to d2/d2d/d6d/fae 0 2024-09-16T08:57:21.147 INFO:tasks.workunit.client.1.smithi097.stdout:4/874: link d0/d36/da7/faa d0/d7/d52/d26/f128 0 2024-09-16T08:57:21.149 INFO:tasks.workunit.client.0.smithi031.stdout:9/465: write d3/d5/da/db/f18 [1022319,30738] 0 2024-09-16T08:57:21.150 INFO:tasks.workunit.client.1.smithi097.stdout:7/849: creat d4/d9/d51/db7/d10e/f121 x:0 0 0 2024-09-16T08:57:21.150 INFO:tasks.workunit.client.1.smithi097.stdout:7/850: fsync d4/d9/d2b/f4d 0 2024-09-16T08:57:21.155 INFO:tasks.workunit.client.0.smithi031.stdout:4/473: rmdir d0/d2/d3a/d78 0 2024-09-16T08:57:21.158 INFO:tasks.workunit.client.1.smithi097.stdout:3/922: symlink d7/d15/dee/d11f/d30/d88/db3/l13b 0 2024-09-16T08:57:21.159 INFO:tasks.workunit.client.0.smithi031.stdout:5/460: rmdir d2/d4b 39 2024-09-16T08:57:21.160 INFO:tasks.workunit.client.1.smithi097.stdout:9/950: dread d2/d5/d56/faf [0,4194304] 0 2024-09-16T08:57:21.160 INFO:tasks.workunit.client.1.smithi097.stdout:9/951: dread - d2/d5/f10b zero size 2024-09-16T08:57:21.161 INFO:tasks.workunit.client.0.smithi031.stdout:2/409: mkdir d6/da/d37/d8b 0 2024-09-16T08:57:21.161 INFO:tasks.workunit.client.0.smithi031.stdout:2/410: chown d6/da/d4b/f83 0 1 2024-09-16T08:57:21.164 INFO:tasks.workunit.client.1.smithi097.stdout:4/875: rename d0/d7/df/d83/fae to d0/d35/f129 0 2024-09-16T08:57:21.164 INFO:tasks.workunit.client.1.smithi097.stdout:4/876: chown d0/d7/f10d 4 1 2024-09-16T08:57:21.166 INFO:tasks.workunit.client.1.smithi097.stdout:9/952: symlink d2/d5/d72/da2/l12d 0 2024-09-16T08:57:21.168 INFO:tasks.workunit.client.1.smithi097.stdout:7/851: read d4/f64 [221035,123960] 0 2024-09-16T08:57:21.169 INFO:tasks.workunit.client.0.smithi031.stdout:4/474: symlink d0/d71/l9a 0 2024-09-16T08:57:21.169 INFO:tasks.workunit.client.0.smithi031.stdout:4/475: chown d0/d2/d16/l92 21 1 2024-09-16T08:57:21.169 INFO:tasks.workunit.client.0.smithi031.stdout:4/476: readlink d0/d8e/d75/l88 0 2024-09-16T08:57:21.173 INFO:tasks.workunit.client.1.smithi097.stdout:3/923: dread d7/d15/dee/d11f/fd4 [0,4194304] 0 2024-09-16T08:57:21.173 INFO:tasks.workunit.client.0.smithi031.stdout:5/461: link d2/f63 d2/dc/d36/f94 0 2024-09-16T08:57:21.173 INFO:tasks.workunit.client.1.smithi097.stdout:7/852: getdents d4/d9/d51 0 2024-09-16T08:57:21.174 INFO:tasks.workunit.client.1.smithi097.stdout:7/853: chown d4/dfa/c117 2874796 1 2024-09-16T08:57:21.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/466: rmdir d3/d5/da/d34/d3c/d5e/d62/d65 39 2024-09-16T08:57:21.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/467: write d3/d5/da/db/d21/fa3 [475722,34390] 0 2024-09-16T08:57:21.178 INFO:tasks.workunit.client.1.smithi097.stdout:4/877: rmdir d0/d7/d52/dbf 39 2024-09-16T08:57:21.179 INFO:tasks.workunit.client.1.smithi097.stdout:3/924: dread d7/de/d3c/fe4 [0,4194304] 0 2024-09-16T08:57:21.179 INFO:tasks.workunit.client.1.smithi097.stdout:3/925: write d7/d15/dee/d11f/d97/fd3 [377951,89501] 0 2024-09-16T08:57:21.180 INFO:tasks.workunit.client.1.smithi097.stdout:4/878: symlink d0/d7/df/d15/l12a 0 2024-09-16T08:57:21.181 INFO:tasks.workunit.client.1.smithi097.stdout:4/879: stat d0/d23 0 2024-09-16T08:57:21.181 INFO:tasks.workunit.client.1.smithi097.stdout:4/880: write d0/d7/fe0 [680166,40996] 0 2024-09-16T08:57:21.181 INFO:tasks.workunit.client.1.smithi097.stdout:4/881: readlink d0/d7/l4d 0 2024-09-16T08:57:21.182 INFO:tasks.workunit.client.0.smithi031.stdout:2/411: read d6/d2e/d40/d6c/d2c/d53/d60/f35 [2392280,79609] 0 2024-09-16T08:57:21.184 INFO:tasks.workunit.client.1.smithi097.stdout:4/882: creat d0/d7/d52/d26/d3d/f12b x:0 0 0 2024-09-16T08:57:21.185 INFO:tasks.workunit.client.0.smithi031.stdout:7/520: dwrite d2/d2d/f77 [4194304,4194304] 0 2024-09-16T08:57:21.186 INFO:tasks.workunit.client.1.smithi097.stdout:4/883: stat d0/d36/c8e 0 2024-09-16T08:57:21.186 INFO:tasks.workunit.client.1.smithi097.stdout:4/884: write d0/fe [1923486,46165] 0 2024-09-16T08:57:21.189 INFO:tasks.workunit.client.1.smithi097.stdout:7/854: dread d4/d12/d1f/fbb [4194304,4194304] 0 2024-09-16T08:57:21.189 INFO:tasks.workunit.client.1.smithi097.stdout:7/855: read - d4/d7/d79/f82 zero size 2024-09-16T08:57:21.192 INFO:tasks.workunit.client.1.smithi097.stdout:8/900: dwrite d8/dd/d15/d16/fa1 [0,4194304] 0 2024-09-16T08:57:21.192 INFO:tasks.workunit.client.1.smithi097.stdout:4/885: link d0/f30 d0/d23/dda/f12c 0 2024-09-16T08:57:21.193 INFO:tasks.workunit.client.0.smithi031.stdout:6/597: dwrite d0/dd/d74/d19/fa1 [0,4194304] 0 2024-09-16T08:57:21.193 INFO:tasks.workunit.client.0.smithi031.stdout:4/477: write d0/d2/f33 [793220,37130] 0 2024-09-16T08:57:21.193 INFO:tasks.workunit.client.0.smithi031.stdout:2/412: dread d6/f9 [0,4194304] 0 2024-09-16T08:57:21.193 INFO:tasks.workunit.client.0.smithi031.stdout:2/413: dread - d6/d2e/d40/d6c/f5a zero size 2024-09-16T08:57:21.193 INFO:tasks.workunit.client.0.smithi031.stdout:2/414: chown d6/da/f1e 7556 1 2024-09-16T08:57:21.196 INFO:tasks.workunit.client.0.smithi031.stdout:7/521: rmdir d2/d8/d11/d5d 39 2024-09-16T08:57:21.199 INFO:tasks.workunit.client.1.smithi097.stdout:8/901: symlink d8/dd/d15/d2c/d4a/d4d/d3f/l121 0 2024-09-16T08:57:21.204 INFO:tasks.workunit.client.0.smithi031.stdout:6/598: mkdir d0/dd/d74/d67/dbb 0 2024-09-16T08:57:21.208 INFO:tasks.workunit.client.0.smithi031.stdout:4/478: mkdir d0/d2/d16/d59/d84/d86/d9b 0 2024-09-16T08:57:21.210 INFO:tasks.workunit.client.1.smithi097.stdout:7/856: mknod d4/d12/d1f/c122 0 2024-09-16T08:57:21.210 INFO:tasks.workunit.client.0.smithi031.stdout:4/479: dread d0/d2/d16/f19 [0,4194304] 0 2024-09-16T08:57:21.210 INFO:tasks.workunit.client.0.smithi031.stdout:4/480: dread - d0/d5b/f89 zero size 2024-09-16T08:57:21.211 INFO:tasks.workunit.client.0.smithi031.stdout:2/415: link d6/d2e/d40/f7a d6/da/d37/d8b/f8c 0 2024-09-16T08:57:21.212 INFO:tasks.workunit.client.0.smithi031.stdout:4/481: read d0/d3c/f7f [86516,54013] 0 2024-09-16T08:57:21.212 INFO:tasks.workunit.client.0.smithi031.stdout:4/482: write d0/d2/d3a/f93 [691944,9940] 0 2024-09-16T08:57:21.212 INFO:tasks.workunit.client.0.smithi031.stdout:4/483: write d0/d5b/f7d [1141099,111356] 0 2024-09-16T08:57:21.215 INFO:tasks.workunit.client.0.smithi031.stdout:2/416: mkdir d6/da/d5e/d8d 0 2024-09-16T08:57:21.215 INFO:tasks.workunit.client.0.smithi031.stdout:2/417: dread - d6/d2e/d40/d6c/d2c/f64 zero size 2024-09-16T08:57:21.215 INFO:tasks.workunit.client.0.smithi031.stdout:2/418: fsync d6/d2e/d40/d6c/f3b 0 2024-09-16T08:57:21.218 INFO:tasks.workunit.client.1.smithi097.stdout:8/902: rename d8/dd/d15/d2c/d4a/f87 to d8/dd/d15/d2c/d4a/d4e/f122 0 2024-09-16T08:57:21.218 INFO:tasks.workunit.client.1.smithi097.stdout:8/903: fdatasync d8/dd/d15/d2c/d4a/d4e/db2/d115/f11b 0 2024-09-16T08:57:21.219 INFO:tasks.workunit.client.0.smithi031.stdout:4/484: rename d0/d71/d96 to d0/d3c/d6b/d9c 0 2024-09-16T08:57:21.219 INFO:tasks.workunit.client.0.smithi031.stdout:4/485: chown d0/d5b 2465387 1 2024-09-16T08:57:21.220 INFO:tasks.workunit.client.1.smithi097.stdout:7/857: rename d4/d9/dc/f10f to d4/d9/d51/db7/df9/f123 0 2024-09-16T08:57:21.224 INFO:tasks.workunit.client.0.smithi031.stdout:2/419: dread d6/d20/f4f [4194304,4194304] 0 2024-09-16T08:57:21.224 INFO:tasks.workunit.client.0.smithi031.stdout:2/420: chown d6/da/d5e/d72/l80 225484 1 2024-09-16T08:57:21.224 INFO:tasks.workunit.client.0.smithi031.stdout:2/421: truncate d6/da/f1e 1057899 0 2024-09-16T08:57:21.225 INFO:tasks.workunit.client.1.smithi097.stdout:9/953: dwrite d2/d5/f11 [4194304,4194304] 0 2024-09-16T08:57:21.225 INFO:tasks.workunit.client.1.smithi097.stdout:9/954: write d2/d5/db0/dc8/f32 [4352466,20549] 0 2024-09-16T08:57:21.226 INFO:tasks.workunit.client.1.smithi097.stdout:9/955: write d2/d5/d56/f5d [2324382,17547] 0 2024-09-16T08:57:21.227 INFO:tasks.workunit.client.0.smithi031.stdout:2/422: mknod d6/d2e/c8e 0 2024-09-16T08:57:21.227 INFO:tasks.workunit.client.0.smithi031.stdout:2/423: mkdir d6/da/d4b/d8f 0 2024-09-16T08:57:21.228 INFO:tasks.workunit.client.0.smithi031.stdout:4/486: dread d0/d3c/d6b/f8a [0,4194304] 0 2024-09-16T08:57:21.229 INFO:tasks.workunit.client.0.smithi031.stdout:4/487: readlink d0/d2/d16/d2b/l57 0 2024-09-16T08:57:21.229 INFO:tasks.workunit.client.0.smithi031.stdout:2/424: creat d6/d2e/d40/d58/f90 x:0 0 0 2024-09-16T08:57:21.229 INFO:tasks.workunit.client.0.smithi031.stdout:2/425: chown d6/f9 14 1 2024-09-16T08:57:21.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/462: dwrite d2/dc/f19 [4194304,4194304] 0 2024-09-16T08:57:21.232 INFO:tasks.workunit.client.1.smithi097.stdout:8/904: write d8/dd/d15/d2c/d4a/d4d/d3d/d7b/fc0 [717445,98111] 0 2024-09-16T08:57:21.232 INFO:tasks.workunit.client.1.smithi097.stdout:8/905: chown d8/dd/d15/d2c/ldb 27560263 1 2024-09-16T08:57:21.232 INFO:tasks.workunit.client.1.smithi097.stdout:8/906: fdatasync d8/dd/d71/d9c/f11c 0 2024-09-16T08:57:21.233 INFO:tasks.workunit.client.1.smithi097.stdout:8/907: write d8/dd/d15/d2c/d4a/d4d/fa5 [334962,119502] 0 2024-09-16T08:57:21.236 INFO:tasks.workunit.client.1.smithi097.stdout:7/858: dread d4/d7/d30/ddf/f9b [0,4194304] 0 2024-09-16T08:57:21.236 INFO:tasks.workunit.client.1.smithi097.stdout:4/886: dwrite d0/d36/da7/f11d [0,4194304] 0 2024-09-16T08:57:21.236 INFO:tasks.workunit.client.0.smithi031.stdout:9/468: dwrite d3/d5/da/f6b [0,4194304] 0 2024-09-16T08:57:21.238 INFO:tasks.workunit.client.1.smithi097.stdout:7/859: write d4/d12/d1f/fc2 [626483,60309] 0 2024-09-16T08:57:21.238 INFO:tasks.workunit.client.1.smithi097.stdout:8/908: mkdir d8/dd/d15/d2c/d4a/d4d/d3d/d123 0 2024-09-16T08:57:21.242 INFO:tasks.workunit.client.0.smithi031.stdout:7/522: dwrite d2/d8/d11/d66/f72 [0,4194304] 0 2024-09-16T08:57:21.246 INFO:tasks.workunit.client.1.smithi097.stdout:4/887: symlink d0/d23/ddc/l12d 0 2024-09-16T08:57:21.249 INFO:tasks.workunit.client.1.smithi097.stdout:8/909: link d8/dd/d15/d16/f3c d8/dd/d15/d2c/d4a/d4d/d3d/db9/f124 0 2024-09-16T08:57:21.249 INFO:tasks.workunit.client.1.smithi097.stdout:8/910: fsync d8/dd/d15/d16/d7c/da2/fbb 0 2024-09-16T08:57:21.249 INFO:tasks.workunit.client.1.smithi097.stdout:8/911: fdatasync d8/dd/f7e 0 2024-09-16T08:57:21.249 INFO:tasks.workunit.client.1.smithi097.stdout:8/912: readlink d8/dd/d71/lf8 0 2024-09-16T08:57:21.250 INFO:tasks.workunit.client.1.smithi097.stdout:4/888: rename d0/d23/ddc/ff6 to d0/d35/f12e 0 2024-09-16T08:57:21.254 INFO:tasks.workunit.client.1.smithi097.stdout:4/889: dread d0/d39/d71/f77 [4194304,4194304] 0 2024-09-16T08:57:21.255 INFO:tasks.workunit.client.0.smithi031.stdout:2/426: dread d6/f13 [0,4194304] 0 2024-09-16T08:57:21.257 INFO:tasks.workunit.client.1.smithi097.stdout:8/913: rename d8/dd/d15/d2c/d4a/d4d/l6b to d8/dd/d15/d2c/d4a/d4e/db2/l125 0 2024-09-16T08:57:21.257 INFO:tasks.workunit.client.0.smithi031.stdout:9/469: symlink d3/d5/da/db/d21/d3b/la4 0 2024-09-16T08:57:21.257 INFO:tasks.workunit.client.0.smithi031.stdout:9/470: read - d3/d48/f9d zero size 2024-09-16T08:57:21.260 INFO:tasks.workunit.client.1.smithi097.stdout:7/860: dread d4/d9/d1c/ff0 [0,4194304] 0 2024-09-16T08:57:21.261 INFO:tasks.workunit.client.1.smithi097.stdout:8/914: symlink d8/dd/d15/d2c/d4a/df3/l126 0 2024-09-16T08:57:21.261 INFO:tasks.workunit.client.1.smithi097.stdout:8/915: chown d8/dd/d15/d2c/d4a/d4d/d3f/l44 116 1 2024-09-16T08:57:21.262 INFO:tasks.workunit.client.0.smithi031.stdout:4/488: dwrite d0/d2/f41 [0,4194304] 0 2024-09-16T08:57:21.262 INFO:tasks.workunit.client.0.smithi031.stdout:7/523: creat d2/d8/d11/d5d/d7e/faf x:0 0 0 2024-09-16T08:57:21.262 INFO:tasks.workunit.client.0.smithi031.stdout:7/524: chown d2/d8/d11/d5d/d6e/l80 3361094 1 2024-09-16T08:57:21.263 INFO:tasks.workunit.client.0.smithi031.stdout:3/563: dwrite d3/d14/d37/d1a/d1d/d1e/f81 [4194304,4194304] 0 2024-09-16T08:57:21.263 INFO:tasks.workunit.client.1.smithi097.stdout:9/956: dwrite d2/d5/fa [0,4194304] 0 2024-09-16T08:57:21.263 INFO:tasks.workunit.client.1.smithi097.stdout:9/957: fdatasync d2/d5/d8d/db1/fd7 0 2024-09-16T08:57:21.264 INFO:tasks.workunit.client.1.smithi097.stdout:7/861: mkdir d4/d7/d77/da8/d124 0 2024-09-16T08:57:21.265 INFO:tasks.workunit.client.0.smithi031.stdout:5/463: dwrite d2/dc/d36/f94 [0,4194304] 0 2024-09-16T08:57:21.266 INFO:tasks.workunit.client.0.smithi031.stdout:2/427: mkdir d6/d2e/d40/d6c/d2c/d53/d60/d91 0 2024-09-16T08:57:21.268 INFO:tasks.workunit.client.1.smithi097.stdout:8/916: truncate d8/dd/d15/d16/ff6 363383 0 2024-09-16T08:57:21.268 INFO:tasks.workunit.client.1.smithi097.stdout:8/917: chown d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/fd3 60991 1 2024-09-16T08:57:21.272 INFO:tasks.workunit.client.0.smithi031.stdout:9/471: creat d3/d5/da/db/d21/d3b/fa5 x:0 0 0 2024-09-16T08:57:21.276 INFO:tasks.workunit.client.1.smithi097.stdout:9/958: getdents d2/d5/d72 0 2024-09-16T08:57:21.279 INFO:tasks.workunit.client.0.smithi031.stdout:4/489: symlink d0/d3c/d6b/l9d 0 2024-09-16T08:57:21.282 INFO:tasks.workunit.client.0.smithi031.stdout:7/525: creat d2/d8/d69/d90/fb0 x:0 0 0 2024-09-16T08:57:21.285 INFO:tasks.workunit.client.1.smithi097.stdout:7/862: mkdir d4/d9/d125 0 2024-09-16T08:57:21.285 INFO:tasks.workunit.client.1.smithi097.stdout:7/863: write d4/d9/dc/f90 [1747815,21373] 0 2024-09-16T08:57:21.286 INFO:tasks.workunit.client.1.smithi097.stdout:7/864: truncate d4/d7/d7b/d43/daf/ffd 493206 0 2024-09-16T08:57:21.286 INFO:tasks.workunit.client.0.smithi031.stdout:3/564: truncate d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 609947 0 2024-09-16T08:57:21.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/428: dwrite d6/da/f3d [0,4194304] 0 2024-09-16T08:57:21.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/429: truncate d6/d2e/d40/f7a 456732 0 2024-09-16T08:57:21.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/430: stat d6/da/d37/d8b 0 2024-09-16T08:57:21.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/431: creat d6/da/f92 x:0 0 0 2024-09-16T08:57:21.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/432: chown f1 556 1 2024-09-16T08:57:21.289 INFO:tasks.workunit.client.0.smithi031.stdout:2/433: creat d6/d2e/d40/d6c/d2c/d53/d60/f93 x:0 0 0 2024-09-16T08:57:21.290 INFO:tasks.workunit.client.0.smithi031.stdout:2/434: creat d6/d2e/d40/f94 x:0 0 0 2024-09-16T08:57:21.291 INFO:tasks.workunit.client.0.smithi031.stdout:9/472: mknod d3/d5/da/d34/ca6 0 2024-09-16T08:57:21.291 INFO:tasks.workunit.client.0.smithi031.stdout:9/473: read - d3/d85/f91 zero size 2024-09-16T08:57:21.293 INFO:tasks.workunit.client.0.smithi031.stdout:5/464: dwrite d2/d4b/f8a [0,4194304] 0 2024-09-16T08:57:21.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/526: stat d2/d8/d11/f44 0 2024-09-16T08:57:21.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/527: readlink d2/d8/d11/d5d/d6e/l80 0 2024-09-16T08:57:21.298 INFO:tasks.workunit.client.1.smithi097.stdout:9/959: rename d2/d5/db0/dc8/d63/da8/dca/df5/f102 to d2/d5/db0/d11f/f12e 0 2024-09-16T08:57:21.308 INFO:tasks.workunit.client.1.smithi097.stdout:9/960: dread d2/d5/db0/dc8/f5a [0,4194304] 0 2024-09-16T08:57:21.308 INFO:tasks.workunit.client.1.smithi097.stdout:9/961: creat d2/d5/d3a/d3c/d73/f12f x:0 0 0 2024-09-16T08:57:21.309 INFO:tasks.workunit.client.0.smithi031.stdout:2/435: symlink d6/da/l95 0 2024-09-16T08:57:21.311 INFO:tasks.workunit.client.0.smithi031.stdout:9/474: mknod d3/d5/da/d34/d3c/d5e/d62/ca7 0 2024-09-16T08:57:21.311 INFO:tasks.workunit.client.0.smithi031.stdout:9/475: fsync d3/d5/da/d34/d4d/f66 0 2024-09-16T08:57:21.314 INFO:tasks.workunit.client.0.smithi031.stdout:5/465: symlink d2/dc/d3d/l95 0 2024-09-16T08:57:21.314 INFO:tasks.workunit.client.0.smithi031.stdout:5/466: write d2/dc/f16 [1506679,51093] 0 2024-09-16T08:57:21.314 INFO:tasks.workunit.client.0.smithi031.stdout:4/490: dwrite d0/d5b/f69 [0,4194304] 0 2024-09-16T08:57:21.315 INFO:tasks.workunit.client.0.smithi031.stdout:4/491: write d0/d2/d16/d1e/f3b [1333809,32003] 0 2024-09-16T08:57:21.315 INFO:tasks.workunit.client.1.smithi097.stdout:7/865: dwrite d4/d12/d1f/f3b [0,4194304] 0 2024-09-16T08:57:21.315 INFO:tasks.workunit.client.1.smithi097.stdout:7/866: write d4/d7/d7b/d43/daf/fc5 [429612,92812] 0 2024-09-16T08:57:21.315 INFO:tasks.workunit.client.1.smithi097.stdout:7/867: write d4/d7/d19/f11d [490613,77971] 0 2024-09-16T08:57:21.316 INFO:tasks.workunit.client.0.smithi031.stdout:7/528: symlink d2/d2d/d6d/lb1 0 2024-09-16T08:57:21.317 INFO:tasks.workunit.client.0.smithi031.stdout:3/565: getdents d3/d14/d37/d33 0 2024-09-16T08:57:21.320 INFO:tasks.workunit.client.0.smithi031.stdout:2/436: symlink d6/da/d37/d42/l96 0 2024-09-16T08:57:21.320 INFO:tasks.workunit.client.1.smithi097.stdout:9/962: symlink d2/d5/d56/l130 0 2024-09-16T08:57:21.321 INFO:tasks.workunit.client.0.smithi031.stdout:9/476: mkdir d3/d5/d6a/d74/da8 0 2024-09-16T08:57:21.321 INFO:tasks.workunit.client.0.smithi031.stdout:9/477: truncate d3/d5/da/f89 598967 0 2024-09-16T08:57:21.322 INFO:tasks.workunit.client.0.smithi031.stdout:9/478: dread - d3/d5/f5d zero size 2024-09-16T08:57:21.322 INFO:tasks.workunit.client.0.smithi031.stdout:9/479: dread - d3/d5/da/d34/d3c/d5e/d62/d65/f94 zero size 2024-09-16T08:57:21.330 INFO:tasks.workunit.client.1.smithi097.stdout:9/963: unlink d2/d5/fa 0 2024-09-16T08:57:21.330 INFO:tasks.workunit.client.1.smithi097.stdout:9/964: creat d2/d5/d72/da2/f131 x:0 0 0 2024-09-16T08:57:21.331 INFO:tasks.workunit.client.0.smithi031.stdout:4/492: getdents d0/d2/d16/d1e 0 2024-09-16T08:57:21.332 INFO:tasks.workunit.client.0.smithi031.stdout:7/529: unlink l0 0 2024-09-16T08:57:21.334 INFO:tasks.workunit.client.0.smithi031.stdout:3/566: dwrite d3/f5a [0,4194304] 0 2024-09-16T08:57:21.334 INFO:tasks.workunit.client.1.smithi097.stdout:9/965: rmdir d2/d5/db0/dc8/d74/d8c/d94/d96/dac/dfd 39 2024-09-16T08:57:21.335 INFO:tasks.workunit.client.1.smithi097.stdout:9/966: rename d2/d5/db0/dc8/d74 to d2/d5/db0/dc8/d74/d8c/dad/d132 22 2024-09-16T08:57:21.335 INFO:tasks.workunit.client.1.smithi097.stdout:9/967: chown d2/d5/db0/dc8/d74/d8c/d94/f125 1015 1 2024-09-16T08:57:21.335 INFO:tasks.workunit.client.1.smithi097.stdout:9/968: write d2/d5/d3a/fa4 [721214,15869] 0 2024-09-16T08:57:21.335 INFO:tasks.workunit.client.1.smithi097.stdout:8/918: rmdir d8/dd/d15 39 2024-09-16T08:57:21.337 INFO:tasks.workunit.client.1.smithi097.stdout:9/969: creat d2/d5/d72/f133 x:0 0 0 2024-09-16T08:57:21.338 INFO:tasks.workunit.client.1.smithi097.stdout:8/919: mknod d8/dd/d15/d2c/d4a/d4e/d85/c127 0 2024-09-16T08:57:21.338 INFO:tasks.workunit.client.1.smithi097.stdout:8/920: dread - d8/dd/d15/d2c/d4a/d4d/d3d/d7b/ff4 zero size 2024-09-16T08:57:21.338 INFO:tasks.workunit.client.1.smithi097.stdout:8/921: chown d8/f13 55 1 2024-09-16T08:57:21.339 INFO:tasks.workunit.client.0.smithi031.stdout:9/480: truncate d3/d5/da/db/fe 1175799 0 2024-09-16T08:57:21.339 INFO:tasks.workunit.client.0.smithi031.stdout:9/481: dread - d3/d5/da/d34/d3c/d5e/d62/d65/f94 zero size 2024-09-16T08:57:21.342 INFO:tasks.workunit.client.1.smithi097.stdout:9/970: rename d2/d5/db0/dc8/d63/da8/dca/cdb to d2/d5/d72/c134 0 2024-09-16T08:57:21.342 INFO:tasks.workunit.client.1.smithi097.stdout:9/971: write d2/d5/db0/dc8/d74/d8c/dad/d10d/fe2 [1026544,100784] 0 2024-09-16T08:57:21.342 INFO:tasks.workunit.client.1.smithi097.stdout:9/972: creat d2/d5/d72/da2/f135 x:0 0 0 2024-09-16T08:57:21.343 INFO:tasks.workunit.client.0.smithi031.stdout:5/467: rename d2/dc/d36/d76/l77 to d2/dc/d25/d21/l96 0 2024-09-16T08:57:21.344 INFO:tasks.workunit.client.1.smithi097.stdout:8/922: creat d8/dd/d15/d2c/d4a/d4d/ddd/f128 x:0 0 0 2024-09-16T08:57:21.344 INFO:tasks.workunit.client.0.smithi031.stdout:5/468: fdatasync d2/f13 0 2024-09-16T08:57:21.345 INFO:tasks.workunit.client.0.smithi031.stdout:4/493: rmdir d0/d3c/d6b 39 2024-09-16T08:57:21.345 INFO:tasks.workunit.client.0.smithi031.stdout:4/494: fdatasync d0/d2/d3a/d55/f64 0 2024-09-16T08:57:21.348 INFO:tasks.workunit.client.0.smithi031.stdout:9/482: read d3/d5/da/db/d21/f2f [1089997,18893] 0 2024-09-16T08:57:21.348 INFO:tasks.workunit.client.0.smithi031.stdout:9/483: chown d3/d5/da/d34/d3c/d5e/d9c 335357 1 2024-09-16T08:57:21.349 INFO:tasks.workunit.client.1.smithi097.stdout:9/973: mknod d2/d5/d3a/d70/c136 0 2024-09-16T08:57:21.352 INFO:tasks.workunit.client.0.smithi031.stdout:3/567: rmdir d3/d14/d37 39 2024-09-16T08:57:21.352 INFO:tasks.workunit.client.0.smithi031.stdout:3/568: stat d3/d14/d37/d1a/d1d/d3b/d94/la1 0 2024-09-16T08:57:21.352 INFO:tasks.workunit.client.0.smithi031.stdout:3/569: fsync d3/d14/f2e 0 2024-09-16T08:57:21.353 INFO:tasks.workunit.client.1.smithi097.stdout:8/923: write d8/fc [1449621,117942] 0 2024-09-16T08:57:21.354 INFO:tasks.workunit.client.0.smithi031.stdout:2/437: rename d6/da/d5e/d72/l80 to d6/d20/d51/l97 0 2024-09-16T08:57:21.354 INFO:tasks.workunit.client.0.smithi031.stdout:2/438: fsync d6/d2e/d40/f7a 0 2024-09-16T08:57:21.355 INFO:tasks.workunit.client.0.smithi031.stdout:5/469: mkdir d2/dc/d1d/d97 0 2024-09-16T08:57:21.355 INFO:tasks.workunit.client.1.smithi097.stdout:9/974: dread d2/f4 [0,4194304] 0 2024-09-16T08:57:21.355 INFO:tasks.workunit.client.1.smithi097.stdout:9/975: write d2/f25 [1641696,110383] 0 2024-09-16T08:57:21.357 INFO:tasks.workunit.client.0.smithi031.stdout:7/530: rename d2/d8/d11/c64 to d2/d8/d11/cb2 0 2024-09-16T08:57:21.358 INFO:tasks.workunit.client.0.smithi031.stdout:9/484: symlink d3/d5/da/db/d21/d67/la9 0 2024-09-16T08:57:21.359 INFO:tasks.workunit.client.0.smithi031.stdout:9/485: chown d3/d5/da/d34 10708 1 2024-09-16T08:57:21.359 INFO:tasks.workunit.client.0.smithi031.stdout:9/486: chown d3/d5/d73/f86 0 1 2024-09-16T08:57:21.359 INFO:tasks.workunit.client.0.smithi031.stdout:9/487: fdatasync d3/d5/da/d34/d3c/d5e/d62/f84 0 2024-09-16T08:57:21.359 INFO:tasks.workunit.client.1.smithi097.stdout:9/976: creat d2/d5/d3a/dfb/f137 x:0 0 0 2024-09-16T08:57:21.360 INFO:tasks.workunit.client.0.smithi031.stdout:3/570: symlink d3/d14/d6f/la5 0 2024-09-16T08:57:21.361 INFO:tasks.workunit.client.0.smithi031.stdout:5/470: mknod d2/d4b/c98 0 2024-09-16T08:57:21.362 INFO:tasks.workunit.client.1.smithi097.stdout:9/977: mknod d2/d3f/d127/c138 0 2024-09-16T08:57:21.363 INFO:tasks.workunit.client.0.smithi031.stdout:7/531: unlink d2/d2d/l76 0 2024-09-16T08:57:21.364 INFO:tasks.workunit.client.0.smithi031.stdout:9/488: getdents d3/d5/d73/d81 0 2024-09-16T08:57:21.367 INFO:tasks.workunit.client.0.smithi031.stdout:9/489: truncate d3/d5/da/f2b 1902882 0 2024-09-16T08:57:21.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/532: getdents d2/d8/d11/d66 0 2024-09-16T08:57:21.369 INFO:tasks.workunit.client.0.smithi031.stdout:4/495: dwrite d0/d2/d3a/d55/f64 [4194304,4194304] 0 2024-09-16T08:57:21.370 INFO:tasks.workunit.client.0.smithi031.stdout:2/439: dread d6/da/fd [0,4194304] 0 2024-09-16T08:57:21.370 INFO:tasks.workunit.client.0.smithi031.stdout:9/490: creat d3/d5/da/db/faa x:0 0 0 2024-09-16T08:57:21.371 INFO:tasks.workunit.client.0.smithi031.stdout:5/471: write d2/dc/f3b [1002899,16318] 0 2024-09-16T08:57:21.371 INFO:tasks.workunit.client.0.smithi031.stdout:5/472: read d2/dc/f3b [2559299,56425] 0 2024-09-16T08:57:21.372 INFO:tasks.workunit.client.0.smithi031.stdout:2/440: rename d6/da/l14 to d6/da/d4b/d4c/d68/l98 0 2024-09-16T08:57:21.372 INFO:tasks.workunit.client.0.smithi031.stdout:2/441: fdatasync d6/da/f1e 0 2024-09-16T08:57:21.373 INFO:tasks.workunit.client.0.smithi031.stdout:5/473: truncate d2/dc/d36/f52 209491 0 2024-09-16T08:57:21.373 INFO:tasks.workunit.client.0.smithi031.stdout:5/474: write d2/d4b/f8b [979408,73306] 0 2024-09-16T08:57:21.374 INFO:tasks.workunit.client.1.smithi097.stdout:9/978: write d2/d5/dd5/f105 [1199441,41382] 0 2024-09-16T08:57:21.381 INFO:tasks.workunit.client.0.smithi031.stdout:7/533: write d2/d2d/f77 [6681788,72674] 0 2024-09-16T08:57:21.381 INFO:tasks.workunit.client.0.smithi031.stdout:7/534: dread - d2/d8/d86/f88 zero size 2024-09-16T08:57:21.381 INFO:tasks.workunit.client.0.smithi031.stdout:2/442: symlink d6/l99 0 2024-09-16T08:57:21.381 INFO:tasks.workunit.client.0.smithi031.stdout:2/443: dread - d6/d2e/d40/f94 zero size 2024-09-16T08:57:21.382 INFO:tasks.workunit.client.0.smithi031.stdout:9/491: rename d3/d5/da/db/l87 to d3/d5/da/db/d21/lab 0 2024-09-16T08:57:21.389 INFO:tasks.workunit.client.0.smithi031.stdout:3/571: dwrite d3/f5a [0,4194304] 0 2024-09-16T08:57:21.389 INFO:tasks.workunit.client.0.smithi031.stdout:3/572: fdatasync d3/d14/d37/d1a/f65 0 2024-09-16T08:57:21.389 INFO:tasks.workunit.client.1.smithi097.stdout:8/924: dwrite d8/fc [0,4194304] 0 2024-09-16T08:57:21.390 INFO:tasks.workunit.client.0.smithi031.stdout:5/475: dread d2/dc/d25/f2f [0,4194304] 0 2024-09-16T08:57:21.395 INFO:tasks.workunit.client.1.smithi097.stdout:9/979: unlink d2/d5/d56/f88 0 2024-09-16T08:57:21.396 INFO:tasks.workunit.client.1.smithi097.stdout:9/980: write d2/d5/d3a/d3c/d73/fda [1037105,94307] 0 2024-09-16T08:57:21.396 INFO:tasks.workunit.client.1.smithi097.stdout:9/981: readlink d2/d5/d72/da2/l12d 0 2024-09-16T08:57:21.396 INFO:tasks.workunit.client.0.smithi031.stdout:2/444: creat d6/d2e/f9a x:0 0 0 2024-09-16T08:57:21.397 INFO:tasks.workunit.client.1.smithi097.stdout:8/925: symlink d8/dd/d15/d16/d7c/da2/l129 0 2024-09-16T08:57:21.399 INFO:tasks.workunit.client.1.smithi097.stdout:9/982: mkdir d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/d139 0 2024-09-16T08:57:21.403 INFO:tasks.workunit.client.0.smithi031.stdout:9/492: dwrite d3/d5/da/d34/f5c [4194304,4194304] 0 2024-09-16T08:57:21.404 INFO:tasks.workunit.client.0.smithi031.stdout:2/445: dread d6/da/f1e [0,4194304] 0 2024-09-16T08:57:21.404 INFO:tasks.workunit.client.0.smithi031.stdout:2/446: truncate d6/da/f84 527669 0 2024-09-16T08:57:21.407 INFO:tasks.workunit.client.0.smithi031.stdout:0/471: sync 2024-09-16T08:57:21.407 INFO:tasks.workunit.client.0.smithi031.stdout:8/401: sync 2024-09-16T08:57:21.408 INFO:tasks.workunit.client.0.smithi031.stdout:8/402: write d4/d34/f6a [4678858,25838] 0 2024-09-16T08:57:21.410 INFO:tasks.workunit.client.1.smithi097.stdout:9/983: write d2/f46 [3026415,120836] 0 2024-09-16T08:57:21.416 INFO:tasks.workunit.client.0.smithi031.stdout:1/470: sync 2024-09-16T08:57:21.416 INFO:tasks.workunit.client.0.smithi031.stdout:6/599: sync 2024-09-16T08:57:21.417 INFO:tasks.workunit.client.1.smithi097.stdout:8/926: dwrite d8/dd/f26 [0,4194304] 0 2024-09-16T08:57:21.417 INFO:tasks.workunit.client.1.smithi097.stdout:8/927: chown d8/dd/d71/le2 215255373 1 2024-09-16T08:57:21.419 INFO:tasks.workunit.client.0.smithi031.stdout:5/476: creat d2/dc/d91/f99 x:0 0 0 2024-09-16T08:57:21.424 INFO:tasks.workunit.client.0.smithi031.stdout:2/447: creat d6/da/d4b/d4c/d73/f9b x:0 0 0 2024-09-16T08:57:21.429 INFO:tasks.workunit.client.0.smithi031.stdout:7/535: rmdir d2 39 2024-09-16T08:57:21.429 INFO:tasks.workunit.client.0.smithi031.stdout:6/600: dread d0/dd/d74/d5e/f77 [0,4194304] 0 2024-09-16T08:57:21.429 INFO:tasks.workunit.client.0.smithi031.stdout:7/536: write d2/d8/d69/d82/f9b [11637,81834] 0 2024-09-16T08:57:21.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/573: mknod d3/d14/d37/d1a/d1d/d3b/ca6 0 2024-09-16T08:57:21.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/574: chown d3/d14/d37/d1a/d1d/d1e/d67/d7f 310861822 1 2024-09-16T08:57:21.432 INFO:tasks.workunit.client.0.smithi031.stdout:9/493: truncate d3/d5/da/f89 587260 0 2024-09-16T08:57:21.432 INFO:tasks.workunit.client.0.smithi031.stdout:5/477: write d2/f34 [3000169,15379] 0 2024-09-16T08:57:21.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/448: symlink d6/da/d4b/d4c/l9c 0 2024-09-16T08:57:21.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/449: stat d6/d2e/d40/d6c/d2c 0 2024-09-16T08:57:21.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/450: fsync f1 0 2024-09-16T08:57:21.433 INFO:tasks.workunit.client.0.smithi031.stdout:2/451: fdatasync d6/da/d4b/d4c/f5f 0 2024-09-16T08:57:21.435 INFO:tasks.workunit.client.0.smithi031.stdout:0/472: rename d6/d12/d13/f5d to d6/d12/d13/d36/f96 0 2024-09-16T08:57:21.436 INFO:tasks.workunit.client.0.smithi031.stdout:1/471: getdents d2/d15 0 2024-09-16T08:57:21.437 INFO:tasks.workunit.client.0.smithi031.stdout:1/472: chown d2/d15/d1f/c5e 0 1 2024-09-16T08:57:21.438 INFO:tasks.workunit.client.0.smithi031.stdout:1/473: dread d2/f3e [0,4194304] 0 2024-09-16T08:57:21.438 INFO:tasks.workunit.client.0.smithi031.stdout:1/474: chown d2/d4 3953353 1 2024-09-16T08:57:21.438 INFO:tasks.workunit.client.0.smithi031.stdout:5/478: rmdir d2/dc/d36/d76 39 2024-09-16T08:57:21.441 INFO:tasks.workunit.client.1.smithi097.stdout:9/984: dwrite d2/d5/dd5/f105 [0,4194304] 0 2024-09-16T08:57:21.441 INFO:tasks.workunit.client.1.smithi097.stdout:9/985: read d2/fe [1211989,111343] 0 2024-09-16T08:57:21.441 INFO:tasks.workunit.client.1.smithi097.stdout:9/986: fsync d2/d5/db0/dc8/d74/d8c/d94/f125 0 2024-09-16T08:57:21.442 INFO:tasks.workunit.client.1.smithi097.stdout:8/928: dwrite d8/dd/d15/d2c/d4a/d4d/ddd/f128 [0,4194304] 0 2024-09-16T08:57:21.442 INFO:tasks.workunit.client.1.smithi097.stdout:8/929: truncate d8/dd/d15/d2c/d4a/d4d/d3d/db9/fda 2210853 0 2024-09-16T08:57:21.442 INFO:tasks.workunit.client.0.smithi031.stdout:6/601: read d0/dd/d35/d6c/d83/f93 [2569952,105132] 0 2024-09-16T08:57:21.442 INFO:tasks.workunit.client.0.smithi031.stdout:7/537: rename d2/d8/d69/d90 to d2/d8/d11/d66/d9a/d9c/db3 0 2024-09-16T08:57:21.442 INFO:tasks.workunit.client.0.smithi031.stdout:2/452: dread d6/f36 [0,4194304] 0 2024-09-16T08:57:21.442 INFO:tasks.workunit.client.0.smithi031.stdout:1/475: mkdir d2/d4/d1b/d5c/d6b/da9 0 2024-09-16T08:57:21.444 INFO:tasks.workunit.client.0.smithi031.stdout:5/479: creat d2/dc/d36/d76/f9a x:0 0 0 2024-09-16T08:57:21.446 INFO:tasks.workunit.client.1.smithi097.stdout:9/987: creat d2/d5/db0/dc8/d74/d8c/f13a x:0 0 0 2024-09-16T08:57:21.448 INFO:tasks.workunit.client.0.smithi031.stdout:6/602: dread d0/d10/f60 [0,4194304] 0 2024-09-16T08:57:21.449 INFO:tasks.workunit.client.0.smithi031.stdout:3/575: rename d3/d14/d37/d33/c12 to d3/d14/d37/d1a/d1d/d9a/ca7 0 2024-09-16T08:57:21.451 INFO:tasks.workunit.client.0.smithi031.stdout:2/453: symlink d6/da/d5e/d8d/l9d 0 2024-09-16T08:57:21.451 INFO:tasks.workunit.client.0.smithi031.stdout:2/454: read - d6/d20/d51/f62 zero size 2024-09-16T08:57:21.454 INFO:tasks.workunit.client.0.smithi031.stdout:9/494: dread d3/d48/f59 [0,4194304] 0 2024-09-16T08:57:21.455 INFO:tasks.workunit.client.0.smithi031.stdout:1/476: symlink d2/d9b/da3/laa 0 2024-09-16T08:57:21.455 INFO:tasks.workunit.client.0.smithi031.stdout:0/473: dwrite d6/d12/d47/d4d/f7a [4194304,4194304] 0 2024-09-16T08:57:21.455 INFO:tasks.workunit.client.0.smithi031.stdout:7/538: dread d2/d8/f2f [0,4194304] 0 2024-09-16T08:57:21.456 INFO:tasks.workunit.client.0.smithi031.stdout:5/480: getdents d2/dc/d3d 0 2024-09-16T08:57:21.457 INFO:tasks.workunit.client.1.smithi097.stdout:8/930: getdents d8/dd/d15/d2c/d4a/d4e/d97 0 2024-09-16T08:57:21.458 INFO:tasks.workunit.client.0.smithi031.stdout:6/603: creat d0/dd/d74/d19/db4/fbc x:0 0 0 2024-09-16T08:57:21.458 INFO:tasks.workunit.client.1.smithi097.stdout:9/988: symlink d2/d3f/d127/l13b 0 2024-09-16T08:57:21.460 INFO:tasks.workunit.client.0.smithi031.stdout:3/576: rename d3/d14/d37/d1a/d1d/d3b/d3f/l46 to d3/d14/d6f/d82/la8 0 2024-09-16T08:57:21.460 INFO:tasks.workunit.client.0.smithi031.stdout:1/477: rename d2 to d2/d4/d1b/dab 22 2024-09-16T08:57:21.460 INFO:tasks.workunit.client.0.smithi031.stdout:3/577: dread - d3/d14/d37/f84 zero size 2024-09-16T08:57:21.460 INFO:tasks.workunit.client.0.smithi031.stdout:3/578: stat d3/d14/d37/d1a/f5c 0 2024-09-16T08:57:21.461 INFO:tasks.workunit.client.1.smithi097.stdout:3/926: sync 2024-09-16T08:57:21.462 INFO:tasks.workunit.client.0.smithi031.stdout:2/455: creat d6/da/d5e/d8d/f9e x:0 0 0 2024-09-16T08:57:21.462 INFO:tasks.workunit.client.0.smithi031.stdout:2/456: read - d6/d2e/d40/d6c/d2c/d53/d60/f93 zero size 2024-09-16T08:57:21.465 INFO:tasks.workunit.client.1.smithi097.stdout:8/931: creat d8/dd/d15/d2c/d4a/d4d/d3d/d123/f12a x:0 0 0 2024-09-16T08:57:21.465 INFO:tasks.workunit.client.1.smithi097.stdout:8/932: getdents d8/dd/d15/d2c/d4a/d4e/d97 0 2024-09-16T08:57:21.468 INFO:tasks.workunit.client.1.smithi097.stdout:9/989: link d2/d5/d3a/d70/c10e d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/d108/d121/c13c 0 2024-09-16T08:57:21.470 INFO:tasks.workunit.client.0.smithi031.stdout:7/539: unlink d2/d8/d11/d5d/d6e/caa 0 2024-09-16T08:57:21.472 INFO:tasks.workunit.client.1.smithi097.stdout:3/927: truncate d7/de/d3c/d3f/d60/dcb/d35/f27 5589025 0 2024-09-16T08:57:21.472 INFO:tasks.workunit.client.1.smithi097.stdout:3/928: chown d7/cd9 54 1 2024-09-16T08:57:21.472 INFO:tasks.workunit.client.1.smithi097.stdout:3/929: write d7/d15/dee/d11f/d30/f8e [435428,106426] 0 2024-09-16T08:57:21.472 INFO:tasks.workunit.client.1.smithi097.stdout:3/930: readlink d7/d15/d52/lc0 0 2024-09-16T08:57:21.473 INFO:tasks.workunit.client.0.smithi031.stdout:5/481: truncate d2/dc/d25/d21/f4e 2392721 0 2024-09-16T08:57:21.473 INFO:tasks.workunit.client.0.smithi031.stdout:9/495: dwrite d3/d5/da/f33 [0,4194304] 0 2024-09-16T08:57:21.473 INFO:tasks.workunit.client.0.smithi031.stdout:9/496: read d3/f5b [1400479,68394] 0 2024-09-16T08:57:21.475 INFO:tasks.workunit.client.0.smithi031.stdout:0/474: write d6/f27 [1293625,122156] 0 2024-09-16T08:57:21.475 INFO:tasks.workunit.client.1.smithi097.stdout:8/933: unlink d8/dd/f19 0 2024-09-16T08:57:21.476 INFO:tasks.workunit.client.1.smithi097.stdout:8/934: chown d8/dd/d15/d2c/d4a/d4e/d85/c127 88880870 1 2024-09-16T08:57:21.476 INFO:tasks.workunit.client.1.smithi097.stdout:8/935: dread - d8/dd/d15/d2c/d4a/d4d/d3d/d123/f12a zero size 2024-09-16T08:57:21.477 INFO:tasks.workunit.client.0.smithi031.stdout:6/604: mknod d0/dd/d74/d19/d54/cbd 0 2024-09-16T08:57:21.480 INFO:tasks.workunit.client.0.smithi031.stdout:3/579: rename d3/d14/d37/d1a/d24/c4e to d3/d14/d6f/d7d/ca9 0 2024-09-16T08:57:21.481 INFO:tasks.workunit.client.1.smithi097.stdout:9/990: truncate d2/d5/d8d/db1/f124 1259260 0 2024-09-16T08:57:21.481 INFO:tasks.workunit.client.0.smithi031.stdout:2/457: mknod d6/d2e/c9f 0 2024-09-16T08:57:21.486 INFO:tasks.workunit.client.0.smithi031.stdout:9/497: dread d3/d5/f6 [0,4194304] 0 2024-09-16T08:57:21.489 INFO:tasks.workunit.client.1.smithi097.stdout:3/931: truncate d7/de/d3c/d3f/d60/dcb/d35/f22 2883037 0 2024-09-16T08:57:21.492 INFO:tasks.workunit.client.1.smithi097.stdout:8/936: link d8/dd/d15/d2c/d4a/d4e/d97/f116 d8/dd/d71/d9c/f12b 0 2024-09-16T08:57:21.494 INFO:tasks.workunit.client.0.smithi031.stdout:7/540: link d2/d8/d69/f6c d2/d2d/d6d/fb4 0 2024-09-16T08:57:21.495 INFO:tasks.workunit.client.1.smithi097.stdout:9/991: symlink d2/d5/d3a/d70/l13d 0 2024-09-16T08:57:21.496 INFO:tasks.workunit.client.1.smithi097.stdout:9/992: dread - d2/d5/d72/dc1/f11d zero size 2024-09-16T08:57:21.496 INFO:tasks.workunit.client.1.smithi097.stdout:9/993: dread - d2/d3f/fb4 zero size 2024-09-16T08:57:21.499 INFO:tasks.workunit.client.1.smithi097.stdout:8/937: getdents d8/dd/d15/d2c/d4a 0 2024-09-16T08:57:21.499 INFO:tasks.workunit.client.1.smithi097.stdout:8/938: fdatasync d8/dd/d15/d2c/d4a/d4d/d3d/fcd 0 2024-09-16T08:57:21.502 INFO:tasks.workunit.client.0.smithi031.stdout:6/605: mknod d0/dd/d35/d6c/dac/dba/cbe 0 2024-09-16T08:57:21.504 INFO:tasks.workunit.client.1.smithi097.stdout:8/939: mkdir d8/dd/d15/d16/d12c 0 2024-09-16T08:57:21.504 INFO:tasks.workunit.client.1.smithi097.stdout:8/940: chown d8/dd/d42/c46 151567336 1 2024-09-16T08:57:21.505 INFO:tasks.workunit.client.0.smithi031.stdout:3/580: creat d3/d14/d37/d1a/d1d/d3b/d3f/d49/faa x:0 0 0 2024-09-16T08:57:21.507 INFO:tasks.workunit.client.0.smithi031.stdout:2/458: dwrite d6/da/fe [0,4194304] 0 2024-09-16T08:57:21.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/482: getdents d2/dc/d36 0 2024-09-16T08:57:21.509 INFO:tasks.workunit.client.0.smithi031.stdout:5/483: readlink d2/dc/d25/d21/d2e/l35 0 2024-09-16T08:57:21.509 INFO:tasks.workunit.client.0.smithi031.stdout:9/498: dwrite d3/d5/da/d34/d3c/d5e/d9c/f9f [4194304,4194304] 0 2024-09-16T08:57:21.510 INFO:tasks.workunit.client.0.smithi031.stdout:6/606: rmdir d0/dd/d74/dad 39 2024-09-16T08:57:21.511 INFO:tasks.workunit.client.1.smithi097.stdout:9/994: write d2/d5/d8d/f9f [3754916,72971] 0 2024-09-16T08:57:21.512 INFO:tasks.workunit.client.1.smithi097.stdout:3/932: rmdir d7/de/d3c/dcc 0 2024-09-16T08:57:21.512 INFO:tasks.workunit.client.1.smithi097.stdout:3/933: creat d7/de/d3c/f13c x:0 0 0 2024-09-16T08:57:21.515 INFO:tasks.workunit.client.0.smithi031.stdout:3/581: dread d3/fb [0,4194304] 0 2024-09-16T08:57:21.516 INFO:tasks.workunit.client.1.smithi097.stdout:9/995: mkdir d2/d5/d3a/d70/d13e 0 2024-09-16T08:57:21.519 INFO:tasks.workunit.client.0.smithi031.stdout:2/459: mkdir d6/da/d4b/da0 0 2024-09-16T08:57:21.521 INFO:tasks.workunit.client.1.smithi097.stdout:3/934: link d7/de/la9 d7/de/d8d/d90/d96/d9f/l13d 0 2024-09-16T08:57:21.523 INFO:tasks.workunit.client.1.smithi097.stdout:8/941: dwrite d8/dd/f2f [0,4194304] 0 2024-09-16T08:57:21.525 INFO:tasks.workunit.client.0.smithi031.stdout:7/541: dwrite d2/f62 [0,4194304] 0 2024-09-16T08:57:21.525 INFO:tasks.workunit.client.1.smithi097.stdout:9/996: unlink d2/d5/l20 0 2024-09-16T08:57:21.525 INFO:tasks.workunit.client.1.smithi097.stdout:9/997: creat d2/d5/d72/dc1/f13f x:0 0 0 2024-09-16T08:57:21.526 INFO:tasks.workunit.client.1.smithi097.stdout:4/890: sync 2024-09-16T08:57:21.526 INFO:tasks.workunit.client.1.smithi097.stdout:7/868: sync 2024-09-16T08:57:21.526 INFO:tasks.workunit.client.1.smithi097.stdout:9/998: write d2/d5/db0/dc8/d74/d8c/d94/d96/dac/d79/fc5 [841988,105484] 0 2024-09-16T08:57:21.526 INFO:tasks.workunit.client.1.smithi097.stdout:9/999: fsync d2/d5/f71 0 2024-09-16T08:57:21.526 INFO:tasks.workunit.client.1.smithi097.stdout:7/869: stat d4/d9/d51/l97 0 2024-09-16T08:57:21.526 INFO:tasks.workunit.client.1.smithi097.stdout:7/870: fsync d4/d7/d19/f100 0 2024-09-16T08:57:21.527 INFO:tasks.workunit.client.0.smithi031.stdout:5/484: symlink d2/dc/d36/d76/l9b 0 2024-09-16T08:57:21.528 INFO:tasks.workunit.client.0.smithi031.stdout:7/542: read d2/d8/f57 [626264,23756] 0 2024-09-16T08:57:21.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/499: link d3/d5/f6 d3/d82/fac 0 2024-09-16T08:57:21.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/500: truncate d3/d85/f98 1030229 0 2024-09-16T08:57:21.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/501: getdents d3/d5/d73/d81 0 2024-09-16T08:57:21.529 INFO:tasks.workunit.client.0.smithi031.stdout:9/502: write d3/d5/da/f6b [5163539,115182] 0 2024-09-16T08:57:21.532 INFO:tasks.workunit.client.0.smithi031.stdout:6/607: write d0/dd/d74/d19/fa1 [1760576,71863] 0 2024-09-16T08:57:21.532 INFO:tasks.workunit.client.0.smithi031.stdout:2/460: mkdir d6/d20/da1 0 2024-09-16T08:57:21.537 INFO:tasks.workunit.client.0.smithi031.stdout:9/503: dread d3/f5b [0,4194304] 0 2024-09-16T08:57:21.538 INFO:tasks.workunit.client.1.smithi097.stdout:8/942: dread d8/dd/f26 [0,4194304] 0 2024-09-16T08:57:21.538 INFO:tasks.workunit.client.1.smithi097.stdout:4/891: rename d0/d7/df/l4a to d0/d35/da4/dc5/l12f 0 2024-09-16T08:57:21.538 INFO:tasks.workunit.client.1.smithi097.stdout:4/892: creat d0/d23/d5f/f130 x:0 0 0 2024-09-16T08:57:21.540 INFO:tasks.workunit.client.1.smithi097.stdout:7/871: creat d4/d9/d51/f126 x:0 0 0 2024-09-16T08:57:21.541 INFO:tasks.workunit.client.0.smithi031.stdout:5/485: rename d2/dc/c8d to d2/dc/d36/c9c 0 2024-09-16T08:57:21.543 INFO:tasks.workunit.client.0.smithi031.stdout:7/543: mknod d2/d8/d11/d66/d9a/d9c/db3/cb5 0 2024-09-16T08:57:21.543 INFO:tasks.workunit.client.0.smithi031.stdout:7/544: chown d2/d8/d11/d5d/d6e/l7b 8930 1 2024-09-16T08:57:21.546 INFO:tasks.workunit.client.1.smithi097.stdout:8/943: creat d8/dd/d15/d2c/d4a/d4d/d3d/f12d x:0 0 0 2024-09-16T08:57:21.547 INFO:tasks.workunit.client.0.smithi031.stdout:3/582: dwrite d3/d14/d37/d33/f6d [0,4194304] 0 2024-09-16T08:57:21.551 INFO:tasks.workunit.client.1.smithi097.stdout:4/893: mkdir d0/d35/da4/dd1/d131 0 2024-09-16T08:57:21.551 INFO:tasks.workunit.client.1.smithi097.stdout:4/894: write d0/d35/da4/f11b [801646,40767] 0 2024-09-16T08:57:21.553 INFO:tasks.workunit.client.0.smithi031.stdout:9/504: mkdir d3/d5/da/db/d21/d3b/dad 0 2024-09-16T08:57:21.555 INFO:tasks.workunit.client.1.smithi097.stdout:8/944: creat d8/dd/d15/d16/d7c/f12e x:0 0 0 2024-09-16T08:57:21.555 INFO:tasks.workunit.client.1.smithi097.stdout:8/945: getdents d8/dd/d15/d16/d12c 0 2024-09-16T08:57:21.557 INFO:tasks.workunit.client.0.smithi031.stdout:7/545: mkdir d2/d8/d11/d5d/d7e/db6 0 2024-09-16T08:57:21.560 INFO:tasks.workunit.client.0.smithi031.stdout:2/461: getdents d6/da/d37 0 2024-09-16T08:57:21.560 INFO:tasks.workunit.client.0.smithi031.stdout:2/462: chown d6/f36 29757 1 2024-09-16T08:57:21.560 INFO:tasks.workunit.client.0.smithi031.stdout:2/463: write d6/da/d5e/f70 [453166,98807] 0 2024-09-16T08:57:21.561 INFO:tasks.workunit.client.0.smithi031.stdout:9/505: creat d3/d5/da/d34/d3c/d5e/d9c/d95/fae x:0 0 0 2024-09-16T08:57:21.562 INFO:tasks.workunit.client.0.smithi031.stdout:5/486: mknod d2/dc/d25/d21/c9d 0 2024-09-16T08:57:21.563 INFO:tasks.workunit.client.0.smithi031.stdout:7/546: symlink d2/d8/d11/d81/lb7 0 2024-09-16T08:57:21.565 INFO:tasks.workunit.client.1.smithi097.stdout:8/946: read d8/dd/d15/d2c/d4a/d4d/f5c [2034403,98052] 0 2024-09-16T08:57:21.565 INFO:tasks.workunit.client.1.smithi097.stdout:8/947: truncate d8/dd/d15/d2c/d4a/d4d/f5c 2724784 0 2024-09-16T08:57:21.565 INFO:tasks.workunit.client.0.smithi031.stdout:9/506: rename d3/d5/da/db/l38 to d3/d5/da/d34/d3c/laf 0 2024-09-16T08:57:21.565 INFO:tasks.workunit.client.0.smithi031.stdout:6/608: dwrite d0/f91 [0,4194304] 0 2024-09-16T08:57:21.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/547: symlink d2/d8/d11/d5d/lb8 0 2024-09-16T08:57:21.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/548: truncate d2/f1a 5272873 0 2024-09-16T08:57:21.569 INFO:tasks.workunit.client.1.smithi097.stdout:8/948: unlink d8/dd/d42/l5f 0 2024-09-16T08:57:21.570 INFO:tasks.workunit.client.0.smithi031.stdout:6/609: rmdir d0/dd/d2b 39 2024-09-16T08:57:21.571 INFO:tasks.workunit.client.1.smithi097.stdout:7/872: dwrite d4/d7/d19/fb3 [0,4194304] 0 2024-09-16T08:57:21.571 INFO:tasks.workunit.client.1.smithi097.stdout:7/873: write d4/d9/d2b/f2e [909557,77088] 0 2024-09-16T08:57:21.571 INFO:tasks.workunit.client.1.smithi097.stdout:7/874: readlink d4/d7/d7b/d43/ld6 0 2024-09-16T08:57:21.571 INFO:tasks.workunit.client.1.smithi097.stdout:7/875: creat d4/d12/d1f/d22/f127 x:0 0 0 2024-09-16T08:57:21.572 INFO:tasks.workunit.client.1.smithi097.stdout:7/876: fsync d4/d12/d1f/f21 0 2024-09-16T08:57:21.572 INFO:tasks.workunit.client.0.smithi031.stdout:7/549: creat d2/d8/d11/d5d/d7e/fb9 x:0 0 0 2024-09-16T08:57:21.572 INFO:tasks.workunit.client.0.smithi031.stdout:9/507: rmdir d3/d5/da/db/d21/d67 39 2024-09-16T08:57:21.573 INFO:tasks.workunit.client.0.smithi031.stdout:2/464: dread d6/d20/f66 [0,4194304] 0 2024-09-16T08:57:21.573 INFO:tasks.workunit.client.0.smithi031.stdout:2/465: dread - d6/da/d4b/d4c/d73/f9b zero size 2024-09-16T08:57:21.573 INFO:tasks.workunit.client.0.smithi031.stdout:2/466: dread - d6/d2e/d40/d6c/d2c/d53/d60/f6f zero size 2024-09-16T08:57:21.575 INFO:tasks.workunit.client.1.smithi097.stdout:7/877: creat d4/d7/d77/da8/de2/dff/f128 x:0 0 0 2024-09-16T08:57:21.579 INFO:tasks.workunit.client.0.smithi031.stdout:7/550: rmdir d2/d2d/d6d/d91 0 2024-09-16T08:57:21.580 INFO:tasks.workunit.client.0.smithi031.stdout:9/508: truncate d3/d5/da/d34/f77 2637149 0 2024-09-16T08:57:21.580 INFO:tasks.workunit.client.0.smithi031.stdout:9/509: dread - d3/d5/da/db/faa zero size 2024-09-16T08:57:21.581 INFO:tasks.workunit.client.0.smithi031.stdout:2/467: rmdir d6/da/d37 39 2024-09-16T08:57:21.581 INFO:tasks.workunit.client.0.smithi031.stdout:2/468: creat d6/d2e/d40/d58/fa2 x:0 0 0 2024-09-16T08:57:21.583 INFO:tasks.workunit.client.0.smithi031.stdout:7/551: creat d2/d8/d11/d66/d9a/d9c/fba x:0 0 0 2024-09-16T08:57:21.584 INFO:tasks.workunit.client.0.smithi031.stdout:9/510: mknod d3/d5/d6a/cb0 0 2024-09-16T08:57:21.587 INFO:tasks.workunit.client.0.smithi031.stdout:5/487: dwrite d2/dc/d25/f2f [0,4194304] 0 2024-09-16T08:57:21.588 INFO:tasks.workunit.client.0.smithi031.stdout:2/469: link d6/da/fe d6/fa3 0 2024-09-16T08:57:21.588 INFO:tasks.workunit.client.1.smithi097.stdout:4/895: dwrite d0/d7/d52/f78 [0,4194304] 0 2024-09-16T08:57:21.590 INFO:tasks.workunit.client.0.smithi031.stdout:9/511: creat d3/d5/da/d34/d3c/d5e/fb1 x:0 0 0 2024-09-16T08:57:21.591 INFO:tasks.workunit.client.1.smithi097.stdout:7/878: dwrite d4/d7/ff [4194304,4194304] 0 2024-09-16T08:57:21.595 INFO:tasks.workunit.client.0.smithi031.stdout:2/470: rename d6/d2e/d40/f7a to d6/da/d4b/fa4 0 2024-09-16T08:57:21.595 INFO:tasks.workunit.client.0.smithi031.stdout:2/471: dread - d6/d2e/d40/f78 zero size 2024-09-16T08:57:21.595 INFO:tasks.workunit.client.0.smithi031.stdout:2/472: creat d6/d2e/d40/fa5 x:0 0 0 2024-09-16T08:57:21.596 INFO:tasks.workunit.client.0.smithi031.stdout:9/512: link d3/d5/da/d34/d3c/d5e/d9c/f9f d3/d5/d6a/d74/da8/fb2 0 2024-09-16T08:57:21.598 INFO:tasks.workunit.client.0.smithi031.stdout:9/513: mkdir d3/d5/db3 0 2024-09-16T08:57:21.600 INFO:tasks.workunit.client.0.smithi031.stdout:9/514: symlink d3/d5/da/d34/d3c/d5e/lb4 0 2024-09-16T08:57:21.601 INFO:tasks.workunit.client.1.smithi097.stdout:4/896: unlink d0/d7/d52/d26/d8c/db0/fca 0 2024-09-16T08:57:21.602 INFO:tasks.workunit.client.1.smithi097.stdout:7/879: dread d4/d7/d7b/d43/fa7 [0,4194304] 0 2024-09-16T08:57:21.604 INFO:tasks.workunit.client.0.smithi031.stdout:7/552: dwrite d2/d8/d11/d5d/f73 [0,4194304] 0 2024-09-16T08:57:21.606 INFO:tasks.workunit.client.0.smithi031.stdout:5/488: dwrite d2/dc/d25/d68/f6b [0,4194304] 0 2024-09-16T08:57:21.607 INFO:tasks.workunit.client.0.smithi031.stdout:5/489: write d2/dc/d25/d21/d2e/f71 [876680,15978] 0 2024-09-16T08:57:21.607 INFO:tasks.workunit.client.0.smithi031.stdout:5/490: dread - d2/dc/d1d/f3f zero size 2024-09-16T08:57:21.607 INFO:tasks.workunit.client.0.smithi031.stdout:2/473: write d6/d2e/d40/f54 [3595197,109873] 0 2024-09-16T08:57:21.609 INFO:tasks.workunit.client.1.smithi097.stdout:4/897: creat d0/d35/f132 x:0 0 0 2024-09-16T08:57:21.610 INFO:tasks.workunit.client.0.smithi031.stdout:7/553: unlink d2/d8/c51 0 2024-09-16T08:57:21.613 INFO:tasks.workunit.client.1.smithi097.stdout:4/898: mkdir d0/d35/da4/dd1/d133 0 2024-09-16T08:57:21.613 INFO:tasks.workunit.client.1.smithi097.stdout:4/899: write d0/d39/d119/fef [764860,97393] 0 2024-09-16T08:57:21.613 INFO:tasks.workunit.client.1.smithi097.stdout:4/900: write d0/d7/d52/dbf/fc6 [1264401,62188] 0 2024-09-16T08:57:21.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/491: link d2/c6c d2/dc/d8c/c9e 0 2024-09-16T08:57:21.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/492: dread - d2/dc/f75 zero size 2024-09-16T08:57:21.617 INFO:tasks.workunit.client.0.smithi031.stdout:5/493: fsync d2/dc/d25/f53 0 2024-09-16T08:57:21.623 INFO:tasks.workunit.client.0.smithi031.stdout:5/494: link d2/dc/d25/d68/f6e d2/dc/d25/f9f 0 2024-09-16T08:57:21.625 INFO:tasks.workunit.client.0.smithi031.stdout:5/495: mknod d2/dc/d36/d76/ca0 0 2024-09-16T08:57:21.627 INFO:tasks.workunit.client.0.smithi031.stdout:9/515: getdents d3/d5/da/d34/d3c/d5e 0 2024-09-16T08:57:21.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/496: rename d2/dc/d25/f22 to d2/dc/d1d/fa1 0 2024-09-16T08:57:21.629 INFO:tasks.workunit.client.0.smithi031.stdout:2/474: dwrite d6/d20/f89 [0,4194304] 0 2024-09-16T08:57:21.630 INFO:tasks.workunit.client.1.smithi097.stdout:4/901: rmdir d0/d7 39 2024-09-16T08:57:21.631 INFO:tasks.workunit.client.1.smithi097.stdout:4/902: chown d0/d7/ff4 844 1 2024-09-16T08:57:21.631 INFO:tasks.workunit.client.0.smithi031.stdout:9/516: symlink d3/d5/da/d34/d3c/d5e/d62/d65/lb5 0 2024-09-16T08:57:21.634 INFO:tasks.workunit.client.0.smithi031.stdout:7/554: dread d2/d8/d11/f40 [0,4194304] 0 2024-09-16T08:57:21.638 INFO:tasks.workunit.client.1.smithi097.stdout:4/903: truncate d0/d7/d52/f78 7093227 0 2024-09-16T08:57:21.640 INFO:tasks.workunit.client.0.smithi031.stdout:9/517: rename d3/d5/lf to d3/d5/da/db/d21/lb6 0 2024-09-16T08:57:21.641 INFO:tasks.workunit.client.1.smithi097.stdout:4/904: symlink d0/d35/da4/dd1/d131/l134 0 2024-09-16T08:57:21.641 INFO:tasks.workunit.client.1.smithi097.stdout:4/905: truncate d0/d7/df/d15/f1b 1100352 0 2024-09-16T08:57:21.644 INFO:tasks.workunit.client.0.smithi031.stdout:2/475: rmdir d6/d2e/d40 39 2024-09-16T08:57:21.644 INFO:tasks.workunit.client.0.smithi031.stdout:2/476: chown d6/da/d4b/d8f 11 1 2024-09-16T08:57:21.644 INFO:tasks.workunit.client.0.smithi031.stdout:7/555: write d2/d8/d11/d5d/f73 [3642398,545] 0 2024-09-16T08:57:21.644 INFO:tasks.workunit.client.0.smithi031.stdout:7/556: dread - d2/d8/d86/da1/da9/fad zero size 2024-09-16T08:57:21.644 INFO:tasks.workunit.client.0.smithi031.stdout:7/557: write d2/d2d/d6d/fae [491231,105030] 0 2024-09-16T08:57:21.645 INFO:tasks.workunit.client.0.smithi031.stdout:7/558: write d2/d8/d11/d81/f9f [514368,104651] 0 2024-09-16T08:57:21.645 INFO:tasks.workunit.client.1.smithi097.stdout:4/906: mkdir d0/d7/df/d83/df8/d135 0 2024-09-16T08:57:21.646 INFO:tasks.workunit.client.0.smithi031.stdout:9/518: chown d3/d5/da/db/c72 1649306 1 2024-09-16T08:57:21.653 INFO:tasks.workunit.client.0.smithi031.stdout:7/559: symlink d2/d8/d11/d66/d9a/d9c/lbb 0 2024-09-16T08:57:21.654 INFO:tasks.workunit.client.0.smithi031.stdout:7/560: truncate d2/d8/f60 3120171 0 2024-09-16T08:57:21.655 INFO:tasks.workunit.client.1.smithi097.stdout:4/907: dread d0/d23/f9f [0,4194304] 0 2024-09-16T08:57:21.655 INFO:tasks.workunit.client.1.smithi097.stdout:4/908: stat d0/d23/ddc/f115 0 2024-09-16T08:57:21.656 INFO:tasks.workunit.client.0.smithi031.stdout:7/561: mknod d2/d8/d11/d66/d9a/d9c/db3/cbc 0 2024-09-16T08:57:21.657 INFO:tasks.workunit.client.1.smithi097.stdout:4/909: symlink d0/d36/d4c/dea/l136 0 2024-09-16T08:57:21.657 INFO:tasks.workunit.client.1.smithi097.stdout:4/910: chown d0/d35/d6c/lc7 6 1 2024-09-16T08:57:21.658 INFO:tasks.workunit.client.1.smithi097.stdout:4/911: creat d0/d36/da7/f137 x:0 0 0 2024-09-16T08:57:21.658 INFO:tasks.workunit.client.1.smithi097.stdout:4/912: read d0/d36/da7/faa [264812,20127] 0 2024-09-16T08:57:21.658 INFO:tasks.workunit.client.0.smithi031.stdout:7/562: symlink d2/d2d/d68/da2/lbd 0 2024-09-16T08:57:21.658 INFO:tasks.workunit.client.0.smithi031.stdout:7/563: dread - d2/d8/d86/da1/da9/fad zero size 2024-09-16T08:57:21.659 INFO:tasks.workunit.client.0.smithi031.stdout:7/564: write d2/d8/d11/d5d/d7e/fb9 [156484,92866] 0 2024-09-16T08:57:21.659 INFO:tasks.workunit.client.0.smithi031.stdout:7/565: fsync d2/f6a 0 2024-09-16T08:57:21.660 INFO:tasks.workunit.client.0.smithi031.stdout:7/566: unlink d2/d8/d11/l52 0 2024-09-16T08:57:21.665 INFO:tasks.workunit.client.0.smithi031.stdout:9/519: dwrite d3/d5/f6 [0,4194304] 0 2024-09-16T08:57:21.669 INFO:tasks.workunit.client.1.smithi097.stdout:8/949: sync 2024-09-16T08:57:21.670 INFO:tasks.workunit.client.1.smithi097.stdout:8/950: write d8/dd/d71/d9c/f11c [4890968,122025] 0 2024-09-16T08:57:21.670 INFO:tasks.workunit.client.1.smithi097.stdout:3/935: sync 2024-09-16T08:57:21.671 INFO:tasks.workunit.client.0.smithi031.stdout:2/477: dwrite d6/da/f27 [0,4194304] 0 2024-09-16T08:57:21.672 INFO:tasks.workunit.client.1.smithi097.stdout:3/936: mkdir d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/daa/d13e 0 2024-09-16T08:57:21.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/478: mknod d6/da/d37/d42/ca6 0 2024-09-16T08:57:21.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/479: stat d6/c10 0 2024-09-16T08:57:21.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/480: write d6/f13 [4566970,46896] 0 2024-09-16T08:57:21.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/481: write d6/d20/f3f [4030685,68059] 0 2024-09-16T08:57:21.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/482: write d6/d2e/d40/d6c/d2c/f64 [957055,31832] 0 2024-09-16T08:57:21.675 INFO:tasks.workunit.client.0.smithi031.stdout:2/483: stat d6/d2e/d40/d58/f90 0 2024-09-16T08:57:21.676 INFO:tasks.workunit.client.1.smithi097.stdout:3/937: creat d7/d15/dee/f13f x:0 0 0 2024-09-16T08:57:21.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/484: dread d6/da/f84 [0,4194304] 0 2024-09-16T08:57:21.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/485: write d6/da/d37/f87 [222272,60917] 0 2024-09-16T08:57:21.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/486: chown d6/da/d37/f87 142461910 1 2024-09-16T08:57:21.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/487: readlink d6/da/l95 0 2024-09-16T08:57:21.694 INFO:tasks.workunit.client.0.smithi031.stdout:2/488: read d6/da/f5d [4113766,31339] 0 2024-09-16T08:57:21.697 INFO:tasks.workunit.client.0.smithi031.stdout:7/567: dwrite d2/d8/f4a [0,4194304] 0 2024-09-16T08:57:21.705 INFO:tasks.workunit.client.1.smithi097.stdout:4/913: dwrite d0/d7/f126 [0,4194304] 0 2024-09-16T08:57:21.705 INFO:tasks.workunit.client.1.smithi097.stdout:4/914: fdatasync d0/d39/d119/fef 0 2024-09-16T08:57:21.705 INFO:tasks.workunit.client.1.smithi097.stdout:4/915: creat d0/d39/d119/f138 x:0 0 0 2024-09-16T08:57:21.708 INFO:tasks.workunit.client.1.smithi097.stdout:8/951: dwrite d8/dd/d15/d2c/d4a/d4d/d3f/d99/f112 [0,4194304] 0 2024-09-16T08:57:21.708 INFO:tasks.workunit.client.1.smithi097.stdout:8/952: fsync d8/dd/d15/f21 0 2024-09-16T08:57:21.709 INFO:tasks.workunit.client.1.smithi097.stdout:8/953: dread - d8/dd/d15/d16/d7c/da2/ffa zero size 2024-09-16T08:57:21.709 INFO:tasks.workunit.client.1.smithi097.stdout:3/938: dwrite d7/de/d3c/d3f/f6d [4194304,4194304] 0 2024-09-16T08:57:21.709 INFO:tasks.workunit.client.1.smithi097.stdout:3/939: write d7/de/d3c/d3f/d11c/fd7 [176148,13361] 0 2024-09-16T08:57:21.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/520: dwrite d3/d5/da/f2b [0,4194304] 0 2024-09-16T08:57:21.710 INFO:tasks.workunit.client.0.smithi031.stdout:9/521: dread - d3/d5/da/db/faa zero size 2024-09-16T08:57:21.710 INFO:tasks.workunit.client.0.smithi031.stdout:9/522: fsync d3/d5/da/db/faa 0 2024-09-16T08:57:21.711 INFO:tasks.workunit.client.0.smithi031.stdout:9/523: symlink d3/d5/da/db/d21/d3b/lb7 0 2024-09-16T08:57:21.714 INFO:tasks.workunit.client.0.smithi031.stdout:2/489: dwrite d6/d2e/d40/d6c/d2c/d53/d60/f93 [0,4194304] 0 2024-09-16T08:57:21.715 INFO:tasks.workunit.client.1.smithi097.stdout:8/954: rename d8/dd/d15/d2c/d4a/d4e/db2/l125 to d8/dd/d15/d2c/d4a/d4d/d3f/l12f 0 2024-09-16T08:57:21.719 INFO:tasks.workunit.client.1.smithi097.stdout:3/940: link d7/de/d3c/d3f/d60/dcb/c55 d7/de/d8d/d90/d96/d9f/c140 0 2024-09-16T08:57:21.721 INFO:tasks.workunit.client.1.smithi097.stdout:8/955: unlink d8/f31 0 2024-09-16T08:57:21.722 INFO:tasks.workunit.client.0.smithi031.stdout:7/568: dwrite d2/d2d/f67 [0,4194304] 0 2024-09-16T08:57:21.722 INFO:tasks.workunit.client.0.smithi031.stdout:7/569: stat d2/d8/c29 0 2024-09-16T08:57:21.722 INFO:tasks.workunit.client.0.smithi031.stdout:7/570: truncate d2/f56 4680641 0 2024-09-16T08:57:21.723 INFO:tasks.workunit.client.0.smithi031.stdout:2/490: rename d6/da/d4b/d4c/d68/l98 to d6/da/d5e/d8d/la7 0 2024-09-16T08:57:21.723 INFO:tasks.workunit.client.0.smithi031.stdout:2/491: fdatasync d6/d2e/d40/d6c/d2c/f34 0 2024-09-16T08:57:21.724 INFO:tasks.workunit.client.1.smithi097.stdout:7/880: sync 2024-09-16T08:57:21.724 INFO:tasks.workunit.client.1.smithi097.stdout:4/916: dwrite d0/d7/df/f37 [0,4194304] 0 2024-09-16T08:57:21.724 INFO:tasks.workunit.client.1.smithi097.stdout:4/917: readlink d0/d36/da7/ldf 0 2024-09-16T08:57:21.725 INFO:tasks.workunit.client.1.smithi097.stdout:3/941: truncate d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/f91 3007590 0 2024-09-16T08:57:21.725 INFO:tasks.workunit.client.1.smithi097.stdout:3/942: stat d7/de/d3c/d4f/f80 0 2024-09-16T08:57:21.725 INFO:tasks.workunit.client.0.smithi031.stdout:7/571: dread d2/d8/d69/d82/f9b [0,4194304] 0 2024-09-16T08:57:21.725 INFO:tasks.workunit.client.1.smithi097.stdout:8/956: mkdir d8/dd/d15/d2c/d130 0 2024-09-16T08:57:21.727 INFO:tasks.workunit.client.1.smithi097.stdout:7/881: creat d4/d7/d19/ddd/f129 x:0 0 0 2024-09-16T08:57:21.727 INFO:tasks.workunit.client.1.smithi097.stdout:7/882: creat d4/d7/d77/da8/de2/f12a x:0 0 0 2024-09-16T08:57:21.727 INFO:tasks.workunit.client.1.smithi097.stdout:7/883: write d4/d1b/d41/f75 [17226,103771] 0 2024-09-16T08:57:21.728 INFO:tasks.workunit.client.0.smithi031.stdout:9/524: rmdir d3/d5/da/db/d21/d3b 39 2024-09-16T08:57:21.728 INFO:tasks.workunit.client.1.smithi097.stdout:3/943: dread d7/de/d3c/d3f/d60/d6b/fc5 [0,4194304] 0 2024-09-16T08:57:21.729 INFO:tasks.workunit.client.0.smithi031.stdout:2/492: truncate d6/d2e/d40/d6c/d2c/d53/d60/f35 1048003 0 2024-09-16T08:57:21.732 INFO:tasks.workunit.client.0.smithi031.stdout:7/572: truncate d2/d8/d11/d5d/f73 2410332 0 2024-09-16T08:57:21.735 INFO:tasks.workunit.client.0.smithi031.stdout:9/525: creat d3/d5/db3/fb8 x:0 0 0 2024-09-16T08:57:21.737 INFO:tasks.workunit.client.1.smithi097.stdout:8/957: rename d8/dd/d15/d16/d7c/da2/fdc to d8/dd/d15/d2c/d4a/d4d/d3d/d123/f131 0 2024-09-16T08:57:21.737 INFO:tasks.workunit.client.1.smithi097.stdout:8/958: chown d8/dd/d15/d2c/d4a/d4d/d3f/l121 78262701 1 2024-09-16T08:57:21.737 INFO:tasks.workunit.client.0.smithi031.stdout:9/526: dread d3/d48/f49 [0,4194304] 0 2024-09-16T08:57:21.738 INFO:tasks.workunit.client.0.smithi031.stdout:2/493: link d6/f13 d6/da/d4b/d4c/d68/fa8 0 2024-09-16T08:57:21.738 INFO:tasks.workunit.client.0.smithi031.stdout:2/494: write d6/d20/f26 [1122598,111281] 0 2024-09-16T08:57:21.739 INFO:tasks.workunit.client.0.smithi031.stdout:7/573: dread d2/d8/d11/d81/fa5 [0,4194304] 0 2024-09-16T08:57:21.741 INFO:tasks.workunit.client.0.smithi031.stdout:9/527: rename d3/d5/da/db/c72 to d3/d5/da/d34/cb9 0 2024-09-16T08:57:21.743 INFO:tasks.workunit.client.1.smithi097.stdout:7/884: creat d4/d9/d125/f12b x:0 0 0 2024-09-16T08:57:21.744 INFO:tasks.workunit.client.0.smithi031.stdout:2/495: truncate d6/fc 1957933 0 2024-09-16T08:57:21.744 INFO:tasks.workunit.client.0.smithi031.stdout:2/496: write d6/d2e/d40/d58/f90 [353057,71164] 0 2024-09-16T08:57:21.745 INFO:tasks.workunit.client.1.smithi097.stdout:3/944: symlink d7/de/d8d/d90/d92/l141 0 2024-09-16T08:57:21.745 INFO:tasks.workunit.client.1.smithi097.stdout:7/885: dread d4/d7/d7b/d43/daf/ffd [0,4194304] 0 2024-09-16T08:57:21.746 INFO:tasks.workunit.client.1.smithi097.stdout:7/886: readlink d4/l47 0 2024-09-16T08:57:21.746 INFO:tasks.workunit.client.1.smithi097.stdout:7/887: creat d4/d1b/da4/f12c x:0 0 0 2024-09-16T08:57:21.746 INFO:tasks.workunit.client.0.smithi031.stdout:7/574: rename d2/l28 to d2/d8/d11/d5d/d7e/db6/lbe 0 2024-09-16T08:57:21.748 INFO:tasks.workunit.client.1.smithi097.stdout:4/918: truncate d0/d7/f126 2085843 0 2024-09-16T08:57:21.751 INFO:tasks.workunit.client.1.smithi097.stdout:8/959: mkdir d8/dd/d15/d2c/d4a/d4d/d3d/d7b/dcb/d132 0 2024-09-16T08:57:21.751 INFO:tasks.workunit.client.1.smithi097.stdout:8/960: write d8/dd/d15/d2c/d4a/d4d/d3d/d7b/ff4 [757613,4687] 0 2024-09-16T08:57:21.755 INFO:tasks.workunit.client.1.smithi097.stdout:8/961: link d8/dd/d15/d2c/d4a/d4d/fbe d8/dd/d15/d2c/d4a/d4e/d85/f133 0 2024-09-16T08:57:21.755 INFO:tasks.workunit.client.1.smithi097.stdout:7/888: dread d4/d7/d7b/f8f [0,4194304] 0 2024-09-16T08:57:21.756 INFO:tasks.workunit.client.1.smithi097.stdout:8/962: mkdir d8/dd/d15/d2c/d4a/d4d/d3f/d134 0 2024-09-16T08:57:21.757 INFO:tasks.workunit.client.0.smithi031.stdout:7/575: creat d2/d8/d11/d81/fbf x:0 0 0 2024-09-16T08:57:21.757 INFO:tasks.workunit.client.0.smithi031.stdout:7/576: write d2/d8/fd [8747254,121115] 0 2024-09-16T08:57:21.757 INFO:tasks.workunit.client.0.smithi031.stdout:7/577: fsync d2/fab 0 2024-09-16T08:57:21.758 INFO:tasks.workunit.client.1.smithi097.stdout:7/889: symlink d4/d7/d77/l12d 0 2024-09-16T08:57:21.758 INFO:tasks.workunit.client.1.smithi097.stdout:7/890: truncate d4/d12/d1f/d22/f8c 215003 0 2024-09-16T08:57:21.759 INFO:tasks.workunit.client.1.smithi097.stdout:7/891: truncate d4/d12/d1f/fb6 2352875 0 2024-09-16T08:57:21.761 INFO:tasks.workunit.client.0.smithi031.stdout:7/578: creat d2/d8/d11/d5d/fc0 x:0 0 0 2024-09-16T08:57:21.762 INFO:tasks.workunit.client.1.smithi097.stdout:3/945: dwrite d7/d15/dee/d11f/f2f [4194304,4194304] 0 2024-09-16T08:57:21.762 INFO:tasks.workunit.client.0.smithi031.stdout:7/579: read d2/d8/f63 [35268,22058] 0 2024-09-16T08:57:21.762 INFO:tasks.workunit.client.0.smithi031.stdout:7/580: dread - d2/d8/d86/f88 zero size 2024-09-16T08:57:21.763 INFO:tasks.workunit.client.1.smithi097.stdout:4/919: write d0/d39/d71/faf [1307751,99955] 0 2024-09-16T08:57:21.763 INFO:tasks.workunit.client.1.smithi097.stdout:7/892: rename d4/d9/d51/c55 to d4/d12/d1f/d22/c12e 0 2024-09-16T08:57:21.763 INFO:tasks.workunit.client.1.smithi097.stdout:4/920: write d0/d7/d52/f94 [311912,114000] 0 2024-09-16T08:57:21.766 INFO:tasks.workunit.client.1.smithi097.stdout:4/921: dread d0/f3e [4194304,4194304] 0 2024-09-16T08:57:21.766 INFO:tasks.workunit.client.1.smithi097.stdout:4/922: write d0/d7/df/d83/ff5 [1119673,63500] 0 2024-09-16T08:57:21.766 INFO:tasks.workunit.client.1.smithi097.stdout:4/923: write d0/d23/d5f/f93 [759631,21373] 0 2024-09-16T08:57:21.766 INFO:tasks.workunit.client.1.smithi097.stdout:4/924: chown d0/f30 31 1 2024-09-16T08:57:21.767 INFO:tasks.workunit.client.0.smithi031.stdout:7/581: rmdir d2/d8/d11/d66/d9a 39 2024-09-16T08:57:21.768 INFO:tasks.workunit.client.1.smithi097.stdout:3/946: mknod d7/de/d8d/d90/d96/d9f/dbb/c142 0 2024-09-16T08:57:21.768 INFO:tasks.workunit.client.1.smithi097.stdout:3/947: dread - d7/d15/dee/f13f zero size 2024-09-16T08:57:21.770 INFO:tasks.workunit.client.1.smithi097.stdout:4/925: mknod d0/d39/d119/c139 0 2024-09-16T08:57:21.772 INFO:tasks.workunit.client.1.smithi097.stdout:4/926: symlink d0/d35/da4/dd1/l13a 0 2024-09-16T08:57:21.773 INFO:tasks.workunit.client.1.smithi097.stdout:4/927: truncate d0/d7/df/d83/fb5 1583900 0 2024-09-16T08:57:21.773 INFO:tasks.workunit.client.1.smithi097.stdout:4/928: truncate d0/d36/d4c/fd7 198497 0 2024-09-16T08:57:21.779 INFO:tasks.workunit.client.1.smithi097.stdout:8/963: dwrite d8/dd/d15/d16/fa1 [4194304,4194304] 0 2024-09-16T08:57:21.779 INFO:tasks.workunit.client.1.smithi097.stdout:8/964: dread - d8/dd/d15/d2c/d4a/df3/d109/d10e/f113 zero size 2024-09-16T08:57:21.781 INFO:tasks.workunit.client.1.smithi097.stdout:8/965: symlink d8/dd/d15/d2c/l135 0 2024-09-16T08:57:21.781 INFO:tasks.workunit.client.0.smithi031.stdout:4/496: sync 2024-09-16T08:57:21.785 INFO:tasks.workunit.client.1.smithi097.stdout:8/966: dread d8/dd/d15/d16/d7c/feb [0,4194304] 0 2024-09-16T08:57:21.787 INFO:tasks.workunit.client.1.smithi097.stdout:7/893: dwrite d4/d7/d19/f100 [0,4194304] 0 2024-09-16T08:57:21.787 INFO:tasks.workunit.client.1.smithi097.stdout:7/894: creat d4/d7/d77/da8/de2/dff/f12f x:0 0 0 2024-09-16T08:57:21.787 INFO:tasks.workunit.client.1.smithi097.stdout:7/895: dread - d4/d7/d7b/fef zero size 2024-09-16T08:57:21.788 INFO:tasks.workunit.client.1.smithi097.stdout:8/967: mknod d8/dd/d15/d2c/d4a/d4e/d85/c136 0 2024-09-16T08:57:21.788 INFO:tasks.workunit.client.1.smithi097.stdout:8/968: getdents d8/dd/d15/d2c/d4a/d4e/d97 0 2024-09-16T08:57:21.789 INFO:tasks.workunit.client.1.smithi097.stdout:3/948: dwrite d7/d15/dee/d11f/d30/d69/ffe [0,4194304] 0 2024-09-16T08:57:21.793 INFO:tasks.workunit.client.1.smithi097.stdout:7/896: dread d4/d9/d1c/f28 [0,4194304] 0 2024-09-16T08:57:21.795 INFO:tasks.workunit.client.0.smithi031.stdout:2/497: dread d6/fc [0,4194304] 0 2024-09-16T08:57:21.803 INFO:tasks.workunit.client.1.smithi097.stdout:7/897: link d4/d9/d2b/c4e d4/d9/d51/d8b/c130 0 2024-09-16T08:57:21.804 INFO:tasks.workunit.client.0.smithi031.stdout:7/582: dwrite d2/d8/d11/f24 [0,4194304] 0 2024-09-16T08:57:21.804 INFO:tasks.workunit.client.1.smithi097.stdout:8/969: rmdir d8/dd/d15/d2c/d4a/d4d/d3d/d123 39 2024-09-16T08:57:21.805 INFO:tasks.workunit.client.1.smithi097.stdout:7/898: write d4/d9/dc/f6b [532573,14501] 0 2024-09-16T08:57:21.805 INFO:tasks.workunit.client.0.smithi031.stdout:7/583: mknod d2/d8/d11/d81/cc1 0 2024-09-16T08:57:21.806 INFO:tasks.workunit.client.0.smithi031.stdout:4/497: dwrite d0/d2/d16/f21 [0,4194304] 0 2024-09-16T08:57:21.807 INFO:tasks.workunit.client.1.smithi097.stdout:8/970: link d8/dd/f82 d8/dd/d15/d2c/d4a/d4e/f137 0 2024-09-16T08:57:21.809 INFO:tasks.workunit.client.0.smithi031.stdout:7/584: rmdir d2/d8/d86/da1 39 2024-09-16T08:57:21.813 INFO:tasks.workunit.client.1.smithi097.stdout:7/899: rename d4/d7/d77/c111 to d4/d1b/c131 0 2024-09-16T08:57:21.813 INFO:tasks.workunit.client.1.smithi097.stdout:7/900: readlink d4/l47 0 2024-09-16T08:57:21.813 INFO:tasks.workunit.client.1.smithi097.stdout:8/971: dread d8/dd/d15/f9b [0,4194304] 0 2024-09-16T08:57:21.814 INFO:tasks.workunit.client.0.smithi031.stdout:7/585: rename d2/f48 to d2/d8/d11/d5d/fc2 0 2024-09-16T08:57:21.817 INFO:tasks.workunit.client.1.smithi097.stdout:7/901: dread d4/d9/dc/f90 [0,4194304] 0 2024-09-16T08:57:21.819 INFO:tasks.workunit.client.1.smithi097.stdout:3/949: dwrite d7/de/d3c/d3f/d60/dcb/f76 [0,4194304] 0 2024-09-16T08:57:21.820 INFO:tasks.workunit.client.1.smithi097.stdout:3/950: read - d7/de/d3c/f13c zero size 2024-09-16T08:57:21.821 INFO:tasks.workunit.client.1.smithi097.stdout:3/951: truncate d7/de/d3c/d3f/d60/dcb/f10a 2338557 0 2024-09-16T08:57:21.822 INFO:tasks.workunit.client.1.smithi097.stdout:3/952: getdents d7/de/d3c/d3f/d11c/d101 0 2024-09-16T08:57:21.822 INFO:tasks.workunit.client.1.smithi097.stdout:3/953: write d7/d15/dee/d11f/d63/dd8/f104 [310332,112333] 0 2024-09-16T08:57:21.822 INFO:tasks.workunit.client.1.smithi097.stdout:3/954: creat d7/de/d9c/daf/f143 x:0 0 0 2024-09-16T08:57:21.828 INFO:tasks.workunit.client.1.smithi097.stdout:8/972: write d8/dd/d15/d2c/d4a/d4d/ddd/f128 [3988467,22195] 0 2024-09-16T08:57:21.829 INFO:tasks.workunit.client.1.smithi097.stdout:8/973: creat d8/dd/d15/d2c/d4a/d4d/f138 x:0 0 0 2024-09-16T08:57:21.829 INFO:tasks.workunit.client.1.smithi097.stdout:8/974: chown d8/dd/d15/d2c/d4a/d4d/d3f/d99/l9a 1250797820 1 2024-09-16T08:57:21.830 INFO:tasks.workunit.client.1.smithi097.stdout:8/975: mknod d8/dd/d15/d2c/d4a/d4d/d3f/d134/c139 0 2024-09-16T08:57:21.830 INFO:tasks.workunit.client.1.smithi097.stdout:8/976: write d8/dd/d15/d2c/fe1 [544586,34719] 0 2024-09-16T08:57:21.833 INFO:tasks.workunit.client.1.smithi097.stdout:8/977: dread d8/dd/f61 [4194304,4194304] 0 2024-09-16T08:57:21.833 INFO:tasks.workunit.client.1.smithi097.stdout:8/978: chown d8/dd/d71/d9c/dac/c111 0 1 2024-09-16T08:57:21.836 INFO:tasks.workunit.client.0.smithi031.stdout:2/498: dwrite d6/da/d4b/d4c/d68/fa8 [0,4194304] 0 2024-09-16T08:57:21.836 INFO:tasks.workunit.client.0.smithi031.stdout:2/499: write d6/da/f6a [710527,12407] 0 2024-09-16T08:57:21.836 INFO:tasks.workunit.client.0.smithi031.stdout:2/500: chown d6/d2e/d40/d58/l7c 980992 1 2024-09-16T08:57:21.837 INFO:tasks.workunit.client.0.smithi031.stdout:4/498: dwrite d0/d2/d16/f6e [0,4194304] 0 2024-09-16T08:57:21.837 INFO:tasks.workunit.client.0.smithi031.stdout:4/499: truncate d0/d5b/f89 228131 0 2024-09-16T08:57:21.840 INFO:tasks.workunit.client.0.smithi031.stdout:7/586: dwrite d2/d8/d11/f5c [4194304,4194304] 0 2024-09-16T08:57:21.842 INFO:tasks.workunit.client.0.smithi031.stdout:2/501: getdents d6/da/d4b 0 2024-09-16T08:57:21.844 INFO:tasks.workunit.client.0.smithi031.stdout:7/587: symlink d2/d8/d11/d5d/lc3 0 2024-09-16T08:57:21.846 INFO:tasks.workunit.client.0.smithi031.stdout:2/502: link d6/da/d4b/d4c/d68/c82 d6/da/d4b/d4c/d73/ca9 0 2024-09-16T08:57:21.846 INFO:tasks.workunit.client.0.smithi031.stdout:2/503: write d6/f16 [5123069,35163] 0 2024-09-16T08:57:21.846 INFO:tasks.workunit.client.0.smithi031.stdout:2/504: readlink d6/da/d37/l6b 0 2024-09-16T08:57:21.846 INFO:tasks.workunit.client.1.smithi097.stdout:8/979: write d8/dd/d15/d2c/d4a/d4d/f49 [3019520,82966] 0 2024-09-16T08:57:21.847 INFO:tasks.workunit.client.1.smithi097.stdout:8/980: creat d8/dd/d15/d16/d7c/f13a x:0 0 0 2024-09-16T08:57:21.847 INFO:tasks.workunit.client.0.smithi031.stdout:7/588: mknod d2/d2d/d6d/cc4 0 2024-09-16T08:57:21.853 INFO:tasks.workunit.client.0.smithi031.stdout:2/505: dread d6/fa3 [0,4194304] 0 2024-09-16T08:57:21.853 INFO:tasks.workunit.client.1.smithi097.stdout:8/981: dread d8/dd/d15/d2c/f41 [0,4194304] 0 2024-09-16T08:57:21.866 INFO:tasks.workunit.client.1.smithi097.stdout:3/955: dwrite d7/de/d3c/d3f/d11c/fdc [0,4194304] 0 2024-09-16T08:57:21.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/506: dread d6/da/f3d [0,4194304] 0 2024-09-16T08:57:21.868 INFO:tasks.workunit.client.0.smithi031.stdout:2/507: mknod d6/da/d4b/d4c/d68/caa 0 2024-09-16T08:57:21.868 INFO:tasks.workunit.client.1.smithi097.stdout:3/956: symlink d7/de/d8d/d90/d96/d9f/dbb/d113/l144 0 2024-09-16T08:57:21.868 INFO:tasks.workunit.client.1.smithi097.stdout:3/957: fsync d7/d15/dee/d11f/d63/f12d 0 2024-09-16T08:57:21.869 INFO:tasks.workunit.client.0.smithi031.stdout:2/508: mkdir d6/da/d37/d8b/dab 0 2024-09-16T08:57:21.869 INFO:tasks.workunit.client.0.smithi031.stdout:2/509: fdatasync d6/da/d37/d42/f71 0 2024-09-16T08:57:21.869 INFO:tasks.workunit.client.0.smithi031.stdout:2/510: write d6/da/d4b/d4c/d73/f9b [1016112,53455] 0 2024-09-16T08:57:21.869 INFO:tasks.workunit.client.0.smithi031.stdout:2/511: creat d6/d20/fac x:0 0 0 2024-09-16T08:57:21.870 INFO:tasks.workunit.client.1.smithi097.stdout:3/958: rename d7/de/d8d/d90/d92 to d7/de/d9c/d145 0 2024-09-16T08:57:21.870 INFO:tasks.workunit.client.1.smithi097.stdout:3/959: stat d7/de/d8d/d90/d96/d9f/db5/d110 0 2024-09-16T08:57:21.872 INFO:tasks.workunit.client.1.smithi097.stdout:3/960: creat d7/de/d8d/d90/d96/f146 x:0 0 0 2024-09-16T08:57:21.872 INFO:tasks.workunit.client.0.smithi031.stdout:4/500: dwrite d0/d2/f3d [0,4194304] 0 2024-09-16T08:57:21.874 INFO:tasks.workunit.client.1.smithi097.stdout:8/982: dwrite d8/fc [4194304,4194304] 0 2024-09-16T08:57:21.882 INFO:tasks.workunit.client.1.smithi097.stdout:8/983: creat d8/dd/d15/d2c/d4a/df3/f13b x:0 0 0 2024-09-16T08:57:21.882 INFO:tasks.workunit.client.0.smithi031.stdout:4/501: write d0/d2/f41 [3747823,9787] 0 2024-09-16T08:57:21.885 INFO:tasks.workunit.client.1.smithi097.stdout:8/984: write d8/dd/d15/d2c/f4b [1552803,63904] 0 2024-09-16T08:57:21.886 INFO:tasks.workunit.client.1.smithi097.stdout:8/985: write d8/dd/f61 [4417635,34366] 0 2024-09-16T08:57:21.886 INFO:tasks.workunit.client.0.smithi031.stdout:4/502: rename d0/d3c/d6b/f8a to d0/d2/d16/d1e/f9e 0 2024-09-16T08:57:21.886 INFO:tasks.workunit.client.0.smithi031.stdout:4/503: write d0/d2/d3a/f6f [372369,101292] 0 2024-09-16T08:57:21.887 INFO:tasks.workunit.client.0.smithi031.stdout:4/504: mknod d0/d8e/c9f 0 2024-09-16T08:57:21.887 INFO:tasks.workunit.client.0.smithi031.stdout:4/505: dread - d0/d2/d16/d59/d84/d86/f8f zero size 2024-09-16T08:57:21.888 INFO:tasks.workunit.client.1.smithi097.stdout:8/986: mknod d8/dd/d15/d2c/d4a/d4d/d3d/c13c 0 2024-09-16T08:57:21.889 INFO:tasks.workunit.client.1.smithi097.stdout:8/987: mknod d8/dd/d42/c13d 0 2024-09-16T08:57:21.889 INFO:tasks.workunit.client.0.smithi031.stdout:2/512: dwrite d6/d2e/d40/d6c/f2b [0,4194304] 0 2024-09-16T08:57:21.890 INFO:tasks.workunit.client.1.smithi097.stdout:8/988: stat d8/dd/d15/d2c/l45 0 2024-09-16T08:57:21.892 INFO:tasks.workunit.client.0.smithi031.stdout:2/513: mkdir d6/da/d37/d8b/dad 0 2024-09-16T08:57:21.893 INFO:tasks.workunit.client.1.smithi097.stdout:8/989: creat d8/dd/d15/d2c/d4a/df3/d109/f13e x:0 0 0 2024-09-16T08:57:21.894 INFO:tasks.workunit.client.0.smithi031.stdout:2/514: mkdir d6/d2e/d40/d58/dae 0 2024-09-16T08:57:21.895 INFO:tasks.workunit.client.1.smithi097.stdout:8/990: creat d8/dd/d15/d2c/f13f x:0 0 0 2024-09-16T08:57:21.895 INFO:tasks.workunit.client.1.smithi097.stdout:8/991: fsync d8/f13 0 2024-09-16T08:57:21.896 INFO:tasks.workunit.client.0.smithi031.stdout:2/515: rename d6/da/d5e/d8d/l9d to d6/da/d37/d8b/dad/laf 0 2024-09-16T08:57:21.896 INFO:tasks.workunit.client.0.smithi031.stdout:2/516: write d6/d20/f3f [145194,90463] 0 2024-09-16T08:57:21.897 INFO:tasks.workunit.client.0.smithi031.stdout:2/517: creat d6/d20/da1/fb0 x:0 0 0 2024-09-16T08:57:21.898 INFO:tasks.workunit.client.0.smithi031.stdout:2/518: mkdir d6/d2e/d40/d58/dae/db1 0 2024-09-16T08:57:21.899 INFO:tasks.workunit.client.0.smithi031.stdout:2/519: unlink d6/fc 0 2024-09-16T08:57:21.899 INFO:tasks.workunit.client.0.smithi031.stdout:2/520: mknod d6/da/d4b/cb2 0 2024-09-16T08:57:21.900 INFO:tasks.workunit.client.1.smithi097.stdout:8/992: read d8/dd/f7e [24349,47928] 0 2024-09-16T08:57:21.909 INFO:tasks.workunit.client.1.smithi097.stdout:3/961: dwrite d7/de/d6c/fe7 [0,4194304] 0 2024-09-16T08:57:21.912 INFO:tasks.workunit.client.0.smithi031.stdout:4/506: dwrite d0/d2/f13 [4194304,4194304] 0 2024-09-16T08:57:21.912 INFO:tasks.workunit.client.1.smithi097.stdout:3/962: symlink d7/d15/dee/d11f/d30/l147 0 2024-09-16T08:57:21.912 INFO:tasks.workunit.client.1.smithi097.stdout:3/963: write d7/de/d8d/d90/f9d [993880,102792] 0 2024-09-16T08:57:21.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/507: mknod d0/d2/d16/d59/d84/d86/d9b/ca0 0 2024-09-16T08:57:21.914 INFO:tasks.workunit.client.1.smithi097.stdout:3/964: mkdir d7/de/d8d/d90/d96/d9f/db5/d110/d148 0 2024-09-16T08:57:21.914 INFO:tasks.workunit.client.1.smithi097.stdout:3/965: dread - d7/de/d9c/daf/fc2 zero size 2024-09-16T08:57:21.924 INFO:tasks.workunit.client.0.smithi031.stdout:2/521: dwrite d6/da/f17 [4194304,4194304] 0 2024-09-16T08:57:21.924 INFO:tasks.workunit.client.0.smithi031.stdout:2/522: chown d6/da/d4b/d4c/d68/caa 258022 1 2024-09-16T08:57:21.927 INFO:tasks.workunit.client.0.smithi031.stdout:2/523: rename d6/d2e/d40/d6c/d2c/f34 to d6/da/d4b/d4c/fb3 0 2024-09-16T08:57:21.927 INFO:tasks.workunit.client.0.smithi031.stdout:2/524: truncate d6/da/d4b/d4c/f5f 425729 0 2024-09-16T08:57:21.931 INFO:tasks.workunit.client.1.smithi097.stdout:8/993: dwrite d8/dd/d15/d2c/f13f [0,4194304] 0 2024-09-16T08:57:21.938 INFO:tasks.workunit.client.1.smithi097.stdout:4/929: sync 2024-09-16T08:57:21.938 INFO:tasks.workunit.client.1.smithi097.stdout:7/902: sync 2024-09-16T08:57:21.940 INFO:tasks.workunit.client.0.smithi031.stdout:4/508: dwrite d0/d2/d3a/d55/f64 [4194304,4194304] 0 2024-09-16T08:57:21.940 INFO:tasks.workunit.client.0.smithi031.stdout:2/525: dread d6/d20/f89 [0,4194304] 0 2024-09-16T08:57:21.940 INFO:tasks.workunit.client.0.smithi031.stdout:4/509: write d0/d2/f1a [753148,46604] 0 2024-09-16T08:57:21.941 INFO:tasks.workunit.client.1.smithi097.stdout:7/903: mknod d4/d7/d7b/d43/daf/c132 0 2024-09-16T08:57:21.941 INFO:tasks.workunit.client.1.smithi097.stdout:7/904: chown d4/d7/d7b/d43/daf/c132 11410528 1 2024-09-16T08:57:21.942 INFO:tasks.workunit.client.0.smithi031.stdout:2/526: getdents d6/d2e/d40/d6c/d2c/d53 0 2024-09-16T08:57:21.944 INFO:tasks.workunit.client.0.smithi031.stdout:2/527: creat d6/da/d5e/d72/fb4 x:0 0 0 2024-09-16T08:57:21.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/528: creat d6/da/d4b/d4c/d73/fb5 x:0 0 0 2024-09-16T08:57:21.945 INFO:tasks.workunit.client.0.smithi031.stdout:2/529: readlink d6/da/d4b/l67 0 2024-09-16T08:57:21.948 INFO:tasks.workunit.client.0.smithi031.stdout:4/510: dread d0/d2/f41 [0,4194304] 0 2024-09-16T08:57:21.948 INFO:tasks.workunit.client.0.smithi031.stdout:4/511: write d0/fa [100991,103543] 0 2024-09-16T08:57:21.950 INFO:tasks.workunit.client.0.smithi031.stdout:2/530: symlink d6/da/d37/d8b/dad/lb6 0 2024-09-16T08:57:21.952 INFO:tasks.workunit.client.0.smithi031.stdout:4/512: creat d0/d2/d16/d1e/fa1 x:0 0 0 2024-09-16T08:57:21.952 INFO:tasks.workunit.client.0.smithi031.stdout:4/513: chown d0/d8e/f95 157463 1 2024-09-16T08:57:21.953 INFO:tasks.workunit.client.0.smithi031.stdout:4/514: creat d0/d2/d16/d59/d84/fa2 x:0 0 0 2024-09-16T08:57:21.953 INFO:tasks.workunit.client.0.smithi031.stdout:4/515: stat d0/d2/f1a 0 2024-09-16T08:57:21.953 INFO:tasks.workunit.client.0.smithi031.stdout:4/516: dread - d0/d2/d16/f99 zero size 2024-09-16T08:57:21.953 INFO:tasks.workunit.client.0.smithi031.stdout:4/517: creat d0/d2/d16/d59/d84/fa3 x:0 0 0 2024-09-16T08:57:21.954 INFO:tasks.workunit.client.0.smithi031.stdout:2/531: write d6/da/fe [3046249,2238] 0 2024-09-16T08:57:21.979 INFO:tasks.workunit.client.1.smithi097.stdout:7/905: dwrite d4/d9/dc/f6b [0,4194304] 0 2024-09-16T08:57:21.979 INFO:tasks.workunit.client.1.smithi097.stdout:8/994: dwrite d8/dd/d15/d16/d7c/feb [0,4194304] 0 2024-09-16T08:57:21.979 INFO:tasks.workunit.client.1.smithi097.stdout:4/930: dwrite d0/d7/df/f8f [0,4194304] 0 2024-09-16T08:57:21.979 INFO:tasks.workunit.client.1.smithi097.stdout:4/931: truncate d0/d23/f10b 1346488 0 2024-09-16T08:57:21.979 INFO:tasks.workunit.client.1.smithi097.stdout:4/932: fsync d0/d7/df/d83/f98 0 2024-09-16T08:57:21.979 INFO:tasks.workunit.client.1.smithi097.stdout:4/933: write d0/d23/f10b [2090347,39915] 0 2024-09-16T08:57:21.980 INFO:tasks.workunit.client.0.smithi031.stdout:2/532: dwrite d6/da/d37/d8b/f8c [0,4194304] 0 2024-09-16T08:57:21.980 INFO:tasks.workunit.client.0.smithi031.stdout:2/533: truncate d6/d20/f32 940450 0 2024-09-16T08:57:21.984 INFO:tasks.workunit.client.0.smithi031.stdout:4/518: dread d0/d2/f1a [0,4194304] 0 2024-09-16T08:57:21.985 INFO:tasks.workunit.client.1.smithi097.stdout:8/995: dread d8/dd/d15/d2c/d4a/d4e/f137 [0,4194304] 0 2024-09-16T08:57:21.992 INFO:tasks.workunit.client.1.smithi097.stdout:8/996: mknod d8/dd/d15/d16/d7c/c140 0 2024-09-16T08:57:21.992 INFO:tasks.workunit.client.1.smithi097.stdout:8/997: fdatasync d8/dd/d15/d2c/d4a/d4d/d3d/db9/fd4 0 2024-09-16T08:57:21.992 INFO:tasks.workunit.client.1.smithi097.stdout:8/998: fdatasync d8/dd/d15/f28 0 2024-09-16T08:57:21.992 INFO:tasks.workunit.client.1.smithi097.stdout:8/999: creat d8/dd/d42/f141 x:0 0 0 2024-09-16T08:57:21.993 INFO:tasks.workunit.client.1.smithi097.stdout:7/906: dread d4/f9d [0,4194304] 0 2024-09-16T08:57:21.993 INFO:tasks.workunit.client.1.smithi097.stdout:7/907: write d4/d7/d30/ddf/f11a [414888,26183] 0 2024-09-16T08:57:21.994 INFO:tasks.workunit.client.0.smithi031.stdout:4/519: write d0/f4c [2639256,39661] 0 2024-09-16T08:57:21.994 INFO:tasks.workunit.client.1.smithi097.stdout:7/908: link d4/d7/dab/cb9 d4/d7/d77/c133 0 2024-09-16T08:57:21.996 INFO:tasks.workunit.client.1.smithi097.stdout:7/909: rename d4/d7/d7b/d43 to d4/d7/d19/ddd/d134 0 2024-09-16T08:57:21.997 INFO:tasks.workunit.client.1.smithi097.stdout:7/910: mkdir d4/dfa/d135 0 2024-09-16T08:57:21.998 INFO:tasks.workunit.client.1.smithi097.stdout:7/911: unlink d4/d9/dc/l66 0 2024-09-16T08:57:22.003 INFO:tasks.workunit.client.0.smithi031.stdout:4/520: dread d0/d2/d16/f6e [0,4194304] 0 2024-09-16T08:57:22.007 INFO:tasks.workunit.client.1.smithi097.stdout:7/912: dread d4/d7/d30/ddf/f9b [0,4194304] 0 2024-09-16T08:57:22.008 INFO:tasks.workunit.client.1.smithi097.stdout:7/913: write d4/d7/d77/da8/de2/dff/f128 [369856,15023] 0 2024-09-16T08:57:22.008 INFO:tasks.workunit.client.1.smithi097.stdout:7/914: readlink d4/d9/d1c/la5 0 2024-09-16T08:57:22.008 INFO:tasks.workunit.client.1.smithi097.stdout:7/915: dread - d4/d9/d1c/fbf zero size 2024-09-16T08:57:22.009 INFO:tasks.workunit.client.1.smithi097.stdout:7/916: rename d4/d7/d77/da8/de2/dff/f128 to d4/d7/d19/ddd/d134/f136 0 2024-09-16T08:57:22.009 INFO:tasks.workunit.client.1.smithi097.stdout:7/917: creat d4/d9/d51/f137 x:0 0 0 2024-09-16T08:57:22.009 INFO:tasks.workunit.client.1.smithi097.stdout:7/918: chown d4/d7/d77/da8/de2 177348 1 2024-09-16T08:57:22.013 INFO:tasks.workunit.client.1.smithi097.stdout:4/934: dwrite d0/d23/d5f/fc9 [0,4194304] 0 2024-09-16T08:57:22.013 INFO:tasks.workunit.client.1.smithi097.stdout:4/935: readlink d0/d7/df/d15/l12a 0 2024-09-16T08:57:22.013 INFO:tasks.workunit.client.1.smithi097.stdout:4/936: chown d0/d35/f129 0 1 2024-09-16T08:57:22.014 INFO:tasks.workunit.client.1.smithi097.stdout:4/937: chown d0/d7/df/d83/ff5 361385168 1 2024-09-16T08:57:22.018 INFO:tasks.workunit.client.0.smithi031.stdout:4/521: dread d0/d2/f3d [0,4194304] 0 2024-09-16T08:57:22.020 INFO:tasks.workunit.client.0.smithi031.stdout:4/522: creat d0/d2/d16/d59/d84/d86/d9b/fa4 x:0 0 0 2024-09-16T08:57:22.020 INFO:tasks.workunit.client.0.smithi031.stdout:4/523: readlink d0/d8e/d75/l88 0 2024-09-16T08:57:22.020 INFO:tasks.workunit.client.0.smithi031.stdout:4/524: getdents d0/d71 0 2024-09-16T08:57:22.021 INFO:tasks.workunit.client.0.smithi031.stdout:4/525: mkdir d0/d3c/d6b/d9c/da5 0 2024-09-16T08:57:22.021 INFO:tasks.workunit.client.0.smithi031.stdout:4/526: creat d0/d2/d16/d59/d84/d86/fa6 x:0 0 0 2024-09-16T08:57:22.023 INFO:tasks.workunit.client.0.smithi031.stdout:4/527: mknod d0/d2/d16/ca7 0 2024-09-16T08:57:22.023 INFO:tasks.workunit.client.0.smithi031.stdout:4/528: creat d0/d2/d16/d59/d84/d86/fa8 x:0 0 0 2024-09-16T08:57:22.025 INFO:tasks.workunit.client.0.smithi031.stdout:4/529: mknod d0/d3c/d6b/d9c/ca9 0 2024-09-16T08:57:22.025 INFO:tasks.workunit.client.0.smithi031.stdout:4/530: creat d0/d2/faa x:0 0 0 2024-09-16T08:57:22.025 INFO:tasks.workunit.client.0.smithi031.stdout:4/531: creat d0/d2/d3a/d55/fab x:0 0 0 2024-09-16T08:57:22.025 INFO:tasks.workunit.client.0.smithi031.stdout:4/532: readlink d0/d3c/d6b/l7a 0 2024-09-16T08:57:22.025 INFO:tasks.workunit.client.0.smithi031.stdout:4/533: readlink d0/d8e/l82 0 2024-09-16T08:57:22.026 INFO:tasks.workunit.client.0.smithi031.stdout:4/534: creat d0/d2/d3a/fac x:0 0 0 2024-09-16T08:57:22.031 INFO:tasks.workunit.client.0.smithi031.stdout:4/535: rename d0/d2/d3a/d55/l60 to d0/d3c/d6b/d9c/lad 0 2024-09-16T08:57:22.031 INFO:tasks.workunit.client.0.smithi031.stdout:4/536: stat d0/d2/d16/f22 0 2024-09-16T08:57:22.033 INFO:tasks.workunit.client.0.smithi031.stdout:4/537: rename d0/d2/d16/d59/d84/f98 to d0/d3c/d6b/d9c/fae 0 2024-09-16T08:57:22.033 INFO:tasks.workunit.client.0.smithi031.stdout:4/538: stat d0/d3c/d6b/f5e 0 2024-09-16T08:57:22.033 INFO:tasks.workunit.client.0.smithi031.stdout:4/539: truncate d0/d2/d16/f35 4258906 0 2024-09-16T08:57:22.036 INFO:tasks.workunit.client.1.smithi097.stdout:4/938: dwrite d0/d23/f7b [0,4194304] 0 2024-09-16T08:57:22.039 INFO:tasks.workunit.client.1.smithi097.stdout:7/919: dwrite d4/d9/dc/f90 [0,4194304] 0 2024-09-16T08:57:22.039 INFO:tasks.workunit.client.1.smithi097.stdout:4/939: unlink d0/d35/f107 0 2024-09-16T08:57:22.045 INFO:tasks.workunit.client.1.smithi097.stdout:4/940: mknod d0/d35/da4/c13b 0 2024-09-16T08:57:22.045 INFO:tasks.workunit.client.1.smithi097.stdout:7/920: rmdir d4/d1b/d41 39 2024-09-16T08:57:22.045 INFO:tasks.workunit.client.1.smithi097.stdout:4/941: write d0/d35/f12e [1296321,18775] 0 2024-09-16T08:57:22.045 INFO:tasks.workunit.client.1.smithi097.stdout:7/921: mknod d4/d9/d1c/c138 0 2024-09-16T08:57:22.047 INFO:tasks.workunit.client.1.smithi097.stdout:7/922: dread d4/d9/dc/d8e/f92 [0,4194304] 0 2024-09-16T08:57:22.047 INFO:tasks.workunit.client.1.smithi097.stdout:7/923: write d4/d1b/d41/fd9 [1363828,87940] 0 2024-09-16T08:57:22.047 INFO:tasks.workunit.client.1.smithi097.stdout:7/924: chown d4/d12/d1f/f21 3809364 1 2024-09-16T08:57:22.048 INFO:tasks.workunit.client.1.smithi097.stdout:7/925: mknod d4/d9/d2b/dad/c139 0 2024-09-16T08:57:22.048 INFO:tasks.workunit.client.1.smithi097.stdout:7/926: readlink d4/d7/d77/l12d 0 2024-09-16T08:57:22.048 INFO:tasks.workunit.client.0.smithi031.stdout:4/540: dwrite d0/f18 [0,4194304] 0 2024-09-16T08:57:22.049 INFO:tasks.workunit.client.0.smithi031.stdout:4/541: chown d0/d2/d16/d59/d8d 1060 1 2024-09-16T08:57:22.049 INFO:tasks.workunit.client.1.smithi097.stdout:7/927: link d4/d7/d19/ddd/d134/daf/f108 d4/d7/d77/da8/d124/f13a 0 2024-09-16T08:57:22.049 INFO:tasks.workunit.client.1.smithi097.stdout:7/928: truncate d4/d7/d79/f113 602994 0 2024-09-16T08:57:22.050 INFO:tasks.workunit.client.1.smithi097.stdout:7/929: stat d4/d9/d1c/ff0 0 2024-09-16T08:57:22.050 INFO:tasks.workunit.client.1.smithi097.stdout:7/930: chown d4/d7/d77/c8d 2028 1 2024-09-16T08:57:22.050 INFO:tasks.workunit.client.1.smithi097.stdout:7/931: stat d4/d7/d7b/cd1 0 2024-09-16T08:57:22.058 INFO:tasks.workunit.client.0.smithi031.stdout:4/542: mkdir d0/d2/d16/d2b/daf 0 2024-09-16T08:57:22.058 INFO:tasks.workunit.client.0.smithi031.stdout:4/543: creat d0/d2/d16/d59/d84/d86/d9b/fb0 x:0 0 0 2024-09-16T08:57:22.062 INFO:tasks.workunit.client.0.smithi031.stdout:4/544: mknod d0/d2/d16/d2b/cb1 0 2024-09-16T08:57:22.064 INFO:tasks.workunit.client.0.smithi031.stdout:4/545: mknod d0/d2/d16/d1e/cb2 0 2024-09-16T08:57:22.065 INFO:tasks.workunit.client.0.smithi031.stdout:8/403: sync 2024-09-16T08:57:22.067 INFO:tasks.workunit.client.0.smithi031.stdout:8/404: symlink d4/d5b/d7b/l8b 0 2024-09-16T08:57:22.070 INFO:tasks.workunit.client.0.smithi031.stdout:8/405: rename d4/d7/d21/d35/d40 to d4/d7/d21/d37/d46/d8c 0 2024-09-16T08:57:22.071 INFO:tasks.workunit.client.1.smithi097.stdout:7/932: dwrite d4/d7/d19/ddd/d134/f44 [0,4194304] 0 2024-09-16T08:57:22.073 INFO:tasks.workunit.client.1.smithi097.stdout:7/933: link d4/d9/dc/fcb d4/dfa/f13b 0 2024-09-16T08:57:22.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/546: dread d0/d2/d16/d1e/f3b [0,4194304] 0 2024-09-16T08:57:22.082 INFO:tasks.workunit.client.1.smithi097.stdout:3/966: sync 2024-09-16T08:57:22.082 INFO:tasks.workunit.client.1.smithi097.stdout:3/967: write d7/de/d3c/d3f/d11c/fd7 [866275,13518] 0 2024-09-16T08:57:22.083 INFO:tasks.workunit.client.0.smithi031.stdout:8/406: write d4/d7/f51 [2837182,63402] 0 2024-09-16T08:57:22.085 INFO:tasks.workunit.client.0.smithi031.stdout:4/547: write d0/d2/f3d [284012,74119] 0 2024-09-16T08:57:22.086 INFO:tasks.workunit.client.1.smithi097.stdout:7/934: dread d4/d1b/f91 [0,4194304] 0 2024-09-16T08:57:22.086 INFO:tasks.workunit.client.1.smithi097.stdout:7/935: write d4/d7/d7b/f8f [1685747,95276] 0 2024-09-16T08:57:22.087 INFO:tasks.workunit.client.1.smithi097.stdout:7/936: dread - d4/d1b/da4/f12c zero size 2024-09-16T08:57:22.092 INFO:tasks.workunit.client.1.smithi097.stdout:4/942: write d0/d7/d52/f8d [2307070,99201] 0 2024-09-16T08:57:22.094 INFO:tasks.workunit.client.0.smithi031.stdout:4/548: dread d0/d2/d16/f6e [0,4194304] 0 2024-09-16T08:57:22.096 INFO:tasks.workunit.client.0.smithi031.stdout:4/549: unlink d0/d2/cf 0 2024-09-16T08:57:22.097 INFO:tasks.workunit.client.1.smithi097.stdout:7/937: dread d4/d7/f32 [0,4194304] 0 2024-09-16T08:57:22.097 INFO:tasks.workunit.client.1.smithi097.stdout:7/938: fdatasync d4/d7/d79/f113 0 2024-09-16T08:57:22.097 INFO:tasks.workunit.client.1.smithi097.stdout:7/939: write d4/d9/d1c/f38 [9270033,122571] 0 2024-09-16T08:57:22.097 INFO:tasks.workunit.client.1.smithi097.stdout:7/940: creat d4/d9/d2b/f13c x:0 0 0 2024-09-16T08:57:22.098 INFO:tasks.workunit.client.0.smithi031.stdout:4/550: link d0/d2/l29 d0/d3c/lb3 0 2024-09-16T08:57:22.098 INFO:tasks.workunit.client.0.smithi031.stdout:4/551: creat d0/d2/d3a/fb4 x:0 0 0 2024-09-16T08:57:22.101 INFO:tasks.workunit.client.1.smithi097.stdout:3/968: dwrite d7/de/d3c/d3f/d60/dcb/d35/d3d/f112 [0,4194304] 0 2024-09-16T08:57:22.101 INFO:tasks.workunit.client.1.smithi097.stdout:3/969: fdatasync d7/de/d3c/d3f/d60/dcb/fe0 0 2024-09-16T08:57:22.101 INFO:tasks.workunit.client.0.smithi031.stdout:4/552: rmdir d0/d5b 39 2024-09-16T08:57:22.102 INFO:tasks.workunit.client.1.smithi097.stdout:4/943: write d0/d7/d52/f88 [4180694,29931] 0 2024-09-16T08:57:22.102 INFO:tasks.workunit.client.1.smithi097.stdout:4/944: chown d0/d7/df/d15/f114 7490986 1 2024-09-16T08:57:22.111 INFO:tasks.workunit.client.0.smithi031.stdout:4/553: write d0/d2/f13 [3969695,8798] 0 2024-09-16T08:57:22.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/407: dwrite d4/d7/d36/f7e [0,4194304] 0 2024-09-16T08:57:22.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/408: mkdir d4/d5b/d8d 0 2024-09-16T08:57:22.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/409: readlink d4/d7/d36/l6d 0 2024-09-16T08:57:22.135 INFO:tasks.workunit.client.1.smithi097.stdout:7/941: dwrite d4/d7/fbd [0,4194304] 0 2024-09-16T08:57:22.135 INFO:tasks.workunit.client.1.smithi097.stdout:7/942: write d4/d9/d2b/f13c [136475,2905] 0 2024-09-16T08:57:22.135 INFO:tasks.workunit.client.1.smithi097.stdout:7/943: creat d4/d7/d19/ddd/f13d x:0 0 0 2024-09-16T08:57:22.140 INFO:tasks.workunit.client.1.smithi097.stdout:3/970: dwrite d7/de/d3c/d3f/d11c/f84 [0,4194304] 0 2024-09-16T08:57:22.140 INFO:tasks.workunit.client.0.smithi031.stdout:4/554: dwrite d0/d2/d16/d59/f63 [0,4194304] 0 2024-09-16T08:57:22.141 INFO:tasks.workunit.client.0.smithi031.stdout:4/555: write d0/d2/d16/f99 [959167,104174] 0 2024-09-16T08:57:22.141 INFO:tasks.workunit.client.0.smithi031.stdout:8/410: dread d4/f43 [0,4194304] 0 2024-09-16T08:57:22.141 INFO:tasks.workunit.client.1.smithi097.stdout:3/971: unlink d7/de/d3c/f13c 0 2024-09-16T08:57:22.145 INFO:tasks.workunit.client.0.smithi031.stdout:8/411: creat d4/d5b/d7b/f8e x:0 0 0 2024-09-16T08:57:22.146 INFO:tasks.workunit.client.0.smithi031.stdout:8/412: fsync d4/f29 0 2024-09-16T08:57:22.150 INFO:tasks.workunit.client.1.smithi097.stdout:3/972: mknod d7/d15/dec/c149 0 2024-09-16T08:57:22.153 INFO:tasks.workunit.client.0.smithi031.stdout:4/556: rmdir d0/d2 39 2024-09-16T08:57:22.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/413: creat d4/d77/f8f x:0 0 0 2024-09-16T08:57:22.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/414: readlink d4/d7/d21/d35/d50/l5f 0 2024-09-16T08:57:22.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/415: fdatasync d4/d7/d21/d37/d46/f75 0 2024-09-16T08:57:22.155 INFO:tasks.workunit.client.0.smithi031.stdout:8/416: chown d4/d7/d21/d35/d79/f8a 222565969 1 2024-09-16T08:57:22.156 INFO:tasks.workunit.client.0.smithi031.stdout:4/557: rename d0/d3c/d6b/d9c/da5 to d0/db5 0 2024-09-16T08:57:22.164 INFO:tasks.workunit.client.0.smithi031.stdout:2/534: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:0/475: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:5/497: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:6/610: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:1/478: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:7/589: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:9/528: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:1/479: chown d2/d15/d5d/l27 0 1 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:1/480: readlink d2/d15/d32/d87/l8f 0 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:3/583: sync 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:3/584: readlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/l51 0 2024-09-16T08:57:22.165 INFO:tasks.workunit.client.0.smithi031.stdout:3/585: truncate d3/d14/d6f/f72 851419 0 2024-09-16T08:57:22.166 INFO:tasks.workunit.client.0.smithi031.stdout:6/611: rename d0/dd/d74/f96 to d0/dd/d74/d67/fbf 0 2024-09-16T08:57:22.168 INFO:tasks.workunit.client.0.smithi031.stdout:2/535: mkdir d6/d2e/d40/d6c/d2c/d53/d60/db7 0 2024-09-16T08:57:22.168 INFO:tasks.workunit.client.0.smithi031.stdout:2/536: stat d6/d2e/d40/d6c/d2c/d53/d60/f35 0 2024-09-16T08:57:22.169 INFO:tasks.workunit.client.0.smithi031.stdout:2/537: creat d6/da/d5e/d72/fb8 x:0 0 0 2024-09-16T08:57:22.170 INFO:tasks.workunit.client.0.smithi031.stdout:1/481: creat d2/d15/d5d/fac x:0 0 0 2024-09-16T08:57:22.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/586: link d3/d14/d6f/d7d/ca9 d3/d14/cab 0 2024-09-16T08:57:22.172 INFO:tasks.workunit.client.0.smithi031.stdout:9/529: getdents d3/d85 0 2024-09-16T08:57:22.174 INFO:tasks.workunit.client.0.smithi031.stdout:1/482: symlink d2/d15/d9c/lad 0 2024-09-16T08:57:22.175 INFO:tasks.workunit.client.0.smithi031.stdout:5/498: read d2/dc/d25/d21/f4e [1043569,18166] 0 2024-09-16T08:57:22.176 INFO:tasks.workunit.client.0.smithi031.stdout:3/587: mkdir d3/d14/d37/d1a/d1d/dac 0 2024-09-16T08:57:22.176 INFO:tasks.workunit.client.0.smithi031.stdout:3/588: dread - d3/d14/d37/d1a/d1d/d3b/d3f/d49/faa zero size 2024-09-16T08:57:22.176 INFO:tasks.workunit.client.0.smithi031.stdout:3/589: chown d3/d14/d6f/d7d 1811989 1 2024-09-16T08:57:22.177 INFO:tasks.workunit.client.0.smithi031.stdout:9/530: mknod d3/d82/cba 0 2024-09-16T08:57:22.177 INFO:tasks.workunit.client.0.smithi031.stdout:9/531: readlink d3/l4a 0 2024-09-16T08:57:22.177 INFO:tasks.workunit.client.0.smithi031.stdout:9/532: chown d3/d5/da/f6b 460312279 1 2024-09-16T08:57:22.178 INFO:tasks.workunit.client.0.smithi031.stdout:3/590: rename d3/d14/d37/d1a/d24/l32 to d3/d14/d6f/d7d/lad 0 2024-09-16T08:57:22.179 INFO:tasks.workunit.client.0.smithi031.stdout:9/533: mkdir d3/d5/da/db/d21/d3b/dbb 0 2024-09-16T08:57:22.180 INFO:tasks.workunit.client.0.smithi031.stdout:6/612: dread d0/d10/f58 [0,4194304] 0 2024-09-16T08:57:22.180 INFO:tasks.workunit.client.0.smithi031.stdout:0/476: dread d6/d12/d13/f24 [0,4194304] 0 2024-09-16T08:57:22.181 INFO:tasks.workunit.client.0.smithi031.stdout:3/591: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/lae 0 2024-09-16T08:57:22.182 INFO:tasks.workunit.client.0.smithi031.stdout:9/534: mkdir d3/d5/da/db/d21/d67/dbc 0 2024-09-16T08:57:22.182 INFO:tasks.workunit.client.0.smithi031.stdout:9/535: dread - d3/d5/da/d34/d3c/d5e/d9c/f90 zero size 2024-09-16T08:57:22.182 INFO:tasks.workunit.client.0.smithi031.stdout:6/613: getdents d0/dd/d74/dad 0 2024-09-16T08:57:22.184 INFO:tasks.workunit.client.0.smithi031.stdout:9/536: creat d3/d5/da/d34/d3c/d5e/d62/d65/fbd x:0 0 0 2024-09-16T08:57:22.185 INFO:tasks.workunit.client.0.smithi031.stdout:6/614: creat d0/dd/d74/d19/d9a/fc0 x:0 0 0 2024-09-16T08:57:22.185 INFO:tasks.workunit.client.0.smithi031.stdout:6/615: readlink d0/l97 0 2024-09-16T08:57:22.185 INFO:tasks.workunit.client.0.smithi031.stdout:6/616: stat d0/dd/d74/d5e/d7c/d3f/caf 0 2024-09-16T08:57:22.187 INFO:tasks.workunit.client.0.smithi031.stdout:3/592: dread d3/d14/d37/d1a/d1d/d1e/d67/f6b [0,4194304] 0 2024-09-16T08:57:22.187 INFO:tasks.workunit.client.0.smithi031.stdout:3/593: dread - d3/d14/d37/d1a/d1d/d3b/f8b zero size 2024-09-16T08:57:22.187 INFO:tasks.workunit.client.0.smithi031.stdout:3/594: fdatasync d3/d14/d37/d33/f7a 0 2024-09-16T08:57:22.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/477: dread d6/f82 [0,4194304] 0 2024-09-16T08:57:22.205 INFO:tasks.workunit.client.0.smithi031.stdout:9/537: dread d3/d5/da/f6b [0,4194304] 0 2024-09-16T08:57:22.205 INFO:tasks.workunit.client.0.smithi031.stdout:9/538: truncate d3/d5/da/db/d21/d3b/fa5 895321 0 2024-09-16T08:57:22.205 INFO:tasks.workunit.client.0.smithi031.stdout:9/539: creat d3/d5/da/d34/fbe x:0 0 0 2024-09-16T08:57:22.205 INFO:tasks.workunit.client.0.smithi031.stdout:9/540: chown d3/d5/d6a/f76 807353385 1 2024-09-16T08:57:22.208 INFO:tasks.workunit.client.0.smithi031.stdout:4/558: dwrite d0/d2/d16/d2b/f67 [0,4194304] 0 2024-09-16T08:57:22.211 INFO:tasks.workunit.client.0.smithi031.stdout:0/478: write d6/f1e [4177454,114863] 0 2024-09-16T08:57:22.211 INFO:tasks.workunit.client.0.smithi031.stdout:7/590: dwrite d2/d8/f8c [0,4194304] 0 2024-09-16T08:57:22.212 INFO:tasks.workunit.client.0.smithi031.stdout:7/591: creat d2/d8/d11/d5d/d7e/fc5 x:0 0 0 2024-09-16T08:57:22.221 INFO:tasks.workunit.client.0.smithi031.stdout:8/417: dwrite d4/f8 [0,4194304] 0 2024-09-16T08:57:22.221 INFO:tasks.workunit.client.0.smithi031.stdout:8/418: chown d4/d7/d36/d38/c6c 15855 1 2024-09-16T08:57:22.221 INFO:tasks.workunit.client.0.smithi031.stdout:8/419: truncate d4/d7/d21/d37/d46/d8c/f58 1311962 0 2024-09-16T08:57:22.222 INFO:tasks.workunit.client.0.smithi031.stdout:8/420: write d4/d7/d21/d37/d46/d8c/f58 [969389,47954] 0 2024-09-16T08:57:22.222 INFO:tasks.workunit.client.0.smithi031.stdout:8/421: stat d4/d7/d21/f23 0 2024-09-16T08:57:22.222 INFO:tasks.workunit.client.0.smithi031.stdout:8/422: write d4/d7/d36/d38/f87 [924156,32371] 0 2024-09-16T08:57:22.223 INFO:tasks.workunit.client.0.smithi031.stdout:8/423: symlink d4/d5b/l90 0 2024-09-16T08:57:22.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/592: dread d2/d8/f60 [0,4194304] 0 2024-09-16T08:57:22.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/593: dread - d2/fab zero size 2024-09-16T08:57:22.227 INFO:tasks.workunit.client.0.smithi031.stdout:7/594: creat d2/d2d/d6d/fc6 x:0 0 0 2024-09-16T08:57:22.227 INFO:tasks.workunit.client.0.smithi031.stdout:7/595: chown d2/f62 11 1 2024-09-16T08:57:22.231 INFO:tasks.workunit.client.0.smithi031.stdout:2/538: dwrite d6/d2e/d40/fa5 [0,4194304] 0 2024-09-16T08:57:22.231 INFO:tasks.workunit.client.0.smithi031.stdout:2/539: chown d6/d2e/d40/d6c 25 1 2024-09-16T08:57:22.233 INFO:tasks.workunit.client.0.smithi031.stdout:2/540: symlink d6/da/d37/d8b/dab/lb9 0 2024-09-16T08:57:22.235 INFO:tasks.workunit.client.0.smithi031.stdout:2/541: symlink d6/lba 0 2024-09-16T08:57:22.235 INFO:tasks.workunit.client.0.smithi031.stdout:1/483: dwrite d2/d15/d5d/f23 [0,4194304] 0 2024-09-16T08:57:22.235 INFO:tasks.workunit.client.0.smithi031.stdout:1/484: truncate d2/d4/d1b/f78 487630 0 2024-09-16T08:57:22.235 INFO:tasks.workunit.client.0.smithi031.stdout:1/485: write d2/d4/d1b/fa6 [418279,96333] 0 2024-09-16T08:57:22.237 INFO:tasks.workunit.client.0.smithi031.stdout:2/542: dread d6/d2e/d40/d6c/d2c/f64 [0,4194304] 0 2024-09-16T08:57:22.239 INFO:tasks.workunit.client.0.smithi031.stdout:2/543: creat d6/da/d37/d42/fbb x:0 0 0 2024-09-16T08:57:22.239 INFO:tasks.workunit.client.0.smithi031.stdout:2/544: chown d6/d2e/d40/d6c/d2c/d53/d60/l77 1464011 1 2024-09-16T08:57:22.239 INFO:tasks.workunit.client.0.smithi031.stdout:2/545: readlink d6/d2e/l7e 0 2024-09-16T08:57:22.246 INFO:tasks.workunit.client.0.smithi031.stdout:3/595: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/f62 [4194304,4194304] 0 2024-09-16T08:57:22.246 INFO:tasks.workunit.client.0.smithi031.stdout:3/596: chown d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/lae 14 1 2024-09-16T08:57:22.249 INFO:tasks.workunit.client.0.smithi031.stdout:1/486: write f0 [3771389,75540] 0 2024-09-16T08:57:22.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/487: getdents d2/d15/d32/d87 0 2024-09-16T08:57:22.251 INFO:tasks.workunit.client.0.smithi031.stdout:1/488: stat d2/d4/l16 0 2024-09-16T08:57:22.254 INFO:tasks.workunit.client.0.smithi031.stdout:2/546: getdents d6/da/d5e/d72 0 2024-09-16T08:57:22.258 INFO:tasks.workunit.client.0.smithi031.stdout:6/617: dwrite d0/f91 [0,4194304] 0 2024-09-16T08:57:22.258 INFO:tasks.workunit.client.0.smithi031.stdout:0/479: dwrite d6/d12/d47/f83 [0,4194304] 0 2024-09-16T08:57:22.259 INFO:tasks.workunit.client.0.smithi031.stdout:0/480: chown d6/d12/d47/c8b 13620 1 2024-09-16T08:57:22.259 INFO:tasks.workunit.client.0.smithi031.stdout:8/424: dwrite d4/d5b/d7b/f8e [0,4194304] 0 2024-09-16T08:57:22.259 INFO:tasks.workunit.client.0.smithi031.stdout:8/425: chown d4/d7/c1b 1471 1 2024-09-16T08:57:22.259 INFO:tasks.workunit.client.0.smithi031.stdout:4/559: dwrite d0/d2/d16/f99 [0,4194304] 0 2024-09-16T08:57:22.259 INFO:tasks.workunit.client.0.smithi031.stdout:4/560: fsync d0/fa 0 2024-09-16T08:57:22.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: pgmap v12: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 308 MiB/s rd, 400 MiB/s wr, 639 op/s 2024-09-16T08:57:22.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:22.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:22.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:21 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:22.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/618: unlink d0/dd/f2f 0 2024-09-16T08:57:22.262 INFO:tasks.workunit.client.0.smithi031.stdout:6/619: read d0/dd/d74/f1e [4121171,109005] 0 2024-09-16T08:57:22.264 INFO:tasks.workunit.client.0.smithi031.stdout:0/481: mkdir d6/d12/d97 0 2024-09-16T08:57:22.265 INFO:tasks.workunit.client.0.smithi031.stdout:1/489: read d2/d15/d5d/f45 [1151840,114826] 0 2024-09-16T08:57:22.266 INFO:tasks.workunit.client.0.smithi031.stdout:7/596: getdents d2/d2d/d6d 0 2024-09-16T08:57:22.267 INFO:tasks.workunit.client.1.smithi097.stdout:4/945: sync 2024-09-16T08:57:22.268 INFO:tasks.workunit.client.0.smithi031.stdout:8/426: symlink d4/d7/d66/l91 0 2024-09-16T08:57:22.268 INFO:tasks.workunit.client.1.smithi097.stdout:4/946: write d0/d35/f12e [28666,64808] 0 2024-09-16T08:57:22.270 INFO:tasks.workunit.client.0.smithi031.stdout:2/547: dwrite d6/da/d5e/d72/fb8 [0,4194304] 0 2024-09-16T08:57:22.270 INFO:tasks.workunit.client.0.smithi031.stdout:2/548: chown d6/d2e/l7e 29 1 2024-09-16T08:57:22.270 INFO:tasks.workunit.client.0.smithi031.stdout:2/549: truncate d6/d20/f89 4328081 0 2024-09-16T08:57:22.270 INFO:tasks.workunit.client.0.smithi031.stdout:2/550: fsync d6/d2e/d40/f54 0 2024-09-16T08:57:22.271 INFO:tasks.workunit.client.0.smithi031.stdout:4/561: creat d0/d2/d16/d1e/fb6 x:0 0 0 2024-09-16T08:57:22.271 INFO:tasks.workunit.client.0.smithi031.stdout:4/562: write d0/d3c/f7f [1132335,28473] 0 2024-09-16T08:57:22.272 INFO:tasks.workunit.client.1.smithi097.stdout:4/947: creat d0/d23/ddc/de5/df9/f13c x:0 0 0 2024-09-16T08:57:22.274 INFO:tasks.workunit.client.0.smithi031.stdout:6/620: unlink d0/l16 0 2024-09-16T08:57:22.276 INFO:tasks.workunit.client.0.smithi031.stdout:0/482: mknod d6/d12/d47/d4d/c98 0 2024-09-16T08:57:22.276 INFO:tasks.workunit.client.0.smithi031.stdout:0/483: write d6/d12/d13/d3c/f45 [3364793,1619] 0 2024-09-16T08:57:22.276 INFO:tasks.workunit.client.1.smithi097.stdout:4/948: stat d0/d23/d5f/cd3 0 2024-09-16T08:57:22.276 INFO:tasks.workunit.client.1.smithi097.stdout:4/949: creat d0/d35/d6c/f13d x:0 0 0 2024-09-16T08:57:22.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/490: mkdir d2/dae 0 2024-09-16T08:57:22.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/491: stat d2/d15/d32/f5f 0 2024-09-16T08:57:22.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/492: write d2/d4/d1b/f78 [517529,51799] 0 2024-09-16T08:57:22.278 INFO:tasks.workunit.client.0.smithi031.stdout:7/597: symlink d2/d8/d11/d5d/d7e/lc7 0 2024-09-16T08:57:22.279 INFO:tasks.workunit.client.0.smithi031.stdout:8/427: read d4/d7/d36/f7e [2323745,1760] 0 2024-09-16T08:57:22.281 INFO:tasks.workunit.client.1.smithi097.stdout:4/950: unlink d0/d7/d52/f78 0 2024-09-16T08:57:22.281 INFO:tasks.workunit.client.1.smithi097.stdout:4/951: fdatasync d0/d36/f124 0 2024-09-16T08:57:22.281 INFO:tasks.workunit.client.1.smithi097.stdout:4/952: truncate d0/d23/ddc/de5/df9/f13c 718140 0 2024-09-16T08:57:22.281 INFO:tasks.workunit.client.1.smithi097.stdout:4/953: chown d0/d7/df/d83/ff5 847888869 1 2024-09-16T08:57:22.282 INFO:tasks.workunit.client.0.smithi031.stdout:2/551: creat d6/da/d37/d42/fbc x:0 0 0 2024-09-16T08:57:22.282 INFO:tasks.workunit.client.0.smithi031.stdout:2/552: stat d6/da/d5e/d72 0 2024-09-16T08:57:22.283 INFO:tasks.workunit.client.1.smithi097.stdout:4/954: rename d0/d7/d52/f88 to d0/d35/da4/dd1/d131/f13e 0 2024-09-16T08:57:22.284 INFO:tasks.workunit.client.0.smithi031.stdout:4/563: creat d0/db5/fb7 x:0 0 0 2024-09-16T08:57:22.285 INFO:tasks.workunit.client.1.smithi097.stdout:4/955: mkdir d0/d23/ddc/d13f 0 2024-09-16T08:57:22.286 INFO:tasks.workunit.client.0.smithi031.stdout:6/621: rename d0/dd/d35/l49 to d0/dd/d74/d19/d9a/lc1 0 2024-09-16T08:57:22.290 INFO:tasks.workunit.client.1.smithi097.stdout:4/956: rename d0/d23/f9f to d0/d35/da4/dd1/d131/f140 0 2024-09-16T08:57:22.290 INFO:tasks.workunit.client.1.smithi097.stdout:4/957: readlink d0/d23/ddc/de5/df9/lfd 0 2024-09-16T08:57:22.290 INFO:tasks.workunit.client.1.smithi097.stdout:4/958: write d0/d23/ddc/de5/df9/f13c [701189,108430] 0 2024-09-16T08:57:22.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/598: link d2/d8/d11/d66/d9a/d9c/lbb d2/d8/d11/d5d/d7e/db6/lc8 0 2024-09-16T08:57:22.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/599: getdents d2/d8/d11/d66 0 2024-09-16T08:57:22.294 INFO:tasks.workunit.client.1.smithi097.stdout:4/959: mknod d0/c141 0 2024-09-16T08:57:22.295 INFO:tasks.workunit.client.0.smithi031.stdout:8/428: mknod d4/d7/d36/c92 0 2024-09-16T08:57:22.297 INFO:tasks.workunit.client.0.smithi031.stdout:2/553: mkdir d6/d2e/d40/d6c/d2c/d53/d60/dbd 0 2024-09-16T08:57:22.299 INFO:tasks.workunit.client.1.smithi097.stdout:4/960: mkdir d0/d7/d52/d26/d8c/d142 0 2024-09-16T08:57:22.300 INFO:tasks.workunit.client.1.smithi097.stdout:4/961: chown d0/d7/d52/d26/l86 1263351 1 2024-09-16T08:57:22.300 INFO:tasks.workunit.client.1.smithi097.stdout:4/962: chown d0/d35/da4/f11b 14138 1 2024-09-16T08:57:22.301 INFO:tasks.workunit.client.0.smithi031.stdout:4/564: mkdir d0/d5b/db8 0 2024-09-16T08:57:22.303 INFO:tasks.workunit.client.0.smithi031.stdout:1/493: rename d2/d15/d32/f5f to d2/d15/d4a/faf 0 2024-09-16T08:57:22.303 INFO:tasks.workunit.client.0.smithi031.stdout:1/494: readlink d2/d15/l7d 0 2024-09-16T08:57:22.304 INFO:tasks.workunit.client.0.smithi031.stdout:0/484: getdents d6/d12/d47/d5b 0 2024-09-16T08:57:22.309 INFO:tasks.workunit.client.0.smithi031.stdout:2/554: symlink d6/d2e/d40/d6c/d2c/d53/d60/dbd/lbe 0 2024-09-16T08:57:22.310 INFO:tasks.workunit.client.1.smithi097.stdout:4/963: dread d0/fe [0,4194304] 0 2024-09-16T08:57:22.311 INFO:tasks.workunit.client.0.smithi031.stdout:4/565: dread d0/f4c [0,4194304] 0 2024-09-16T08:57:22.312 INFO:tasks.workunit.client.0.smithi031.stdout:7/600: dwrite d2/d8/f8c [0,4194304] 0 2024-09-16T08:57:22.312 INFO:tasks.workunit.client.0.smithi031.stdout:8/429: rename d4/d2d/l39 to d4/d7/d21/d24/l93 0 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: pgmap v12: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 308 MiB/s rd, 400 MiB/s wr, 639 op/s 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:22.313 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:21 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:22.314 INFO:tasks.workunit.client.0.smithi031.stdout:1/495: symlink d2/d4/d1b/d5c/lb0 0 2024-09-16T08:57:22.315 INFO:tasks.workunit.client.0.smithi031.stdout:0/485: link d6/d12/d47/c8d d6/d12/d13/d36/c99 0 2024-09-16T08:57:22.315 INFO:tasks.workunit.client.0.smithi031.stdout:6/622: dwrite d0/d10/f60 [0,4194304] 0 2024-09-16T08:57:22.315 INFO:tasks.workunit.client.0.smithi031.stdout:6/623: fsync d0/dd/d74/d67/fbf 0 2024-09-16T08:57:22.315 INFO:tasks.workunit.client.1.smithi097.stdout:4/964: getdents d0/d7/df/d15 0 2024-09-16T08:57:22.316 INFO:tasks.workunit.client.0.smithi031.stdout:4/566: truncate d0/f2d 2651661 0 2024-09-16T08:57:22.318 INFO:tasks.workunit.client.1.smithi097.stdout:4/965: getdents d0/d7/d52 0 2024-09-16T08:57:22.321 INFO:tasks.workunit.client.1.smithi097.stdout:4/966: link d0/d23/d5f/cd3 d0/d7/df/d83/df8/c143 0 2024-09-16T08:57:22.323 INFO:tasks.workunit.client.0.smithi031.stdout:8/430: unlink c3 0 2024-09-16T08:57:22.327 INFO:tasks.workunit.client.1.smithi097.stdout:7/944: sync 2024-09-16T08:57:22.327 INFO:tasks.workunit.client.1.smithi097.stdout:3/973: sync 2024-09-16T08:57:22.327 INFO:tasks.workunit.client.1.smithi097.stdout:3/974: fsync d7/de/d3c/d3f/f50 0 2024-09-16T08:57:22.327 INFO:tasks.workunit.client.1.smithi097.stdout:7/945: dread - d4/d1b/da4/fee zero size 2024-09-16T08:57:22.327 INFO:tasks.workunit.client.1.smithi097.stdout:7/946: creat d4/d9/d2b/f13e x:0 0 0 2024-09-16T08:57:22.327 INFO:tasks.workunit.client.1.smithi097.stdout:7/947: chown d4/d9/d51/d8b/c130 4 1 2024-09-16T08:57:22.331 INFO:tasks.workunit.client.1.smithi097.stdout:3/975: rename d7/de/c2e to d7/de/d9c/daf/c14a 0 2024-09-16T08:57:22.334 INFO:tasks.workunit.client.0.smithi031.stdout:6/624: creat d0/dd/d74/d19/fc2 x:0 0 0 2024-09-16T08:57:22.335 INFO:tasks.workunit.client.0.smithi031.stdout:4/567: fsync d0/d2/d3a/f97 0 2024-09-16T08:57:22.335 INFO:tasks.workunit.client.1.smithi097.stdout:7/948: mknod d4/d7/d19/ddd/c13f 0 2024-09-16T08:57:22.336 INFO:tasks.workunit.client.1.smithi097.stdout:7/949: dread - d4/d7/f107 zero size 2024-09-16T08:57:22.336 INFO:tasks.workunit.client.1.smithi097.stdout:7/950: write d4/d7/d19/ddd/d134/daf/f108 [182218,67275] 0 2024-09-16T08:57:22.336 INFO:tasks.workunit.client.1.smithi097.stdout:3/976: creat d7/d15/dee/d11f/d30/d88/df7/f14b x:0 0 0 2024-09-16T08:57:22.337 INFO:tasks.workunit.client.0.smithi031.stdout:2/555: rename d6/da/d37/d42/l96 to d6/da/d37/d8b/dad/lbf 0 2024-09-16T08:57:22.337 INFO:tasks.workunit.client.0.smithi031.stdout:2/556: chown d6/d2e/d40/d6c/d2c/d53/d60/f35 8404578 1 2024-09-16T08:57:22.338 INFO:tasks.workunit.client.1.smithi097.stdout:7/951: symlink d4/dfa/d135/l140 0 2024-09-16T08:57:22.338 INFO:tasks.workunit.client.0.smithi031.stdout:8/431: dwrite d4/d7/d21/f3b [4194304,4194304] 0 2024-09-16T08:57:22.339 INFO:tasks.workunit.client.0.smithi031.stdout:0/486: mknod d6/d61/c9a 0 2024-09-16T08:57:22.339 INFO:tasks.workunit.client.0.smithi031.stdout:0/487: dread - d6/d12/f49 zero size 2024-09-16T08:57:22.339 INFO:tasks.workunit.client.0.smithi031.stdout:0/488: readlink d6/d12/d13/l73 0 2024-09-16T08:57:22.339 INFO:tasks.workunit.client.0.smithi031.stdout:0/489: chown d6/d12/d13/c35 107 1 2024-09-16T08:57:22.339 INFO:tasks.workunit.client.0.smithi031.stdout:6/625: creat d0/dd/d74/da6/fc3 x:0 0 0 2024-09-16T08:57:22.341 INFO:tasks.workunit.client.0.smithi031.stdout:4/568: mkdir d0/d2/d16/d1e/db9 0 2024-09-16T08:57:22.342 INFO:tasks.workunit.client.1.smithi097.stdout:7/952: dread d4/ff6 [0,4194304] 0 2024-09-16T08:57:22.342 INFO:tasks.workunit.client.1.smithi097.stdout:7/953: fsync d4/d1b/fac 0 2024-09-16T08:57:22.342 INFO:tasks.workunit.client.1.smithi097.stdout:7/954: chown d4/d9/dc/dc6/l10d 164724 1 2024-09-16T08:57:22.346 INFO:tasks.workunit.client.0.smithi031.stdout:7/601: rename d2/d2d/f58 to d2/d8/d11/d66/d9a/fc9 0 2024-09-16T08:57:22.348 INFO:tasks.workunit.client.0.smithi031.stdout:2/557: symlink d6/d2e/d40/d58/lc0 0 2024-09-16T08:57:22.349 INFO:tasks.workunit.client.0.smithi031.stdout:8/432: mknod d4/d5b/d8d/c94 0 2024-09-16T08:57:22.349 INFO:tasks.workunit.client.0.smithi031.stdout:8/433: chown d4/d5b/l90 62176 1 2024-09-16T08:57:22.349 INFO:tasks.workunit.client.0.smithi031.stdout:8/434: truncate d4/d7/d36/f7e 4803776 0 2024-09-16T08:57:22.352 INFO:tasks.workunit.client.0.smithi031.stdout:6/626: creat d0/dd/d74/d67/dbb/fc4 x:0 0 0 2024-09-16T08:57:22.353 INFO:tasks.workunit.client.0.smithi031.stdout:1/496: rename d2/d4/d1b/f78 to d2/de/d6c/fb1 0 2024-09-16T08:57:22.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/558: creat d6/da/d4b/fc1 x:0 0 0 2024-09-16T08:57:22.356 INFO:tasks.workunit.client.0.smithi031.stdout:2/559: stat d6/d2e/d40/d6c/d2c/d53/d60/f35 0 2024-09-16T08:57:22.358 INFO:tasks.workunit.client.0.smithi031.stdout:8/435: creat d4/d2f/f95 x:0 0 0 2024-09-16T08:57:22.359 INFO:tasks.workunit.client.0.smithi031.stdout:6/627: mkdir d0/dd/d74/d5e/d7c/d56/dc5 0 2024-09-16T08:57:22.359 INFO:tasks.workunit.client.0.smithi031.stdout:6/628: dread - d0/dd/d74/d19/d9a/fc0 zero size 2024-09-16T08:57:22.362 INFO:tasks.workunit.client.0.smithi031.stdout:7/602: unlink d2/d8/d11/f74 0 2024-09-16T08:57:22.362 INFO:tasks.workunit.client.0.smithi031.stdout:7/603: fdatasync d2/d8/d11/d5d/d7e/fb9 0 2024-09-16T08:57:22.362 INFO:tasks.workunit.client.0.smithi031.stdout:7/604: fsync d2/d2d/d6d/fae 0 2024-09-16T08:57:22.362 INFO:tasks.workunit.client.0.smithi031.stdout:7/605: truncate d2/d8/d11/d66/d9a/d9c/fba 276949 0 2024-09-16T08:57:22.362 INFO:tasks.workunit.client.0.smithi031.stdout:8/436: symlink d4/d5b/d8d/l96 0 2024-09-16T08:57:22.364 INFO:tasks.workunit.client.0.smithi031.stdout:6/629: creat d0/dd/d74/d5e/d7c/d3f/d92/fc6 x:0 0 0 2024-09-16T08:57:22.364 INFO:tasks.workunit.client.0.smithi031.stdout:6/630: chown d0/dd/d74/d19/d9a/fc0 89646 1 2024-09-16T08:57:22.364 INFO:tasks.workunit.client.0.smithi031.stdout:6/631: chown d0/dd/d74/d19/c8c 62 1 2024-09-16T08:57:22.365 INFO:tasks.workunit.client.0.smithi031.stdout:1/497: getdents d2/d15/d1f/d43 0 2024-09-16T08:57:22.366 INFO:tasks.workunit.client.0.smithi031.stdout:7/606: creat d2/d8/d11/d66/d9a/fca x:0 0 0 2024-09-16T08:57:22.367 INFO:tasks.workunit.client.0.smithi031.stdout:6/632: creat d0/d10/fc7 x:0 0 0 2024-09-16T08:57:22.369 INFO:tasks.workunit.client.0.smithi031.stdout:7/607: creat d2/d2d/d68/fcb x:0 0 0 2024-09-16T08:57:22.372 INFO:tasks.workunit.client.0.smithi031.stdout:1/498: dread d2/de/f2f [0,4194304] 0 2024-09-16T08:57:22.372 INFO:tasks.workunit.client.0.smithi031.stdout:2/560: write d6/da/f3d [3144282,45786] 0 2024-09-16T08:57:22.373 INFO:tasks.workunit.client.0.smithi031.stdout:6/633: unlink d0/dd/d35/cb5 0 2024-09-16T08:57:22.373 INFO:tasks.workunit.client.0.smithi031.stdout:6/634: chown d0/d8e 196623 1 2024-09-16T08:57:22.373 INFO:tasks.workunit.client.0.smithi031.stdout:6/635: truncate d0/dd/d74/d19/f85 9425494 0 2024-09-16T08:57:22.375 INFO:tasks.workunit.client.0.smithi031.stdout:7/608: mknod d2/ccc 0 2024-09-16T08:57:22.378 INFO:tasks.workunit.client.0.smithi031.stdout:1/499: rename d2/f29 to d2/d15/d38/fb2 0 2024-09-16T08:57:22.378 INFO:tasks.workunit.client.0.smithi031.stdout:1/500: chown d2/d4/d52/c71 241531 1 2024-09-16T08:57:22.378 INFO:tasks.workunit.client.0.smithi031.stdout:1/501: creat d2/d4/d1b/fb3 x:0 0 0 2024-09-16T08:57:22.379 INFO:tasks.workunit.client.0.smithi031.stdout:7/609: symlink d2/d8/d86/lcd 0 2024-09-16T08:57:22.383 INFO:tasks.workunit.client.0.smithi031.stdout:7/610: rmdir d2/d8/d11/d5d 39 2024-09-16T08:57:22.384 INFO:tasks.workunit.client.0.smithi031.stdout:7/611: creat d2/d8/d11/d5d/d7e/db6/fce x:0 0 0 2024-09-16T08:57:22.384 INFO:tasks.workunit.client.0.smithi031.stdout:7/612: fdatasync d2/d2d/f77 0 2024-09-16T08:57:22.385 INFO:tasks.workunit.client.0.smithi031.stdout:7/613: rename d2/d2d/d68/f6b to d2/d2d/fcf 0 2024-09-16T08:57:22.385 INFO:tasks.workunit.client.0.smithi031.stdout:7/614: chown d2/d8/f57 256421474 1 2024-09-16T08:57:22.385 INFO:tasks.workunit.client.0.smithi031.stdout:7/615: read - d2/d8/d11/fa4 zero size 2024-09-16T08:57:22.387 INFO:tasks.workunit.client.0.smithi031.stdout:7/616: creat d2/d8/d69/d82/fd0 x:0 0 0 2024-09-16T08:57:22.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/617: creat d2/d8/d69/d82/fd1 x:0 0 0 2024-09-16T08:57:22.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/618: write d2/d8/d11/d66/d9a/d9c/fba [304250,18656] 0 2024-09-16T08:57:22.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/619: creat d2/d8/d11/d5d/d7e/db6/fd2 x:0 0 0 2024-09-16T08:57:22.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/620: truncate d2/d8/d11/d5d/fc0 851758 0 2024-09-16T08:57:22.388 INFO:tasks.workunit.client.0.smithi031.stdout:7/621: readlink d2/d8/l53 0 2024-09-16T08:57:22.389 INFO:tasks.workunit.client.0.smithi031.stdout:1/502: write d2/d4/fa2 [4086778,84240] 0 2024-09-16T08:57:22.391 INFO:tasks.workunit.client.0.smithi031.stdout:1/503: rename d2/d9b to d2/d15/d5d/db4 0 2024-09-16T08:57:22.391 INFO:tasks.workunit.client.0.smithi031.stdout:1/504: mkdir d2/de/db5 0 2024-09-16T08:57:22.394 INFO:tasks.workunit.client.1.smithi097.stdout:7/955: dwrite d4/d7/d77/da8/de2/f12a [0,4194304] 0 2024-09-16T08:57:22.394 INFO:tasks.workunit.client.1.smithi097.stdout:7/956: chown d4/d12/d1f/f3b 41692803 1 2024-09-16T08:57:22.397 INFO:tasks.workunit.client.0.smithi031.stdout:2/561: dwrite d6/da/d5e/d8d/f9e [0,4194304] 0 2024-09-16T08:57:22.397 INFO:tasks.workunit.client.0.smithi031.stdout:2/562: write d6/da/fd [4456586,113006] 0 2024-09-16T08:57:22.397 INFO:tasks.workunit.client.0.smithi031.stdout:0/490: dwrite d6/d12/d13/d3c/f70 [0,4194304] 0 2024-09-16T08:57:22.398 INFO:tasks.workunit.client.0.smithi031.stdout:6/636: dread d0/dd/d74/d5e/d7c/f47 [0,4194304] 0 2024-09-16T08:57:22.398 INFO:tasks.workunit.client.0.smithi031.stdout:8/437: dwrite d4/d7/d36/d38/f3c [0,4194304] 0 2024-09-16T08:57:22.398 INFO:tasks.workunit.client.0.smithi031.stdout:4/569: dwrite d0/d2/f1a [0,4194304] 0 2024-09-16T08:57:22.399 INFO:tasks.workunit.client.1.smithi097.stdout:3/977: dwrite d7/d15/dee/d11f/d30/f53 [0,4194304] 0 2024-09-16T08:57:22.399 INFO:tasks.workunit.client.1.smithi097.stdout:7/957: creat d4/daa/f141 x:0 0 0 2024-09-16T08:57:22.399 INFO:tasks.workunit.client.1.smithi097.stdout:7/958: write d4/d9/dc/f90 [5032354,40861] 0 2024-09-16T08:57:22.400 INFO:tasks.workunit.client.1.smithi097.stdout:7/959: mkdir d4/d1b/d142 0 2024-09-16T08:57:22.401 INFO:tasks.workunit.client.1.smithi097.stdout:7/960: readlink d4/d7/d30/lf8 0 2024-09-16T08:57:22.402 INFO:tasks.workunit.client.0.smithi031.stdout:2/563: rename d6/da/d37/c79 to d6/da/d5e/d8d/cc2 0 2024-09-16T08:57:22.403 INFO:tasks.workunit.client.0.smithi031.stdout:0/491: mkdir d6/d12/d47/d5b/d9b 0 2024-09-16T08:57:22.403 INFO:tasks.workunit.client.0.smithi031.stdout:2/564: dread d6/d20/f32 [0,4194304] 0 2024-09-16T08:57:22.403 INFO:tasks.workunit.client.0.smithi031.stdout:0/492: fsync d6/d12/d47/d5b/f5e 0 2024-09-16T08:57:22.404 INFO:tasks.workunit.client.1.smithi097.stdout:7/961: mkdir d4/d7/d77/da8/de2/d143 0 2024-09-16T08:57:22.404 INFO:tasks.workunit.client.1.smithi097.stdout:7/962: write d4/daa/f141 [241324,103494] 0 2024-09-16T08:57:22.406 INFO:tasks.workunit.client.0.smithi031.stdout:8/438: mkdir d4/d2d/d97 0 2024-09-16T08:57:22.406 INFO:tasks.workunit.client.0.smithi031.stdout:8/439: readlink d4/d7/d21/d37/d46/l5c 0 2024-09-16T08:57:22.408 INFO:tasks.workunit.client.1.smithi097.stdout:7/963: write d4/d1b/f91 [1828795,112529] 0 2024-09-16T08:57:22.410 INFO:tasks.workunit.client.0.smithi031.stdout:4/570: truncate d0/d8e/f77 1023324 0 2024-09-16T08:57:22.413 INFO:tasks.workunit.client.0.smithi031.stdout:1/505: dwrite d2/d4/d1b/f50 [0,4194304] 0 2024-09-16T08:57:22.413 INFO:tasks.workunit.client.0.smithi031.stdout:1/506: creat d2/d15/d4a/fb6 x:0 0 0 2024-09-16T08:57:22.413 INFO:tasks.workunit.client.0.smithi031.stdout:6/637: rename d0/dd/d74/d8b to d0/d10/dc8 0 2024-09-16T08:57:22.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/638: truncate d0/dd/d74/da6/fc3 27153 0 2024-09-16T08:57:22.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/639: chown d0/c9d 131 1 2024-09-16T08:57:22.414 INFO:tasks.workunit.client.0.smithi031.stdout:6/640: stat d0/dd/d74/d5e/c6b 0 2024-09-16T08:57:22.415 INFO:tasks.workunit.client.0.smithi031.stdout:2/565: truncate d6/d2e/d40/d6c/f2b 2835352 0 2024-09-16T08:57:22.415 INFO:tasks.workunit.client.0.smithi031.stdout:7/622: dwrite d2/d8/d11/d81/fa5 [0,4194304] 0 2024-09-16T08:57:22.416 INFO:tasks.workunit.client.0.smithi031.stdout:7/623: write d2/d8/d69/d82/f96 [1032363,41552] 0 2024-09-16T08:57:22.417 INFO:tasks.workunit.client.0.smithi031.stdout:0/493: symlink d6/d12/d13/d36/l9c 0 2024-09-16T08:57:22.418 INFO:tasks.workunit.client.0.smithi031.stdout:8/440: creat d4/d7/d21/d35/d50/f98 x:0 0 0 2024-09-16T08:57:22.422 INFO:tasks.workunit.client.0.smithi031.stdout:1/507: creat d2/d15/d1f/d43/fb7 x:0 0 0 2024-09-16T08:57:22.423 INFO:tasks.workunit.client.1.smithi097.stdout:3/978: dwrite d7/d15/dee/d11f/d30/f94 [0,4194304] 0 2024-09-16T08:57:22.423 INFO:tasks.workunit.client.1.smithi097.stdout:3/979: write d7/de/d8d/d90/f126 [363922,2859] 0 2024-09-16T08:57:22.424 INFO:tasks.workunit.client.0.smithi031.stdout:6/641: rename d0/dd/d74/d19/f1d to d0/dd/d74/dad/fc9 0 2024-09-16T08:57:22.424 INFO:tasks.workunit.client.0.smithi031.stdout:1/508: rename d2/d15 to d2/d15/d32/db8 22 2024-09-16T08:57:22.424 INFO:tasks.workunit.client.0.smithi031.stdout:6/642: dread - d0/dd/d74/d19/fc2 zero size 2024-09-16T08:57:22.425 INFO:tasks.workunit.client.1.smithi097.stdout:7/964: dread d4/d7/d19/ddd/d134/daf/f108 [0,4194304] 0 2024-09-16T08:57:22.428 INFO:tasks.workunit.client.1.smithi097.stdout:3/980: dread d7/de/d3c/d3f/d60/dcb/fd5 [0,4194304] 0 2024-09-16T08:57:22.429 INFO:tasks.workunit.client.0.smithi031.stdout:2/566: write d6/da/f27 [1246723,10917] 0 2024-09-16T08:57:22.429 INFO:tasks.workunit.client.0.smithi031.stdout:2/567: truncate d6/da/f5d 4664180 0 2024-09-16T08:57:22.431 INFO:tasks.workunit.client.0.smithi031.stdout:0/494: truncate d6/d12/d13/f3e 613062 0 2024-09-16T08:57:22.432 INFO:tasks.workunit.client.0.smithi031.stdout:8/441: mkdir d4/d7/d99 0 2024-09-16T08:57:22.433 INFO:tasks.workunit.client.0.smithi031.stdout:8/442: write d4/d7/d21/d35/f5d [4543772,28126] 0 2024-09-16T08:57:22.433 INFO:tasks.workunit.client.1.smithi097.stdout:3/981: mknod d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/c14c 0 2024-09-16T08:57:22.439 INFO:tasks.workunit.client.0.smithi031.stdout:6/643: write d0/dd/d74/d19/f2e [6406147,70318] 0 2024-09-16T08:57:22.439 INFO:tasks.workunit.client.0.smithi031.stdout:8/443: dread d4/d7/d21/d37/d46/d8c/f58 [0,4194304] 0 2024-09-16T08:57:22.441 INFO:tasks.workunit.client.1.smithi097.stdout:3/982: read d7/de/d3c/d3f/d60/dcb/d35/f22 [471011,1560] 0 2024-09-16T08:57:22.441 INFO:tasks.workunit.client.0.smithi031.stdout:8/444: read d4/d7/d36/d38/f87 [778206,125834] 0 2024-09-16T08:57:22.441 INFO:tasks.workunit.client.0.smithi031.stdout:8/445: fdatasync d4/d34/f6a 0 2024-09-16T08:57:22.441 INFO:tasks.workunit.client.0.smithi031.stdout:7/624: dwrite d2/d8/d69/d82/fa0 [0,4194304] 0 2024-09-16T08:57:22.446 INFO:tasks.workunit.client.1.smithi097.stdout:3/983: creat d7/de/d9c/dca/f14d x:0 0 0 2024-09-16T08:57:22.449 INFO:tasks.workunit.client.0.smithi031.stdout:6/644: mkdir d0/dd/d74/d19/d54/dca 0 2024-09-16T08:57:22.449 INFO:tasks.workunit.client.0.smithi031.stdout:6/645: dread - d0/dd/d74/d19/d54/f9c zero size 2024-09-16T08:57:22.450 INFO:tasks.workunit.client.0.smithi031.stdout:1/509: dwrite d2/f2c [0,4194304] 0 2024-09-16T08:57:22.451 INFO:tasks.workunit.client.0.smithi031.stdout:0/495: dwrite d6/d12/d13/d36/f56 [4194304,4194304] 0 2024-09-16T08:57:22.452 INFO:tasks.workunit.client.1.smithi097.stdout:3/984: getdents d7/d15/dee/d11f/d30/d88/db3 0 2024-09-16T08:57:22.452 INFO:tasks.workunit.client.1.smithi097.stdout:3/985: stat d7/de/d3c/d3f/d11c/d101 0 2024-09-16T08:57:22.452 INFO:tasks.workunit.client.1.smithi097.stdout:3/986: chown d7/de/d3c/dde 50438 1 2024-09-16T08:57:22.452 INFO:tasks.workunit.client.1.smithi097.stdout:3/987: write d7/de/d9c/dca/f14d [383615,14731] 0 2024-09-16T08:57:22.453 INFO:tasks.workunit.client.0.smithi031.stdout:7/625: unlink d2/d8/d11/d5d/d7e/fc5 0 2024-09-16T08:57:22.453 INFO:tasks.workunit.client.0.smithi031.stdout:7/626: chown d2 0 1 2024-09-16T08:57:22.454 INFO:tasks.workunit.client.0.smithi031.stdout:8/446: dread d4/d7/d66/f70 [0,4194304] 0 2024-09-16T08:57:22.454 INFO:tasks.workunit.client.0.smithi031.stdout:8/447: read d4/d7/d36/f7e [3631226,116528] 0 2024-09-16T08:57:22.455 INFO:tasks.workunit.client.0.smithi031.stdout:6/646: mknod d0/dd/d35/d6c/d83/ccb 0 2024-09-16T08:57:22.455 INFO:tasks.workunit.client.0.smithi031.stdout:6/647: read - d0/dd/d74/d19/d54/f9c zero size 2024-09-16T08:57:22.459 INFO:tasks.workunit.client.0.smithi031.stdout:0/496: write d6/f69 [750090,55335] 0 2024-09-16T08:57:22.460 INFO:tasks.workunit.client.0.smithi031.stdout:7/627: rmdir d2/d8/d86 39 2024-09-16T08:57:22.461 INFO:tasks.workunit.client.0.smithi031.stdout:6/648: symlink d0/dd/d74/d67/lcc 0 2024-09-16T08:57:22.462 INFO:tasks.workunit.client.0.smithi031.stdout:1/510: getdents d2/d4/d1b 0 2024-09-16T08:57:22.463 INFO:tasks.workunit.client.0.smithi031.stdout:7/628: creat d2/d8/d86/da1/da9/fd3 x:0 0 0 2024-09-16T08:57:22.472 INFO:tasks.workunit.client.0.smithi031.stdout:6/649: read d0/ff [264374,24734] 0 2024-09-16T08:57:22.476 INFO:tasks.workunit.client.1.smithi097.stdout:4/967: sync 2024-09-16T08:57:22.477 INFO:tasks.workunit.client.1.smithi097.stdout:4/968: readlink d0/d7/l61 0 2024-09-16T08:57:22.477 INFO:tasks.workunit.client.1.smithi097.stdout:4/969: creat d0/d23/dda/f144 x:0 0 0 2024-09-16T08:57:22.478 INFO:tasks.workunit.client.1.smithi097.stdout:4/970: mknod d0/c145 0 2024-09-16T08:57:22.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/650: write d0/dd/f3e [3321121,2555] 0 2024-09-16T08:57:22.480 INFO:tasks.workunit.client.0.smithi031.stdout:8/448: dread d4/d7/d36/f7e [4194304,4194304] 0 2024-09-16T08:57:22.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/651: write d0/dd/d74/f1e [6338433,32126] 0 2024-09-16T08:57:22.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/652: write d0/dd/d74/d5e/d7c/d3f/f9f [504621,16744] 0 2024-09-16T08:57:22.480 INFO:tasks.workunit.client.1.smithi097.stdout:4/971: unlink d0/d7/d52/d26/d8c/db0/cc2 0 2024-09-16T08:57:22.484 INFO:tasks.workunit.client.0.smithi031.stdout:8/449: unlink d4/d7/d36/l6d 0 2024-09-16T08:57:22.486 INFO:tasks.workunit.client.1.smithi097.stdout:4/972: rename d0/d36/d4c/dea/l136 to d0/d39/l146 0 2024-09-16T08:57:22.487 INFO:tasks.workunit.client.0.smithi031.stdout:6/653: mkdir d0/dd/d74/d19/dcd 0 2024-09-16T08:57:22.489 INFO:tasks.workunit.client.0.smithi031.stdout:8/450: creat d4/d7/d36/d5e/f9a x:0 0 0 2024-09-16T08:57:22.489 INFO:tasks.workunit.client.0.smithi031.stdout:8/451: creat d4/d77/f9b x:0 0 0 2024-09-16T08:57:22.489 INFO:tasks.workunit.client.0.smithi031.stdout:7/629: read d2/d8/d11/f5c [1585491,4283] 0 2024-09-16T08:57:22.489 INFO:tasks.workunit.client.0.smithi031.stdout:7/630: dread - d2/d2d/d68/fcb zero size 2024-09-16T08:57:22.490 INFO:tasks.workunit.client.0.smithi031.stdout:7/631: read - d2/d8/d69/d82/fd0 zero size 2024-09-16T08:57:22.490 INFO:tasks.workunit.client.0.smithi031.stdout:7/632: fsync d2/d8/f2f 0 2024-09-16T08:57:22.491 INFO:tasks.workunit.client.1.smithi097.stdout:4/973: rename d0/d7/d52/d26/d8c/d11e to d0/d7/df/d147 0 2024-09-16T08:57:22.492 INFO:tasks.workunit.client.0.smithi031.stdout:0/497: dread d6/d12/d47/f55 [0,4194304] 0 2024-09-16T08:57:22.492 INFO:tasks.workunit.client.0.smithi031.stdout:6/654: creat d0/dd/d74/d5e/d7c/d56/fce x:0 0 0 2024-09-16T08:57:22.493 INFO:tasks.workunit.client.0.smithi031.stdout:6/655: fdatasync d0/dd/d74/d19/f34 0 2024-09-16T08:57:22.496 INFO:tasks.workunit.client.1.smithi097.stdout:4/974: unlink d0/f30 0 2024-09-16T08:57:22.496 INFO:tasks.workunit.client.1.smithi097.stdout:4/975: chown d0/d39/d119/f11f 48674 1 2024-09-16T08:57:22.496 INFO:tasks.workunit.client.1.smithi097.stdout:4/976: write d0/d36/da7/f112 [785536,86609] 0 2024-09-16T08:57:22.497 INFO:tasks.workunit.client.0.smithi031.stdout:8/452: rename d4/d7/d21/f82 to d4/d7/d36/d38/f9c 0 2024-09-16T08:57:22.497 INFO:tasks.workunit.client.0.smithi031.stdout:8/453: write d4/d2d/f3a [5273818,41764] 0 2024-09-16T08:57:22.497 INFO:tasks.workunit.client.0.smithi031.stdout:8/454: readlink d4/l1d 0 2024-09-16T08:57:22.500 INFO:tasks.workunit.client.1.smithi097.stdout:4/977: dread d0/d35/da4/f11b [0,4194304] 0 2024-09-16T08:57:22.500 INFO:tasks.workunit.client.1.smithi097.stdout:3/988: dwrite d7/de/d3c/f73 [0,4194304] 0 2024-09-16T08:57:22.500 INFO:tasks.workunit.client.1.smithi097.stdout:3/989: fdatasync d7/d15/dee/d11f/d63/fa6 0 2024-09-16T08:57:22.500 INFO:tasks.workunit.client.0.smithi031.stdout:0/498: mknod d6/d12/d47/c9d 0 2024-09-16T08:57:22.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/455: symlink d4/d7/d21/d37/d46/l9d 0 2024-09-16T08:57:22.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/456: chown d4/d7/d21/d35/d79/f8a 6874 1 2024-09-16T08:57:22.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/457: chown d4/d77/l80 16734 1 2024-09-16T08:57:22.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/458: truncate d4/d7/d21/f23 4988793 0 2024-09-16T08:57:22.501 INFO:tasks.workunit.client.0.smithi031.stdout:7/633: getdents d2/d8/d11/d66/d9a/d9c 0 2024-09-16T08:57:22.502 INFO:tasks.workunit.client.0.smithi031.stdout:7/634: dread - d2/d8/d11/d5d/d7e/db6/fd2 zero size 2024-09-16T08:57:22.503 INFO:tasks.workunit.client.0.smithi031.stdout:0/499: mkdir d6/d12/d97/d9e 0 2024-09-16T08:57:22.503 INFO:tasks.workunit.client.0.smithi031.stdout:0/500: chown d6 148861194 1 2024-09-16T08:57:22.503 INFO:tasks.workunit.client.0.smithi031.stdout:8/459: truncate d4/d7/d21/f3b 1857384 0 2024-09-16T08:57:22.503 INFO:tasks.workunit.client.0.smithi031.stdout:8/460: dread - d4/d5b/d7b/f7d zero size 2024-09-16T08:57:22.505 INFO:tasks.workunit.client.0.smithi031.stdout:7/635: mkdir d2/d8/d86/dd4 0 2024-09-16T08:57:22.507 INFO:tasks.workunit.client.0.smithi031.stdout:0/501: dread d6/d12/d47/f55 [0,4194304] 0 2024-09-16T08:57:22.509 INFO:tasks.workunit.client.1.smithi097.stdout:3/990: mknod d7/de/d3c/d3f/d60/d6b/db8/deb/c14e 0 2024-09-16T08:57:22.509 INFO:tasks.workunit.client.1.smithi097.stdout:4/978: truncate d0/d7/d52/dbf/f100 4166961 0 2024-09-16T08:57:22.511 INFO:tasks.workunit.client.0.smithi031.stdout:8/461: getdents d4/d5b/d7b 0 2024-09-16T08:57:22.512 INFO:tasks.workunit.client.0.smithi031.stdout:0/502: creat d6/d61/d91/f9f x:0 0 0 2024-09-16T08:57:22.515 INFO:tasks.workunit.client.0.smithi031.stdout:0/503: symlink d6/d12/d47/d5b/la0 0 2024-09-16T08:57:22.515 INFO:tasks.workunit.client.0.smithi031.stdout:0/504: creat d6/d12/d47/d4d/fa1 x:0 0 0 2024-09-16T08:57:22.516 INFO:tasks.workunit.client.1.smithi097.stdout:4/979: read d0/d7/d52/d26/f6b [679551,92689] 0 2024-09-16T08:57:22.516 INFO:tasks.workunit.client.1.smithi097.stdout:4/980: creat d0/d23/f148 x:0 0 0 2024-09-16T08:57:22.520 INFO:tasks.workunit.client.0.smithi031.stdout:6/656: dwrite d0/dd/d74/d19/f61 [4194304,4194304] 0 2024-09-16T08:57:22.521 INFO:tasks.workunit.client.1.smithi097.stdout:4/981: creat d0/d7/d52/d26/d3d/f149 x:0 0 0 2024-09-16T08:57:22.522 INFO:tasks.workunit.client.0.smithi031.stdout:0/505: mkdir d6/d12/d47/d5b/d9b/da2 0 2024-09-16T08:57:22.526 INFO:tasks.workunit.client.0.smithi031.stdout:8/462: rename d4/d7/d36 to d4/d9e 0 2024-09-16T08:57:22.526 INFO:tasks.workunit.client.0.smithi031.stdout:8/463: truncate d4/d2f/f95 92889 0 2024-09-16T08:57:22.526 INFO:tasks.workunit.client.0.smithi031.stdout:0/506: mknod d6/d12/d13/ca3 0 2024-09-16T08:57:22.527 INFO:tasks.workunit.client.0.smithi031.stdout:0/507: truncate d6/d12/d47/f55 1745912 0 2024-09-16T08:57:22.529 INFO:tasks.workunit.client.0.smithi031.stdout:0/508: truncate d6/d12/d47/f83 2039361 0 2024-09-16T08:57:22.530 INFO:tasks.workunit.client.0.smithi031.stdout:0/509: symlink d6/d12/d47/d7c/la4 0 2024-09-16T08:57:22.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/510: truncate d6/d12/d47/f7e 151844 0 2024-09-16T08:57:22.534 INFO:tasks.workunit.client.1.smithi097.stdout:7/965: sync 2024-09-16T08:57:22.534 INFO:tasks.workunit.client.1.smithi097.stdout:7/966: chown d4/f23 2511 1 2024-09-16T08:57:22.534 INFO:tasks.workunit.client.1.smithi097.stdout:7/967: creat d4/d7/d19/f144 x:0 0 0 2024-09-16T08:57:22.536 INFO:tasks.workunit.client.1.smithi097.stdout:3/991: dwrite d7/de/d3c/d3f/d60/dcb/d35/d3d/d57/d64/f120 [0,4194304] 0 2024-09-16T08:57:22.536 INFO:tasks.workunit.client.1.smithi097.stdout:3/992: chown d7/de/d3c/d3f/d60/d7a/c12e 496736 1 2024-09-16T08:57:22.537 INFO:tasks.workunit.client.0.smithi031.stdout:0/511: mkdir d6/d12/d13/da5 0 2024-09-16T08:57:22.537 INFO:tasks.workunit.client.0.smithi031.stdout:0/512: write d6/d12/d13/f68 [755074,1369] 0 2024-09-16T08:57:22.537 INFO:tasks.workunit.client.0.smithi031.stdout:7/636: dwrite d2/d8/d11/d66/d9a/d9c/fba [0,4194304] 0 2024-09-16T08:57:22.538 INFO:tasks.workunit.client.1.smithi097.stdout:7/968: rmdir d4/d1b/d41 39 2024-09-16T08:57:22.538 INFO:tasks.workunit.client.1.smithi097.stdout:7/969: chown d4/d12/d1f 47 1 2024-09-16T08:57:22.539 INFO:tasks.workunit.client.0.smithi031.stdout:6/657: write d0/dd/d74/dad/fc9 [5434822,80844] 0 2024-09-16T08:57:22.539 INFO:tasks.workunit.client.0.smithi031.stdout:6/658: write d0/dd/d74/d19/fa1 [1934217,82170] 0 2024-09-16T08:57:22.545 INFO:tasks.workunit.client.1.smithi097.stdout:3/993: symlink d7/de/d6c/dcf/l14f 0 2024-09-16T08:57:22.547 INFO:tasks.workunit.client.0.smithi031.stdout:7/637: write d2/d8/d11/d81/fa5 [3894390,113054] 0 2024-09-16T08:57:22.548 INFO:tasks.workunit.client.0.smithi031.stdout:6/659: getdents d0/d10 0 2024-09-16T08:57:22.552 INFO:tasks.workunit.client.1.smithi097.stdout:7/970: rmdir d4/d7/d79 39 2024-09-16T08:57:22.555 INFO:tasks.workunit.client.1.smithi097.stdout:3/994: rename d7/de/d3c/d3f/d60/dcb/d35 to d7/d15/dee/d11f/d30/d150 0 2024-09-16T08:57:22.555 INFO:tasks.workunit.client.1.smithi097.stdout:3/995: chown d7/d15/dee/d11f/d30/d150/f22 16 1 2024-09-16T08:57:22.555 INFO:tasks.workunit.client.0.smithi031.stdout:7/638: mkdir d2/d8/d11/dd5 0 2024-09-16T08:57:22.556 INFO:tasks.workunit.client.0.smithi031.stdout:6/660: rmdir d0/dd/d74/d19/db4 39 2024-09-16T08:57:22.556 INFO:tasks.workunit.client.0.smithi031.stdout:6/661: creat d0/dd/d74/da6/fcf x:0 0 0 2024-09-16T08:57:22.556 INFO:tasks.workunit.client.0.smithi031.stdout:8/464: dwrite d4/d2f/f41 [0,4194304] 0 2024-09-16T08:57:22.557 INFO:tasks.workunit.client.0.smithi031.stdout:0/513: dwrite d6/d61/f62 [0,4194304] 0 2024-09-16T08:57:22.557 INFO:tasks.workunit.client.0.smithi031.stdout:0/514: fdatasync d6/d12/d13/d36/f51 0 2024-09-16T08:57:22.557 INFO:tasks.workunit.client.0.smithi031.stdout:7/639: read d2/f1a [5121475,29184] 0 2024-09-16T08:57:22.558 INFO:tasks.workunit.client.0.smithi031.stdout:7/640: read - d2/d8/d11/f44 zero size 2024-09-16T08:57:22.558 INFO:tasks.workunit.client.0.smithi031.stdout:7/641: write d2/d8/d69/d82/f9b [442361,49673] 0 2024-09-16T08:57:22.558 INFO:tasks.workunit.client.0.smithi031.stdout:7/642: getdents d2/d8/d11/dd5 0 2024-09-16T08:57:22.559 INFO:tasks.workunit.client.1.smithi097.stdout:3/996: rename d7/de/d6c/l39 to d7/de/d6c/l151 0 2024-09-16T08:57:22.563 INFO:tasks.workunit.client.1.smithi097.stdout:3/997: rename d7/de/d3c/d3f/d60/d6b to d7/de/d8d/d90/d96/d9f/db5/d152 0 2024-09-16T08:57:22.563 INFO:tasks.workunit.client.1.smithi097.stdout:3/998: getdents d7/d15/dee/d11f/d30/d150/d3d 0 2024-09-16T08:57:22.565 INFO:tasks.workunit.client.1.smithi097.stdout:3/999: link d7/de/d3c/d3f/d60/d7a/fa1 d7/de/d3c/d3f/d60/f153 0 2024-09-16T08:57:22.567 INFO:tasks.workunit.client.0.smithi031.stdout:0/515: rmdir d6/d12/d47/d7c 39 2024-09-16T08:57:22.568 INFO:tasks.workunit.client.0.smithi031.stdout:0/516: dread - d6/d12/d47/d5b/f92 zero size 2024-09-16T08:57:22.568 INFO:tasks.workunit.client.0.smithi031.stdout:0/517: fsync d6/d12/d13/f21 0 2024-09-16T08:57:22.568 INFO:tasks.workunit.client.1.smithi097.stdout:7/971: read d4/f6f [881355,109060] 0 2024-09-16T08:57:22.569 INFO:tasks.workunit.client.0.smithi031.stdout:0/518: dread d6/f69 [0,4194304] 0 2024-09-16T08:57:22.571 INFO:tasks.workunit.client.1.smithi097.stdout:7/972: creat d4/d7/d79/f145 x:0 0 0 2024-09-16T08:57:22.572 INFO:tasks.workunit.client.0.smithi031.stdout:8/465: dread d4/d7/f51 [0,4194304] 0 2024-09-16T08:57:22.599 INFO:tasks.workunit.client.0.smithi031.stdout:6/662: dwrite d0/dd/d74/f30 [0,4194304] 0 2024-09-16T08:57:22.599 INFO:tasks.workunit.client.0.smithi031.stdout:6/663: getdents d0/dd/d74/d5e/d7c/d56/dc5 0 2024-09-16T08:57:22.608 INFO:tasks.workunit.client.0.smithi031.stdout:7/643: dwrite d2/d8/d11/d66/d9a/fca [0,4194304] 0 2024-09-16T08:57:22.609 INFO:tasks.workunit.client.0.smithi031.stdout:7/644: truncate d2/d8/f2c 2167256 0 2024-09-16T08:57:22.610 INFO:tasks.workunit.client.0.smithi031.stdout:7/645: link d2/d8/f8c d2/d8/d11/d66/fd6 0 2024-09-16T08:57:22.612 INFO:tasks.workunit.client.1.smithi097.stdout:7/973: dwrite d4/d9/d2b/f4d [8388608,4194304] 0 2024-09-16T08:57:22.613 INFO:tasks.workunit.client.1.smithi097.stdout:7/974: rename d4/d9/d51/f106 to d4/d12/f146 0 2024-09-16T08:57:22.614 INFO:tasks.workunit.client.1.smithi097.stdout:7/975: creat d4/d7/d19/ddd/d134/daf/f147 x:0 0 0 2024-09-16T08:57:22.615 INFO:tasks.workunit.client.0.smithi031.stdout:7/646: dread d2/d8/d11/d66/d9a/fc9 [0,4194304] 0 2024-09-16T08:57:22.615 INFO:tasks.workunit.client.0.smithi031.stdout:7/647: truncate d2/d8/d11/d5d/d6e/f71 1626455 0 2024-09-16T08:57:22.616 INFO:tasks.workunit.client.0.smithi031.stdout:0/519: dwrite d6/f3b [0,4194304] 0 2024-09-16T08:57:22.616 INFO:tasks.workunit.client.0.smithi031.stdout:8/466: dwrite d4/d34/f6a [0,4194304] 0 2024-09-16T08:57:22.616 INFO:tasks.workunit.client.0.smithi031.stdout:0/520: creat d6/d12/d13/d36/fa6 x:0 0 0 2024-09-16T08:57:22.616 INFO:tasks.workunit.client.1.smithi097.stdout:7/976: symlink d4/d9/l148 0 2024-09-16T08:57:22.617 INFO:tasks.workunit.client.1.smithi097.stdout:7/977: getdents d4/d12/dec 0 2024-09-16T08:57:22.617 INFO:tasks.workunit.client.1.smithi097.stdout:7/978: creat d4/d9/d125/f149 x:0 0 0 2024-09-16T08:57:22.617 INFO:tasks.workunit.client.1.smithi097.stdout:7/979: chown d4/d9/d51/db7/df9 740 1 2024-09-16T08:57:22.617 INFO:tasks.workunit.client.0.smithi031.stdout:6/664: dwrite d0/dd/d74/d19/f85 [8388608,4194304] 0 2024-09-16T08:57:22.617 INFO:tasks.workunit.client.1.smithi097.stdout:4/982: sync 2024-09-16T08:57:22.620 INFO:tasks.workunit.client.0.smithi031.stdout:0/521: dread d6/d12/d13/f39 [0,4194304] 0 2024-09-16T08:57:22.621 INFO:tasks.workunit.client.1.smithi097.stdout:7/980: mknod d4/d12/dec/c14a 0 2024-09-16T08:57:22.625 INFO:tasks.workunit.client.1.smithi097.stdout:7/981: mkdir d4/dfa/d135/d14b 0 2024-09-16T08:57:22.629 INFO:tasks.workunit.client.0.smithi031.stdout:6/665: dread d0/dd/d74/d67/fbf [0,4194304] 0 2024-09-16T08:57:22.631 INFO:tasks.workunit.client.0.smithi031.stdout:6/666: mknod d0/dd/d74/d5e/d7c/d3f/d92/cd0 0 2024-09-16T08:57:22.631 INFO:tasks.workunit.client.0.smithi031.stdout:6/667: stat d0/dd/d74/d67/dbb 0 2024-09-16T08:57:22.631 INFO:tasks.workunit.client.0.smithi031.stdout:0/522: rmdir d6/d12/d47/d4d 39 2024-09-16T08:57:22.631 INFO:tasks.workunit.client.0.smithi031.stdout:0/523: chown d6/d12/d13/d36/f96 530242845 1 2024-09-16T08:57:22.631 INFO:tasks.workunit.client.0.smithi031.stdout:0/524: write d6/d12/d13/f21 [1053368,100958] 0 2024-09-16T08:57:22.633 INFO:tasks.workunit.client.0.smithi031.stdout:6/668: link d0/dd/d74/cb3 d0/dd/d74/d5e/d7c/d3f/cd1 0 2024-09-16T08:57:22.635 INFO:tasks.workunit.client.0.smithi031.stdout:6/669: unlink d0/dd/d74/f1e 0 2024-09-16T08:57:22.635 INFO:tasks.workunit.client.0.smithi031.stdout:0/525: write d6/d12/d13/f23 [89583,92709] 0 2024-09-16T08:57:22.642 INFO:tasks.workunit.client.0.smithi031.stdout:0/526: dread d6/d12/f22 [0,4194304] 0 2024-09-16T08:57:22.642 INFO:tasks.workunit.client.0.smithi031.stdout:0/527: truncate d6/f69 901398 0 2024-09-16T08:57:22.642 INFO:tasks.workunit.client.0.smithi031.stdout:0/528: chown d6/d61 607299983 1 2024-09-16T08:57:22.663 INFO:tasks.workunit.client.0.smithi031.stdout:6/670: write d0/f91 [3557788,40260] 0 2024-09-16T08:57:22.664 INFO:tasks.workunit.client.1.smithi097.stdout:4/983: dwrite d0/d36/d4c/f91 [0,4194304] 0 2024-09-16T08:57:22.665 INFO:tasks.workunit.client.0.smithi031.stdout:7/648: dwrite d2/d8/d86/da1/da9/fad [0,4194304] 0 2024-09-16T08:57:22.665 INFO:tasks.workunit.client.0.smithi031.stdout:7/649: fsync d2/d8/f87 0 2024-09-16T08:57:22.665 INFO:tasks.workunit.client.0.smithi031.stdout:8/467: dwrite d4/d5b/d7b/f7d [0,4194304] 0 2024-09-16T08:57:22.665 INFO:tasks.workunit.client.1.smithi097.stdout:7/982: dwrite d4/d7/d19/f144 [0,4194304] 0 2024-09-16T08:57:22.666 INFO:tasks.workunit.client.1.smithi097.stdout:4/984: truncate d0/d7/f126 787207 0 2024-09-16T08:57:22.672 INFO:tasks.workunit.client.1.smithi097.stdout:7/983: link d4/d7/d19/ddd/d134/lfe d4/d9/d2b/d11b/l14c 0 2024-09-16T08:57:22.674 INFO:tasks.workunit.client.0.smithi031.stdout:0/529: rmdir d6/d12 39 2024-09-16T08:57:22.679 INFO:tasks.workunit.client.0.smithi031.stdout:0/530: mknod d6/d12/d47/d5b/ca7 0 2024-09-16T08:57:22.681 INFO:tasks.workunit.client.0.smithi031.stdout:0/531: creat d6/d12/d47/d5b/d9b/fa8 x:0 0 0 2024-09-16T08:57:22.689 INFO:tasks.workunit.client.1.smithi097.stdout:4/985: dread d0/d7/df/d15/f11 [0,4194304] 0 2024-09-16T08:57:22.689 INFO:tasks.workunit.client.1.smithi097.stdout:4/986: truncate d0/d35/f129 665592 0 2024-09-16T08:57:22.690 INFO:tasks.workunit.client.1.smithi097.stdout:4/987: link d0/d23/c60 d0/d7/df/c14a 0 2024-09-16T08:57:22.691 INFO:tasks.workunit.client.1.smithi097.stdout:4/988: mknod d0/d7/df/d83/df8/c14b 0 2024-09-16T08:57:22.692 INFO:tasks.workunit.client.1.smithi097.stdout:4/989: symlink d0/d7/df/d83/l14c 0 2024-09-16T08:57:22.693 INFO:tasks.workunit.client.1.smithi097.stdout:4/990: readlink d0/d36/lf2 0 2024-09-16T08:57:22.693 INFO:tasks.workunit.client.1.smithi097.stdout:4/991: creat d0/d7/df/d83/df8/dbc/dcc/f14d x:0 0 0 2024-09-16T08:57:22.694 INFO:tasks.workunit.client.0.smithi031.stdout:3/597: sync 2024-09-16T08:57:22.694 INFO:tasks.workunit.client.0.smithi031.stdout:5/499: sync 2024-09-16T08:57:22.694 INFO:tasks.workunit.client.0.smithi031.stdout:9/541: sync 2024-09-16T08:57:22.694 INFO:tasks.workunit.client.0.smithi031.stdout:0/532: read d6/d12/d47/d5b/f71 [976075,52773] 0 2024-09-16T08:57:22.694 INFO:tasks.workunit.client.0.smithi031.stdout:0/533: fdatasync d6/d12/d13/d3c/f70 0 2024-09-16T08:57:22.694 INFO:tasks.workunit.client.0.smithi031.stdout:6/671: dwrite d0/f4 [4194304,4194304] 0 2024-09-16T08:57:22.695 INFO:tasks.workunit.client.1.smithi097.stdout:4/992: write d0/d35/f12e [617894,79078] 0 2024-09-16T08:57:22.695 INFO:tasks.workunit.client.0.smithi031.stdout:5/500: rmdir d2/dc/d8c 39 2024-09-16T08:57:22.696 INFO:tasks.workunit.client.0.smithi031.stdout:3/598: dread d3/d14/d37/d33/f4d [0,4194304] 0 2024-09-16T08:57:22.696 INFO:tasks.workunit.client.0.smithi031.stdout:3/599: creat d3/d14/d6f/faf x:0 0 0 2024-09-16T08:57:22.696 INFO:tasks.workunit.client.0.smithi031.stdout:9/542: unlink d3/d5/da/db/d21/l63 0 2024-09-16T08:57:22.696 INFO:tasks.workunit.client.0.smithi031.stdout:9/543: fsync d3/d5/d6a/f76 0 2024-09-16T08:57:22.698 INFO:tasks.workunit.client.1.smithi097.stdout:4/993: creat d0/d36/d4c/f14e x:0 0 0 2024-09-16T08:57:22.698 INFO:tasks.workunit.client.1.smithi097.stdout:4/994: truncate d0/d7/f126 928364 0 2024-09-16T08:57:22.698 INFO:tasks.workunit.client.1.smithi097.stdout:4/995: readlink d0/d23/d5f/lb1 0 2024-09-16T08:57:22.698 INFO:tasks.workunit.client.1.smithi097.stdout:4/996: write d0/d39/d119/f81 [339533,125991] 0 2024-09-16T08:57:22.699 INFO:tasks.workunit.client.0.smithi031.stdout:9/544: rename d3/d5/c6c to d3/d5/d6a/cbf 0 2024-09-16T08:57:22.700 INFO:tasks.workunit.client.1.smithi097.stdout:4/997: unlink d0/d39/l9b 0 2024-09-16T08:57:22.701 INFO:tasks.workunit.client.0.smithi031.stdout:0/534: link d6/f7 d6/d12/d13/d36/fa9 0 2024-09-16T08:57:22.702 INFO:tasks.workunit.client.0.smithi031.stdout:9/545: truncate d3/d5/da/db/fe 514063 0 2024-09-16T08:57:22.702 INFO:tasks.workunit.client.0.smithi031.stdout:9/546: stat d3/d5/da/d34/d3c/d5e/d62/d65/l97 0 2024-09-16T08:57:22.703 INFO:tasks.workunit.client.0.smithi031.stdout:9/547: stat d3/d5/da/d34/cb9 0 2024-09-16T08:57:22.703 INFO:tasks.workunit.client.0.smithi031.stdout:9/548: write d3/d5/da/d34/d3c/d5e/fb1 [74810,111941] 0 2024-09-16T08:57:22.715 INFO:tasks.workunit.client.0.smithi031.stdout:5/501: dread d2/dc/d25/d21/f26 [0,4194304] 0 2024-09-16T08:57:22.717 INFO:tasks.workunit.client.0.smithi031.stdout:5/502: mkdir d2/dc/d25/da2 0 2024-09-16T08:57:22.717 INFO:tasks.workunit.client.0.smithi031.stdout:5/503: creat d2/dc/d25/d21/fa3 x:0 0 0 2024-09-16T08:57:22.718 INFO:tasks.workunit.client.0.smithi031.stdout:6/672: dread d0/dd/f43 [0,4194304] 0 2024-09-16T08:57:22.719 INFO:tasks.workunit.client.0.smithi031.stdout:6/673: mkdir d0/d10/dd2 0 2024-09-16T08:57:22.720 INFO:tasks.workunit.client.0.smithi031.stdout:6/674: symlink d0/dd/d35/d6c/dac/dba/ld3 0 2024-09-16T08:57:22.720 INFO:tasks.workunit.client.0.smithi031.stdout:6/675: rmdir d0/dd/d74/d5e/d7c 39 2024-09-16T08:57:22.726 INFO:tasks.workunit.client.0.smithi031.stdout:7/650: dwrite d2/fab [0,4194304] 0 2024-09-16T08:57:22.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/468: dwrite d4/d9e/d38/f73 [0,4194304] 0 2024-09-16T08:57:22.727 INFO:tasks.workunit.client.1.smithi097.stdout:7/984: dwrite d4/d7/d19/ddd/d134/daf/ffd [0,4194304] 0 2024-09-16T08:57:22.729 INFO:tasks.workunit.client.0.smithi031.stdout:0/535: dwrite d6/d61/d91/f9f [0,4194304] 0 2024-09-16T08:57:22.729 INFO:tasks.workunit.client.0.smithi031.stdout:0/536: write d6/d12/d13/d36/f51 [4397064,66662] 0 2024-09-16T08:57:22.729 INFO:tasks.workunit.client.0.smithi031.stdout:0/537: stat d6/d12/d47/d5b/f67 0 2024-09-16T08:57:22.729 INFO:tasks.workunit.client.0.smithi031.stdout:7/651: rmdir d2/d8/d86/da1 39 2024-09-16T08:57:22.731 INFO:tasks.workunit.client.1.smithi097.stdout:4/998: dwrite d0/d7/df/d83/f98 [0,4194304] 0 2024-09-16T08:57:22.731 INFO:tasks.workunit.client.1.smithi097.stdout:4/999: creat d0/d35/da4/dc5/f14f x:0 0 0 2024-09-16T08:57:22.732 INFO:tasks.workunit.client.0.smithi031.stdout:6/676: write d0/dd/d74/f18 [11351403,105039] 0 2024-09-16T08:57:22.733 INFO:tasks.workunit.client.1.smithi097.stdout:7/985: dread d4/d9/d1c/f2c [0,4194304] 0 2024-09-16T08:57:22.736 INFO:tasks.workunit.client.0.smithi031.stdout:8/469: dread d4/d7/d21/f3b [0,4194304] 0 2024-09-16T08:57:22.738 INFO:tasks.workunit.client.0.smithi031.stdout:3/600: write d3/d14/d6f/f72 [679735,70751] 0 2024-09-16T08:57:22.739 INFO:tasks.workunit.client.1.smithi097.stdout:7/986: creat d4/d7/d77/da8/d124/f14d x:0 0 0 2024-09-16T08:57:22.740 INFO:tasks.workunit.client.1.smithi097.stdout:7/987: write d4/d7/d7b/f8f [4632207,67523] 0 2024-09-16T08:57:22.741 INFO:tasks.workunit.client.0.smithi031.stdout:0/538: mknod d6/d12/d47/d7c/caa 0 2024-09-16T08:57:22.742 INFO:tasks.workunit.client.0.smithi031.stdout:7/652: mknod d2/d8/d11/d66/cd7 0 2024-09-16T08:57:22.743 INFO:tasks.workunit.client.1.smithi097.stdout:7/988: mknod d4/d1b/da4/c14e 0 2024-09-16T08:57:22.744 INFO:tasks.workunit.client.0.smithi031.stdout:6/677: rename d0/dd/c50 to d0/d8e/cd4 0 2024-09-16T08:57:22.744 INFO:tasks.workunit.client.0.smithi031.stdout:6/678: chown d0/dd/d74/d5e/d7c/d3f/d92/fc6 516491 1 2024-09-16T08:57:22.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/470: symlink d4/d7/d21/l9f 0 2024-09-16T08:57:22.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/471: chown d4/d2d/d6e 1323597 1 2024-09-16T08:57:22.746 INFO:tasks.workunit.client.0.smithi031.stdout:8/472: read - d4/d77/f8f zero size 2024-09-16T08:57:22.746 INFO:tasks.workunit.client.1.smithi097.stdout:7/989: mkdir d4/d7/d19/ddd/d134/daf/d14f 0 2024-09-16T08:57:22.748 INFO:tasks.workunit.client.0.smithi031.stdout:0/539: creat d6/d12/d47/d5b/d9b/da2/fab x:0 0 0 2024-09-16T08:57:22.748 INFO:tasks.workunit.client.0.smithi031.stdout:9/549: dread d3/fc [0,4194304] 0 2024-09-16T08:57:22.748 INFO:tasks.workunit.client.0.smithi031.stdout:0/540: dread d6/d12/d47/f7e [0,4194304] 0 2024-09-16T08:57:22.749 INFO:tasks.workunit.client.0.smithi031.stdout:7/653: creat d2/d2d/fd8 x:0 0 0 2024-09-16T08:57:22.749 INFO:tasks.workunit.client.0.smithi031.stdout:7/654: truncate d2/d8/d11/fa4 741160 0 2024-09-16T08:57:22.751 INFO:tasks.workunit.client.1.smithi097.stdout:7/990: rename d4/d9/d51/l97 to d4/d9/l150 0 2024-09-16T08:57:22.753 INFO:tasks.workunit.client.0.smithi031.stdout:3/601: rename d3/d14/d37/d33/f95 to d3/d14/d37/fb0 0 2024-09-16T08:57:22.756 INFO:tasks.workunit.client.0.smithi031.stdout:3/602: dread d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 [0,4194304] 0 2024-09-16T08:57:22.756 INFO:tasks.workunit.client.0.smithi031.stdout:6/679: creat d0/dd/d74/d19/dcd/fd5 x:0 0 0 2024-09-16T08:57:22.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/473: creat d4/d34/fa0 x:0 0 0 2024-09-16T08:57:22.759 INFO:tasks.workunit.client.0.smithi031.stdout:9/550: creat d3/d5/da/db/fc0 x:0 0 0 2024-09-16T08:57:22.760 INFO:tasks.workunit.client.0.smithi031.stdout:4/571: sync 2024-09-16T08:57:22.761 INFO:tasks.workunit.client.0.smithi031.stdout:7/655: mknod d2/d8/d69/d82/cd9 0 2024-09-16T08:57:22.762 INFO:tasks.workunit.client.0.smithi031.stdout:7/656: creat d2/d8/d86/da1/da9/fda x:0 0 0 2024-09-16T08:57:22.763 INFO:tasks.workunit.client.0.smithi031.stdout:7/657: read d2/d8/d11/d81/f9f [126978,117285] 0 2024-09-16T08:57:22.765 INFO:tasks.workunit.client.0.smithi031.stdout:3/603: mkdir d3/d14/d37/d1a/d1d/d3b/d60/db1 0 2024-09-16T08:57:22.765 INFO:tasks.workunit.client.0.smithi031.stdout:8/474: creat d4/d2d/d97/fa1 x:0 0 0 2024-09-16T08:57:22.766 INFO:tasks.workunit.client.0.smithi031.stdout:4/572: dread - d0/d2/d3a/fac zero size 2024-09-16T08:57:22.766 INFO:tasks.workunit.client.0.smithi031.stdout:4/573: write d0/d2/d16/d1e/fa1 [308114,84084] 0 2024-09-16T08:57:22.767 INFO:tasks.workunit.client.0.smithi031.stdout:0/541: mkdir d6/dac 0 2024-09-16T08:57:22.768 INFO:tasks.workunit.client.0.smithi031.stdout:3/604: dread d3/d14/d37/d1a/d1d/d1e/d67/f6b [0,4194304] 0 2024-09-16T08:57:22.768 INFO:tasks.workunit.client.0.smithi031.stdout:3/605: creat d3/d14/d37/d1a/d1d/d3b/d60/fb2 x:0 0 0 2024-09-16T08:57:22.769 INFO:tasks.workunit.client.0.smithi031.stdout:7/658: symlink d2/d8/d11/d5d/d7e/db6/ldb 0 2024-09-16T08:57:22.770 INFO:tasks.workunit.client.0.smithi031.stdout:9/551: getdents d3/d5/db3 0 2024-09-16T08:57:22.770 INFO:tasks.workunit.client.0.smithi031.stdout:9/552: creat d3/d5/da/db/d21/fc1 x:0 0 0 2024-09-16T08:57:22.772 INFO:tasks.workunit.client.0.smithi031.stdout:4/574: rename d0/d2/c6 to d0/d2/d16/cba 0 2024-09-16T08:57:22.774 INFO:tasks.workunit.client.0.smithi031.stdout:3/606: mknod d3/d14/d37/d33/d97/cb3 0 2024-09-16T08:57:22.775 INFO:tasks.workunit.client.0.smithi031.stdout:9/553: write d3/d5/da/f2c [149747,5880] 0 2024-09-16T08:57:22.775 INFO:tasks.workunit.client.0.smithi031.stdout:3/607: read d3/d14/d37/d1a/d1d/f92 [110266,108930] 0 2024-09-16T08:57:22.776 INFO:tasks.workunit.client.0.smithi031.stdout:3/608: chown d3/d14/d37/d47/c8d 5881 1 2024-09-16T08:57:22.776 INFO:tasks.workunit.client.0.smithi031.stdout:3/609: dread - d3/d14/d37/d1a/f9b zero size 2024-09-16T08:57:22.777 INFO:tasks.workunit.client.0.smithi031.stdout:9/554: link d3/l4a d3/d5/d73/lc2 0 2024-09-16T08:57:22.778 INFO:tasks.workunit.client.0.smithi031.stdout:8/475: read d4/d9e/d38/f73 [2246897,115679] 0 2024-09-16T08:57:22.785 INFO:tasks.workunit.client.0.smithi031.stdout:4/575: dread d0/d2/d16/d1e/f9e [0,4194304] 0 2024-09-16T08:57:22.785 INFO:tasks.workunit.client.0.smithi031.stdout:4/576: fdatasync d0/d2/d3a/d55/f64 0 2024-09-16T08:57:22.787 INFO:tasks.workunit.client.0.smithi031.stdout:9/555: dread d3/d5/d6a/d74/da8/fb2 [0,4194304] 0 2024-09-16T08:57:22.789 INFO:tasks.workunit.client.1.smithi097.stdout:7/991: dwrite d4/d9/dc/ff7 [4194304,4194304] 0 2024-09-16T08:57:22.790 INFO:tasks.workunit.client.1.smithi097.stdout:7/992: fdatasync d4/d7/f32 0 2024-09-16T08:57:22.791 INFO:tasks.workunit.client.0.smithi031.stdout:0/542: write d6/fa [2662138,100655] 0 2024-09-16T08:57:22.791 INFO:tasks.workunit.client.0.smithi031.stdout:9/556: rename d3/d5/da/db/d21/d67 to d3/d5/d6a/d74/da8/dc3 0 2024-09-16T08:57:22.791 INFO:tasks.workunit.client.0.smithi031.stdout:9/557: stat d3/d5/da/d34/f5c 0 2024-09-16T08:57:22.794 INFO:tasks.workunit.client.0.smithi031.stdout:9/558: dread d3/d5/da/f6b [4194304,4194304] 0 2024-09-16T08:57:22.794 INFO:tasks.workunit.client.0.smithi031.stdout:9/559: getdents d3/d5/db3 0 2024-09-16T08:57:22.794 INFO:tasks.workunit.client.0.smithi031.stdout:6/680: dwrite d0/dd/d74/d19/f85 [4194304,4194304] 0 2024-09-16T08:57:22.794 INFO:tasks.workunit.client.0.smithi031.stdout:3/610: dread d3/d14/d37/d1a/d1d/d3b/f4f [0,4194304] 0 2024-09-16T08:57:22.794 INFO:tasks.workunit.client.0.smithi031.stdout:6/681: truncate d0/dd/d35/d6c/d83/f89 698610 0 2024-09-16T08:57:22.794 INFO:tasks.workunit.client.0.smithi031.stdout:3/611: write d3/d14/d6f/faf [29886,10993] 0 2024-09-16T08:57:22.794 INFO:tasks.workunit.client.1.smithi097.stdout:7/993: link d4/d9/d51/db7/fd4 d4/dfa/d135/d14b/f151 0 2024-09-16T08:57:22.795 INFO:tasks.workunit.client.1.smithi097.stdout:7/994: creat d4/d7/d19/ddd/d134/daf/f152 x:0 0 0 2024-09-16T08:57:22.795 INFO:tasks.workunit.client.0.smithi031.stdout:0/543: mkdir d6/d12/d13/dad 0 2024-09-16T08:57:22.795 INFO:tasks.workunit.client.0.smithi031.stdout:1/511: sync 2024-09-16T08:57:22.796 INFO:tasks.workunit.client.0.smithi031.stdout:2/568: sync 2024-09-16T08:57:22.796 INFO:tasks.workunit.client.0.smithi031.stdout:2/569: fsync d6/da/d5e/d72/fb4 0 2024-09-16T08:57:22.796 INFO:tasks.workunit.client.0.smithi031.stdout:8/476: dread d4/d7/d21/d37/d46/f75 [0,4194304] 0 2024-09-16T08:57:22.797 INFO:tasks.workunit.client.0.smithi031.stdout:4/577: read d0/d2/d3a/f49 [1861393,111049] 0 2024-09-16T08:57:22.797 INFO:tasks.workunit.client.0.smithi031.stdout:4/578: truncate d0/d5b/f91 227319 0 2024-09-16T08:57:22.800 INFO:tasks.workunit.client.0.smithi031.stdout:6/682: creat d0/dd/d74/d19/d54/fd6 x:0 0 0 2024-09-16T08:57:22.803 INFO:tasks.workunit.client.0.smithi031.stdout:3/612: symlink d3/d14/d85/lb4 0 2024-09-16T08:57:22.807 INFO:tasks.workunit.client.0.smithi031.stdout:0/544: creat d6/d12/d47/d4d/fae x:0 0 0 2024-09-16T08:57:22.807 INFO:tasks.workunit.client.0.smithi031.stdout:1/512: mkdir d2/dae/db9 0 2024-09-16T08:57:22.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/570: mkdir d6/d2e/d40/dc3 0 2024-09-16T08:57:22.808 INFO:tasks.workunit.client.0.smithi031.stdout:4/579: dread d0/f38 [0,4194304] 0 2024-09-16T08:57:22.808 INFO:tasks.workunit.client.0.smithi031.stdout:8/477: dread d4/d5b/d7b/f8e [0,4194304] 0 2024-09-16T08:57:22.812 INFO:tasks.workunit.client.0.smithi031.stdout:3/613: creat d3/d14/d37/d33/d97/fb5 x:0 0 0 2024-09-16T08:57:22.812 INFO:tasks.workunit.client.0.smithi031.stdout:3/614: chown d3/l19 33198376 1 2024-09-16T08:57:22.812 INFO:tasks.workunit.client.0.smithi031.stdout:1/513: mknod d2/d4/d36/cba 0 2024-09-16T08:57:22.815 INFO:tasks.workunit.client.0.smithi031.stdout:8/478: mknod d4/d7/d21/d35/ca2 0 2024-09-16T08:57:22.818 INFO:tasks.workunit.client.0.smithi031.stdout:3/615: rmdir d3/d14/d91 0 2024-09-16T08:57:22.818 INFO:tasks.workunit.client.0.smithi031.stdout:3/616: write d3/d14/d37/fb0 [800144,3397] 0 2024-09-16T08:57:22.820 INFO:tasks.workunit.client.0.smithi031.stdout:0/545: read d6/d61/d91/f9f [1970746,11742] 0 2024-09-16T08:57:22.820 INFO:tasks.workunit.client.0.smithi031.stdout:9/560: dwrite d3/d5/da/d34/d3c/d5e/d62/d65/f93 [0,4194304] 0 2024-09-16T08:57:22.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/479: dread d4/f43 [0,4194304] 0 2024-09-16T08:57:22.821 INFO:tasks.workunit.client.0.smithi031.stdout:4/580: mknod d0/d2/d3a/cbb 0 2024-09-16T08:57:22.821 INFO:tasks.workunit.client.0.smithi031.stdout:0/546: dread d6/d12/d47/f7e [0,4194304] 0 2024-09-16T08:57:22.822 INFO:tasks.workunit.client.0.smithi031.stdout:0/547: fdatasync d6/d12/d13/d3c/f70 0 2024-09-16T08:57:22.822 INFO:tasks.workunit.client.0.smithi031.stdout:0/548: chown d6/d12/d13/f39 0 1 2024-09-16T08:57:22.822 INFO:tasks.workunit.client.0.smithi031.stdout:0/549: truncate d6/d12/d13/f3e 1598958 0 2024-09-16T08:57:22.822 INFO:tasks.workunit.client.0.smithi031.stdout:9/561: rmdir d3/d5 39 2024-09-16T08:57:22.824 INFO:tasks.workunit.client.0.smithi031.stdout:2/571: write d6/da/d4b/d4c/d68/fa8 [3613690,34483] 0 2024-09-16T08:57:22.824 INFO:tasks.workunit.client.0.smithi031.stdout:8/480: truncate d4/f8 11981158 0 2024-09-16T08:57:22.824 INFO:tasks.workunit.client.0.smithi031.stdout:8/481: write d4/d7/f51 [3557816,56583] 0 2024-09-16T08:57:22.824 INFO:tasks.workunit.client.0.smithi031.stdout:8/482: truncate d4/d77/f8f 679192 0 2024-09-16T08:57:22.824 INFO:tasks.workunit.client.0.smithi031.stdout:1/514: read d2/d4/f28 [1386877,112409] 0 2024-09-16T08:57:22.824 INFO:tasks.workunit.client.0.smithi031.stdout:1/515: fdatasync d2/de/f13 0 2024-09-16T08:57:22.825 INFO:tasks.workunit.client.1.smithi097.stdout:7/995: dwrite d4/d9/d1c/d5f/f6c [0,4194304] 0 2024-09-16T08:57:22.826 INFO:tasks.workunit.client.1.smithi097.stdout:7/996: read d4/dfa/f13b [462629,120276] 0 2024-09-16T08:57:22.826 INFO:tasks.workunit.client.1.smithi097.stdout:7/997: fsync d4/d7/fdc 0 2024-09-16T08:57:22.826 INFO:tasks.workunit.client.1.smithi097.stdout:7/998: stat d4/d1b/da4/f12c 0 2024-09-16T08:57:22.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/572: rename d6/da/f3d to d6/d2e/d40/d58/dae/db1/fc4 0 2024-09-16T08:57:22.829 INFO:tasks.workunit.client.0.smithi031.stdout:1/516: mknod d2/d15/d4a/cbb 0 2024-09-16T08:57:22.831 INFO:tasks.workunit.client.0.smithi031.stdout:0/550: mknod d6/caf 0 2024-09-16T08:57:22.831 INFO:tasks.workunit.client.0.smithi031.stdout:0/551: dread - d6/d12/d13/d36/fa6 zero size 2024-09-16T08:57:22.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/573: unlink d6/d2e/d40/d6c/f2b 0 2024-09-16T08:57:22.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/574: fsync d6/f2d 0 2024-09-16T08:57:22.834 INFO:tasks.workunit.client.0.smithi031.stdout:4/581: dread d0/d5b/f69 [0,4194304] 0 2024-09-16T08:57:22.836 INFO:tasks.workunit.client.1.smithi097.stdout:7/999: symlink d4/d9/d125/l153 0 2024-09-16T08:57:22.838 INFO:tasks.workunit.client.0.smithi031.stdout:1/517: truncate d2/d4/fa2 2266671 0 2024-09-16T08:57:22.842 INFO:tasks.workunit.client.0.smithi031.stdout:2/575: creat d6/d2e/d40/fc5 x:0 0 0 2024-09-16T08:57:22.842 INFO:tasks.workunit.client.0.smithi031.stdout:2/576: write d6/f74 [159740,116330] 0 2024-09-16T08:57:22.842 INFO:tasks.workunit.client.0.smithi031.stdout:3/617: dwrite d3/d14/d37/d1a/f9b [0,4194304] 0 2024-09-16T08:57:22.843 INFO:tasks.workunit.client.1.smithi097.stderr:+ rm -rf -- ./tmp.wACgg2kGuI 2024-09-16T08:57:22.846 INFO:tasks.workunit.client.0.smithi031.stdout:1/518: unlink d2/d4/l16 0 2024-09-16T08:57:22.846 INFO:tasks.workunit.client.0.smithi031.stdout:1/519: readlink d2/d4/l11 0 2024-09-16T08:57:22.846 INFO:tasks.workunit.client.0.smithi031.stdout:1/520: stat d2/d4/f12 0 2024-09-16T08:57:22.846 INFO:tasks.workunit.client.0.smithi031.stdout:1/521: creat d2/d4/d67/fbc x:0 0 0 2024-09-16T08:57:22.848 INFO:tasks.workunit.client.0.smithi031.stdout:0/552: unlink d6/f26 0 2024-09-16T08:57:22.848 INFO:tasks.workunit.client.0.smithi031.stdout:0/553: dread - d6/d12/d13/d36/fa6 zero size 2024-09-16T08:57:22.850 INFO:tasks.workunit.client.0.smithi031.stdout:8/483: dwrite d4/d7/d66/f70 [4194304,4194304] 0 2024-09-16T08:57:22.859 INFO:tasks.workunit.client.0.smithi031.stdout:0/554: truncate d6/d12/f17 255312 0 2024-09-16T08:57:22.859 INFO:tasks.workunit.client.0.smithi031.stdout:0/555: write d6/d12/d13/f23 [778222,48255] 0 2024-09-16T08:57:22.862 INFO:tasks.workunit.client.0.smithi031.stdout:8/484: mkdir d4/d7/d21/d35/d50/da3 0 2024-09-16T08:57:22.869 INFO:tasks.workunit.client.0.smithi031.stdout:0/556: rmdir d6/d12/d13/dad 0 2024-09-16T08:57:22.870 INFO:tasks.workunit.client.0.smithi031.stdout:4/582: dwrite d0/d2/f90 [0,4194304] 0 2024-09-16T08:57:22.872 INFO:tasks.workunit.client.0.smithi031.stdout:0/557: getdents d6/d12/d13/d36 0 2024-09-16T08:57:22.873 INFO:tasks.workunit.client.0.smithi031.stdout:3/618: dwrite d3/d14/d37/d1a/f76 [0,4194304] 0 2024-09-16T08:57:22.874 INFO:tasks.workunit.client.0.smithi031.stdout:0/558: read d6/f69 [434946,110605] 0 2024-09-16T08:57:22.876 INFO:tasks.workunit.client.0.smithi031.stdout:2/577: dwrite d6/da/f85 [0,4194304] 0 2024-09-16T08:57:22.876 INFO:tasks.workunit.client.0.smithi031.stdout:2/578: chown d6/d2e/d40/f78 3 1 2024-09-16T08:57:22.877 INFO:tasks.workunit.client.0.smithi031.stdout:4/583: truncate d0/d2/d3a/f93 851915 0 2024-09-16T08:57:22.877 INFO:tasks.workunit.client.0.smithi031.stdout:4/584: write d0/f4c [3373361,54024] 0 2024-09-16T08:57:22.877 INFO:tasks.workunit.client.0.smithi031.stdout:4/585: chown d0/l45 12784044 1 2024-09-16T08:57:22.881 INFO:tasks.workunit.client.0.smithi031.stdout:3/619: getdents d3/d14/d37/d1a/d1d/d1e/d67 0 2024-09-16T08:57:22.883 INFO:tasks.workunit.client.0.smithi031.stdout:1/522: dwrite d2/f14 [4194304,4194304] 0 2024-09-16T08:57:22.884 INFO:tasks.workunit.client.0.smithi031.stdout:0/559: rename d6/d12/d47/d7c/la4 to d6/d61/lb0 0 2024-09-16T08:57:22.890 INFO:tasks.workunit.client.0.smithi031.stdout:4/586: mknod d0/db5/cbc 0 2024-09-16T08:57:22.890 INFO:tasks.workunit.client.0.smithi031.stdout:8/485: dwrite d4/d5b/d7b/f7d [0,4194304] 0 2024-09-16T08:57:22.890 INFO:tasks.workunit.client.0.smithi031.stdout:8/486: creat d4/d2d/d6e/fa4 x:0 0 0 2024-09-16T08:57:22.891 INFO:tasks.workunit.client.0.smithi031.stdout:3/620: creat d3/d14/fb6 x:0 0 0 2024-09-16T08:57:22.897 INFO:tasks.workunit.client.0.smithi031.stdout:0/560: symlink d6/d12/d13/da5/lb1 0 2024-09-16T08:57:22.908 INFO:tasks.workunit.client.0.smithi031.stdout:3/621: dwrite d3/d14/d37/fb0 [0,4194304] 0 2024-09-16T08:57:22.911 INFO:tasks.workunit.client.0.smithi031.stdout:8/487: symlink d4/d7/d99/la5 0 2024-09-16T08:57:22.911 INFO:tasks.workunit.client.0.smithi031.stdout:8/488: truncate d4/d7/d21/d37/f76 5241678 0 2024-09-16T08:57:22.914 INFO:tasks.workunit.client.0.smithi031.stdout:1/523: getdents d2/d15/d5d 0 2024-09-16T08:57:22.918 INFO:tasks.workunit.client.0.smithi031.stdout:0/561: rename d6/d61/f6b to d6/d12/d47/d5b/fb2 0 2024-09-16T08:57:22.919 INFO:tasks.workunit.client.0.smithi031.stdout:4/587: getdents d0/d8e 0 2024-09-16T08:57:22.919 INFO:tasks.workunit.client.0.smithi031.stdout:4/588: stat d0/d2 0 2024-09-16T08:57:22.919 INFO:tasks.workunit.client.0.smithi031.stdout:4/589: dread - d0/d71/f81 zero size 2024-09-16T08:57:22.921 INFO:tasks.workunit.client.0.smithi031.stdout:0/562: dread d6/d12/d13/f46 [0,4194304] 0 2024-09-16T08:57:22.925 INFO:tasks.workunit.client.0.smithi031.stdout:3/622: truncate d3/d14/d37/d1a/d1d/d3b/d60/f7e 3161756 0 2024-09-16T08:57:22.925 INFO:tasks.workunit.client.0.smithi031.stdout:3/623: write d3/d14/d37/d1a/d1d/d3b/d60/f9c [74781,34682] 0 2024-09-16T08:57:22.925 INFO:tasks.workunit.client.0.smithi031.stdout:8/489: dwrite d4/d9e/d38/f9c [0,4194304] 0 2024-09-16T08:57:22.925 INFO:tasks.workunit.client.0.smithi031.stdout:3/624: write d3/d14/d37/d1a/d1d/d3b/d3f/f52 [1866697,108088] 0 2024-09-16T08:57:22.944 INFO:tasks.workunit.client.0.smithi031.stdout:1/524: dwrite d2/d4/d52/f74 [0,4194304] 0 2024-09-16T08:57:23.022 INFO:tasks.workunit.client.0.smithi031.stdout:4/590: mknod d0/d2/d16/d59/d84/d86/cbd 0 2024-09-16T08:57:23.024 INFO:tasks.workunit.client.0.smithi031.stdout:0/563: mkdir d6/d12/d97/db3 0 2024-09-16T08:57:23.044 INFO:tasks.workunit.client.0.smithi031.stdout:1/525: chown d2/d4/d1b/d5c 14445 1 2024-09-16T08:57:23.044 INFO:tasks.workunit.client.0.smithi031.stdout:1/526: chown d2/d15/d32/d87/l8f 3 1 2024-09-16T08:57:23.045 INFO:tasks.workunit.client.0.smithi031.stdout:4/591: getdents d0/d2/d16/d59/d84/d86/d9b 0 2024-09-16T08:57:23.045 INFO:tasks.workunit.client.0.smithi031.stdout:4/592: readlink d0/d3c/d6b/l7a 0 2024-09-16T08:57:23.046 INFO:tasks.workunit.client.0.smithi031.stdout:0/564: mknod d6/d12/d47/d5b/d9b/cb4 0 2024-09-16T08:57:23.046 INFO:tasks.workunit.client.0.smithi031.stdout:3/625: rmdir d3/d14/d6f/d7d 39 2024-09-16T08:57:23.057 INFO:tasks.workunit.client.0.smithi031.stdout:0/565: write d6/f7 [2035208,88978] 0 2024-09-16T08:57:23.057 INFO:tasks.workunit.client.0.smithi031.stdout:0/566: getdents d6/dac 0 2024-09-16T08:57:23.058 INFO:tasks.workunit.client.0.smithi031.stdout:0/567: creat d6/d12/d97/db3/fb5 x:0 0 0 2024-09-16T08:57:23.059 INFO:tasks.workunit.client.0.smithi031.stdout:0/568: mknod d6/d12/d97/cb6 0 2024-09-16T08:57:23.080 INFO:tasks.workunit.client.0.smithi031.stdout:4/593: dwrite d0/d2/d16/d1e/f9e [0,4194304] 0 2024-09-16T08:57:23.080 INFO:tasks.workunit.client.0.smithi031.stdout:3/626: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/f74 [0,4194304] 0 2024-09-16T08:57:23.080 INFO:tasks.workunit.client.0.smithi031.stdout:3/627: creat d3/d14/d6f/fb7 x:0 0 0 2024-09-16T08:57:23.089 INFO:tasks.workunit.client.0.smithi031.stdout:3/628: mkdir d3/d14/d37/d1a/d1d/d1e/db8 0 2024-09-16T08:57:23.092 INFO:tasks.workunit.client.0.smithi031.stdout:3/629: symlink d3/d14/d6f/lb9 0 2024-09-16T08:57:23.105 INFO:tasks.workunit.client.0.smithi031.stdout:3/630: dread d3/d14/f8c [0,4194304] 0 2024-09-16T08:57:23.105 INFO:tasks.workunit.client.0.smithi031.stdout:4/594: dwrite d0/d3c/f62 [0,4194304] 0 2024-09-16T08:57:23.108 INFO:tasks.workunit.client.0.smithi031.stdout:4/595: mkdir d0/d2/d16/d1e/dbe 0 2024-09-16T08:57:23.114 INFO:tasks.workunit.client.0.smithi031.stdout:3/631: write d3/d14/d37/fb0 [912558,121192] 0 2024-09-16T08:57:23.114 INFO:tasks.workunit.client.0.smithi031.stdout:3/632: stat d3/d14/c55 0 2024-09-16T08:57:23.114 INFO:tasks.workunit.client.0.smithi031.stdout:3/633: creat d3/d14/d6f/fba x:0 0 0 2024-09-16T08:57:23.114 INFO:tasks.workunit.client.0.smithi031.stdout:3/634: write d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 [419800,53035] 0 2024-09-16T08:57:23.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:22 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi097 to 2240M 2024-09-16T08:57:23.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:22 smithi031 ceph-mon[27844]: pgmap v13: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 228 MiB/s rd, 291 MiB/s wr, 403 op/s 2024-09-16T08:57:23.125 INFO:tasks.workunit.client.0.smithi031.stdout:4/596: dwrite d0/d2/d3a/fac [0,4194304] 0 2024-09-16T08:57:23.126 INFO:tasks.workunit.client.0.smithi031.stdout:5/504: sync 2024-09-16T08:57:23.126 INFO:tasks.workunit.client.0.smithi031.stdout:5/505: write d2/dc/d25/d21/f85 [599983,33187] 0 2024-09-16T08:57:23.126 INFO:tasks.workunit.client.0.smithi031.stdout:5/506: chown d2/dc/d3d/f86 565 1 2024-09-16T08:57:23.131 INFO:tasks.workunit.client.0.smithi031.stdout:5/507: dread d2/dc/d25/d21/f45 [0,4194304] 0 2024-09-16T08:57:23.131 INFO:tasks.workunit.client.0.smithi031.stdout:5/508: chown d2/dc/d36/d76 3 1 2024-09-16T08:57:23.133 INFO:tasks.workunit.client.0.smithi031.stdout:5/509: creat d2/dc/d25/d21/d81/fa4 x:0 0 0 2024-09-16T08:57:23.133 INFO:tasks.workunit.client.0.smithi031.stdout:5/510: truncate d2/dc/d25/f53 95746 0 2024-09-16T08:57:23.133 INFO:tasks.workunit.client.0.smithi031.stdout:5/511: creat d2/dc/d25/d21/d2e/fa5 x:0 0 0 2024-09-16T08:57:23.135 INFO:tasks.workunit.client.0.smithi031.stdout:5/512: symlink d2/dc/d36/la6 0 2024-09-16T08:57:23.135 INFO:tasks.workunit.client.0.smithi031.stdout:4/597: write d0/d3c/f62 [172182,44246] 0 2024-09-16T08:57:23.136 INFO:tasks.workunit.client.0.smithi031.stdout:5/513: mknod d2/dc/d3d/ca7 0 2024-09-16T08:57:23.136 INFO:tasks.workunit.client.0.smithi031.stdout:5/514: chown d2/d4b 0 1 2024-09-16T08:57:23.136 INFO:tasks.workunit.client.0.smithi031.stdout:5/515: creat d2/dc/d25/d21/d2e/fa8 x:0 0 0 2024-09-16T08:57:23.137 INFO:tasks.workunit.client.0.smithi031.stdout:4/598: dread - d0/d2/d3a/fb4 zero size 2024-09-16T08:57:23.137 INFO:tasks.workunit.client.0.smithi031.stdout:4/599: dread - d0/d2/d16/d59/d84/d86/d9b/fa4 zero size 2024-09-16T08:57:23.139 INFO:tasks.workunit.client.0.smithi031.stdout:5/516: mknod d2/dc/d91/ca9 0 2024-09-16T08:57:23.139 INFO:tasks.workunit.client.0.smithi031.stdout:5/517: dread - d2/dc/d3d/d4c/f62 zero size 2024-09-16T08:57:23.139 INFO:tasks.workunit.client.0.smithi031.stdout:5/518: write d2/f34 [749334,99447] 0 2024-09-16T08:57:23.139 INFO:tasks.workunit.client.0.smithi031.stdout:4/600: symlink d0/d2/d16/d1e/dbe/lbf 0 2024-09-16T08:57:23.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/519: truncate d2/f1a 1715824 0 2024-09-16T08:57:23.145 INFO:tasks.workunit.client.0.smithi031.stdout:5/520: link d2/dc/d1d/l7b d2/dc/laa 0 2024-09-16T08:57:23.145 INFO:tasks.workunit.client.0.smithi031.stdout:5/521: write d2/dc/d25/d21/d2e/f71 [1613217,122432] 0 2024-09-16T08:57:23.145 INFO:tasks.workunit.client.0.smithi031.stdout:5/522: readlink d2/dc/l67 0 2024-09-16T08:57:23.156 INFO:tasks.workunit.client.0.smithi031.stdout:4/601: dwrite d0/d2/f90 [0,4194304] 0 2024-09-16T08:57:23.159 INFO:tasks.workunit.client.0.smithi031.stdout:5/523: dread d2/dc/d25/d21/d2e/f5c [0,4194304] 0 2024-09-16T08:57:23.159 INFO:tasks.workunit.client.0.smithi031.stdout:5/524: write d2/dc/d25/d21/f6d [4608879,115517] 0 2024-09-16T08:57:23.176 INFO:tasks.workunit.client.0.smithi031.stdout:4/602: dwrite d0/d2/d16/d59/d84/d86/f8f [0,4194304] 0 2024-09-16T08:57:23.180 INFO:tasks.workunit.client.0.smithi031.stdout:5/525: dwrite d2/dc/d25/f2f [0,4194304] 0 2024-09-16T08:57:23.180 INFO:tasks.workunit.client.0.smithi031.stdout:5/526: read - d2/dc/d25/d21/d81/fa4 zero size 2024-09-16T08:57:23.180 INFO:tasks.workunit.client.0.smithi031.stdout:4/603: getdents d0 0 2024-09-16T08:57:23.189 INFO:tasks.workunit.client.0.smithi031.stdout:5/527: write d2/dc/d25/d68/f6b [217172,76727] 0 2024-09-16T08:57:23.195 INFO:tasks.workunit.client.0.smithi031.stdout:5/528: creat d2/dc/d25/d21/fab x:0 0 0 2024-09-16T08:57:23.202 INFO:tasks.workunit.client.0.smithi031.stdout:5/529: dread d2/dc/f3b [0,4194304] 0 2024-09-16T08:57:23.207 INFO:tasks.workunit.client.0.smithi031.stdout:4/604: dwrite d0/d2/d16/d59/d84/d86/d9b/fa4 [0,4194304] 0 2024-09-16T08:57:23.207 INFO:tasks.workunit.client.0.smithi031.stdout:4/605: chown d0/d2/f7e 303164 1 2024-09-16T08:57:23.207 INFO:tasks.workunit.client.0.smithi031.stdout:4/606: fsync d0/f2d 0 2024-09-16T08:57:23.217 INFO:tasks.workunit.client.0.smithi031.stdout:4/607: dread d0/d2/d16/d59/f5d [0,4194304] 0 2024-09-16T08:57:23.217 INFO:tasks.workunit.client.0.smithi031.stdout:4/608: chown d0/d2/d16/cba 109270858 1 2024-09-16T08:57:23.217 INFO:tasks.workunit.client.0.smithi031.stdout:4/609: fdatasync d0/f4c 0 2024-09-16T08:57:23.219 INFO:tasks.workunit.client.0.smithi031.stdout:4/610: rename d0/d3c/d6b/l9d to d0/d2/d16/d59/d8d/lc0 0 2024-09-16T08:57:23.221 INFO:tasks.workunit.client.0.smithi031.stdout:4/611: truncate d0/d2/d16/f4e 4130270 0 2024-09-16T08:57:23.243 INFO:tasks.workunit.client.0.smithi031.stdout:4/612: dwrite d0/d2/faa [0,4194304] 0 2024-09-16T08:57:23.243 INFO:tasks.workunit.client.0.smithi031.stdout:4/613: read - d0/d2/d3a/d55/fab zero size 2024-09-16T08:57:23.243 INFO:tasks.workunit.client.0.smithi031.stdout:4/614: write d0/f2d [844003,41329] 0 2024-09-16T08:57:23.257 INFO:tasks.workunit.client.0.smithi031.stdout:4/615: dwrite d0/f2e [4194304,4194304] 0 2024-09-16T08:57:23.258 INFO:tasks.workunit.client.0.smithi031.stdout:4/616: creat d0/d2/d16/d59/d8d/fc1 x:0 0 0 2024-09-16T08:57:23.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:22 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi097 to 2240M 2024-09-16T08:57:23.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:22 smithi097 ceph-mon[35478]: pgmap v13: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail; 228 MiB/s rd, 291 MiB/s wr, 403 op/s 2024-09-16T08:57:23.268 INFO:tasks.workunit.client.0.smithi031.stdout:7/659: sync 2024-09-16T08:57:23.269 INFO:tasks.workunit.client.0.smithi031.stdout:7/660: fdatasync d2/d8/f8b 0 2024-09-16T08:57:23.269 INFO:tasks.workunit.client.0.smithi031.stdout:7/661: rmdir d2/d8/d69 39 2024-09-16T08:57:23.270 INFO:tasks.workunit.client.0.smithi031.stdout:7/662: read - d2/d8/d69/d82/fd1 zero size 2024-09-16T08:57:23.270 INFO:tasks.workunit.client.0.smithi031.stdout:7/663: readlink d2/d8/d11/l1f 0 2024-09-16T08:57:23.271 INFO:tasks.workunit.client.0.smithi031.stdout:7/664: write d2/d8/d69/d82/f9b [227524,2452] 0 2024-09-16T08:57:23.271 INFO:tasks.workunit.client.0.smithi031.stdout:7/665: fsync d2/d2d/f43 0 2024-09-16T08:57:23.272 INFO:tasks.workunit.client.0.smithi031.stdout:7/666: chown d2/d2d/d6d/fa8 380085 1 2024-09-16T08:57:23.272 INFO:tasks.workunit.client.0.smithi031.stdout:7/667: unlink d2/c70 0 2024-09-16T08:57:23.280 INFO:tasks.workunit.client.0.smithi031.stdout:4/617: dwrite d0/d2/d16/d2b/f61 [0,4194304] 0 2024-09-16T08:57:23.289 INFO:tasks.workunit.client.0.smithi031.stdout:4/618: dread d0/d2/d16/f4e [0,4194304] 0 2024-09-16T08:57:23.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/668: dwrite d2/f42 [0,4194304] 0 2024-09-16T08:57:23.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/669: truncate d2/d8/d11/d5d/d7e/fb9 672611 0 2024-09-16T08:57:23.300 INFO:tasks.workunit.client.0.smithi031.stdout:4/619: read d0/d2/d16/d1e/f52 [3177173,77231] 0 2024-09-16T08:57:23.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/670: creat d2/d2d/fdc x:0 0 0 2024-09-16T08:57:23.300 INFO:tasks.workunit.client.0.smithi031.stdout:7/671: chown d2/d8/d11/d66/d9a/fc9 45358535 1 2024-09-16T08:57:23.337 INFO:tasks.workunit.client.0.smithi031.stdout:4/620: dwrite d0/d8e/f95 [0,4194304] 0 2024-09-16T08:57:23.338 INFO:tasks.workunit.client.0.smithi031.stdout:7/672: dwrite d2/d2d/d6d/fae [0,4194304] 0 2024-09-16T08:57:23.346 INFO:tasks.workunit.client.0.smithi031.stdout:7/673: rename d2/d8/d11/fa4 to d2/d8/d86/dd4/fdd 0 2024-09-16T08:57:23.356 INFO:tasks.workunit.client.0.smithi031.stdout:4/621: dwrite d0/d3c/f7f [0,4194304] 0 2024-09-16T08:57:23.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/622: creat d0/d8e/fc2 x:0 0 0 2024-09-16T08:57:23.365 INFO:tasks.workunit.client.0.smithi031.stdout:4/623: creat d0/d2/d16/d59/d84/d86/fc3 x:0 0 0 2024-09-16T08:57:23.366 INFO:tasks.workunit.client.0.smithi031.stdout:4/624: creat d0/d2/d16/d59/d84/fc4 x:0 0 0 2024-09-16T08:57:23.399 INFO:tasks.workunit.client.0.smithi031.stdout:4/625: dwrite d0/f4c [0,4194304] 0 2024-09-16T08:57:23.399 INFO:tasks.workunit.client.0.smithi031.stdout:4/626: write d0/d5b/f7d [1892423,22696] 0 2024-09-16T08:57:23.400 INFO:tasks.workunit.client.0.smithi031.stdout:4/627: chown d0/c15 275117562 1 2024-09-16T08:57:23.411 INFO:tasks.workunit.client.0.smithi031.stdout:4/628: write d0/d2/d16/d1e/f3f [2199095,26218] 0 2024-09-16T08:57:23.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/629: dwrite d0/d2/d3a/f97 [0,4194304] 0 2024-09-16T08:57:23.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/630: stat d0/db5/fb7 0 2024-09-16T08:57:23.440 INFO:tasks.workunit.client.0.smithi031.stdout:4/631: unlink d0/d2/d3a/c5a 0 2024-09-16T08:57:23.441 INFO:tasks.workunit.client.0.smithi031.stdout:4/632: mknod d0/d2/d16/d2b/cc5 0 2024-09-16T08:57:23.443 INFO:tasks.workunit.client.0.smithi031.stdout:4/633: stat d0/d2/d16/d59/l6c 0 2024-09-16T08:57:23.445 INFO:tasks.workunit.client.0.smithi031.stdout:4/634: creat d0/d8e/d75/fc6 x:0 0 0 2024-09-16T08:57:23.447 INFO:tasks.workunit.client.0.smithi031.stdout:9/562: sync 2024-09-16T08:57:23.447 INFO:tasks.workunit.client.0.smithi031.stdout:6/683: sync 2024-09-16T08:57:23.448 INFO:tasks.workunit.client.0.smithi031.stdout:4/635: rename d0/d2/d16/d59/d84/d86/fa6 to d0/d8e/d75/fc7 0 2024-09-16T08:57:23.450 INFO:tasks.workunit.client.0.smithi031.stdout:6/684: truncate d0/dd/d2b/f31 3043494 0 2024-09-16T08:57:23.450 INFO:tasks.workunit.client.0.smithi031.stdout:6/685: dread - d0/dd/d74/da6/fcf zero size 2024-09-16T08:57:23.452 INFO:tasks.workunit.client.0.smithi031.stdout:6/686: creat d0/fd7 x:0 0 0 2024-09-16T08:57:23.453 INFO:tasks.workunit.client.0.smithi031.stdout:6/687: getdents d0/dd/d74/da6 0 2024-09-16T08:57:23.453 INFO:tasks.workunit.client.0.smithi031.stdout:6/688: stat d0/d10/f5a 0 2024-09-16T08:57:23.456 INFO:tasks.workunit.client.0.smithi031.stdout:6/689: symlink d0/dd/d74/d67/dbb/ld8 0 2024-09-16T08:57:23.459 INFO:tasks.workunit.client.0.smithi031.stdout:6/690: symlink d0/dd/d74/d5e/ld9 0 2024-09-16T08:57:23.463 INFO:tasks.workunit.client.0.smithi031.stdout:6/691: truncate d0/dd/d35/d6c/d83/f89 1068093 0 2024-09-16T08:57:23.465 INFO:tasks.workunit.client.0.smithi031.stdout:6/692: getdents d0/dd/d74/d5e/d7c/d56 0 2024-09-16T08:57:23.468 INFO:tasks.workunit.client.0.smithi031.stdout:6/693: symlink d0/dd/d35/d6c/dac/lda 0 2024-09-16T08:57:23.470 INFO:tasks.workunit.client.0.smithi031.stdout:6/694: mknod d0/cdb 0 2024-09-16T08:57:23.488 INFO:tasks.workunit.client.0.smithi031.stdout:9/563: dwrite d3/d85/f91 [0,4194304] 0 2024-09-16T08:57:23.488 INFO:tasks.workunit.client.0.smithi031.stdout:4/636: dwrite d0/d2/d16/d59/f5d [4194304,4194304] 0 2024-09-16T08:57:23.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/695: dwrite d0/dd/d74/d19/fc2 [0,4194304] 0 2024-09-16T08:57:23.488 INFO:tasks.workunit.client.0.smithi031.stdout:6/696: creat d0/dd/d74/d19/d54/fdc x:0 0 0 2024-09-16T08:57:23.491 INFO:tasks.workunit.client.0.smithi031.stdout:9/564: symlink d3/d82/lc4 0 2024-09-16T08:57:23.492 INFO:tasks.workunit.client.0.smithi031.stdout:4/637: mkdir d0/d5b/db8/dc8 0 2024-09-16T08:57:23.496 INFO:tasks.workunit.client.0.smithi031.stdout:4/638: dread d0/d2/d16/d1e/f52 [0,4194304] 0 2024-09-16T08:57:23.496 INFO:tasks.workunit.client.0.smithi031.stdout:6/697: mkdir d0/dd/d74/d5e/d7c/d3f/ddd 0 2024-09-16T08:57:23.499 INFO:tasks.workunit.client.0.smithi031.stdout:4/639: mknod d0/db5/cc9 0 2024-09-16T08:57:23.501 INFO:tasks.workunit.client.0.smithi031.stdout:6/698: rename d0/d10/dc8/l94 to d0/dd/d74/d19/lde 0 2024-09-16T08:57:23.501 INFO:tasks.workunit.client.0.smithi031.stdout:6/699: creat d0/dd/d74/d19/fdf x:0 0 0 2024-09-16T08:57:23.501 INFO:tasks.workunit.client.0.smithi031.stdout:6/700: creat d0/dd/d74/d67/fe0 x:0 0 0 2024-09-16T08:57:23.504 INFO:tasks.workunit.client.0.smithi031.stdout:9/565: getdents d3/d5/da/db 0 2024-09-16T08:57:23.504 INFO:tasks.workunit.client.0.smithi031.stdout:9/566: stat d3 0 2024-09-16T08:57:23.504 INFO:tasks.workunit.client.0.smithi031.stdout:9/567: fdatasync d3/d5/da/f4f 0 2024-09-16T08:57:23.510 INFO:tasks.workunit.client.0.smithi031.stdout:9/568: link d3/d5/d6a/c8a d3/d5/da/d34/d3c/d5e/cc5 0 2024-09-16T08:57:23.512 INFO:tasks.workunit.client.0.smithi031.stdout:9/569: truncate d3/d5/da/f33 512319 0 2024-09-16T08:57:23.515 INFO:tasks.workunit.client.0.smithi031.stdout:9/570: mknod d3/d5/d6a/d74/da8/dc3/cc6 0 2024-09-16T08:57:23.517 INFO:tasks.workunit.client.0.smithi031.stdout:9/571: symlink d3/d5/d6a/d74/da8/dc3/dbc/lc7 0 2024-09-16T08:57:23.517 INFO:tasks.workunit.client.0.smithi031.stdout:9/572: chown d3/d5/d6a/d74/da8/dc3/la9 3 1 2024-09-16T08:57:23.529 INFO:tasks.workunit.client.0.smithi031.stdout:6/701: dwrite d0/dd/d74/d19/db4/fbc [0,4194304] 0 2024-09-16T08:57:23.530 INFO:tasks.workunit.client.0.smithi031.stdout:6/702: truncate d0/dd/d74/d5e/f7b 1438039 0 2024-09-16T08:57:23.530 INFO:tasks.workunit.client.0.smithi031.stdout:4/640: dwrite d0/f18 [0,4194304] 0 2024-09-16T08:57:23.533 INFO:tasks.workunit.client.0.smithi031.stdout:6/703: rmdir d0/dd/d74/d5e/d7c 39 2024-09-16T08:57:23.542 INFO:tasks.workunit.client.0.smithi031.stdout:4/641: symlink d0/lca 0 2024-09-16T08:57:23.552 INFO:tasks.workunit.client.0.smithi031.stdout:2/579: sync 2024-09-16T08:57:23.567 INFO:tasks.workunit.client.0.smithi031.stdout:6/704: dwrite d0/dd/d74/d19/f26 [0,4194304] 0 2024-09-16T08:57:23.569 INFO:tasks.workunit.client.0.smithi031.stdout:6/705: mkdir d0/dd/d74/d19/de1 0 2024-09-16T08:57:23.572 INFO:tasks.workunit.client.0.smithi031.stdout:2/580: write d6/f5b [507046,10904] 0 2024-09-16T08:57:23.573 INFO:tasks.workunit.client.0.smithi031.stdout:2/581: truncate d6/da/f85 119973 0 2024-09-16T08:57:23.580 INFO:tasks.workunit.client.0.smithi031.stdout:4/642: dwrite d0/d8e/f77 [0,4194304] 0 2024-09-16T08:57:23.580 INFO:tasks.workunit.client.0.smithi031.stdout:8/490: sync 2024-09-16T08:57:23.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/706: write d0/dd/d2b/f31 [2066418,6666] 0 2024-09-16T08:57:23.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/707: write d0/d10/f60 [4844911,36161] 0 2024-09-16T08:57:23.584 INFO:tasks.workunit.client.0.smithi031.stdout:6/708: chown d0/dd/d74/d5e/f77 13 1 2024-09-16T08:57:23.585 INFO:tasks.workunit.client.0.smithi031.stdout:4/643: rename d0/d2/d3a/d55/f64 to d0/d71/fcb 0 2024-09-16T08:57:23.586 INFO:tasks.workunit.client.0.smithi031.stdout:8/491: mknod d4/d7/d21/d37/ca6 0 2024-09-16T08:57:23.590 INFO:tasks.workunit.client.0.smithi031.stdout:6/709: truncate d0/dd/d74/d5e/d7c/d56/f78 177170 0 2024-09-16T08:57:23.597 INFO:tasks.workunit.client.0.smithi031.stdout:2/582: dwrite d6/da/d5e/f70 [0,4194304] 0 2024-09-16T08:57:23.597 INFO:tasks.workunit.client.0.smithi031.stdout:2/583: readlink d6/da/d37/d8b/dad/lb6 0 2024-09-16T08:57:23.598 INFO:tasks.workunit.client.0.smithi031.stdout:8/492: write d4/d77/f81 [2320655,121476] 0 2024-09-16T08:57:23.602 INFO:tasks.workunit.client.0.smithi031.stdout:2/584: unlink d6/da/l55 0 2024-09-16T08:57:23.661 INFO:tasks.workunit.client.0.smithi031.stdout:6/710: dwrite d0/f7f [0,4194304] 0 2024-09-16T08:57:23.661 INFO:tasks.workunit.client.0.smithi031.stdout:4/644: dwrite d0/f94 [0,4194304] 0 2024-09-16T08:57:23.661 INFO:tasks.workunit.client.0.smithi031.stdout:4/645: write d0/d2/d16/d59/d84/d86/fa8 [240798,128241] 0 2024-09-16T08:57:23.661 INFO:tasks.workunit.client.0.smithi031.stdout:8/493: dwrite d4/d7/d21/d24/f31 [0,4194304] 0 2024-09-16T08:57:23.662 INFO:tasks.workunit.client.0.smithi031.stdout:2/585: dwrite d6/da/d4b/fc1 [0,4194304] 0 2024-09-16T08:57:23.662 INFO:tasks.workunit.client.0.smithi031.stdout:2/586: write d6/d2e/d40/d6c/d2c/d53/d60/f39 [67017,114777] 0 2024-09-16T08:57:23.669 INFO:tasks.workunit.client.0.smithi031.stdout:8/494: rmdir d4/d9e/d5e 39 2024-09-16T08:57:23.674 INFO:tasks.workunit.client.0.smithi031.stdout:4/646: dread d0/d8e/f95 [0,4194304] 0 2024-09-16T08:57:23.677 INFO:tasks.workunit.client.0.smithi031.stdout:8/495: truncate d4/d5b/f62 439612 0 2024-09-16T08:57:23.680 INFO:tasks.workunit.client.0.smithi031.stdout:8/496: link d4/l5a d4/d7/d21/d35/d79/la7 0 2024-09-16T08:57:23.682 INFO:tasks.workunit.client.0.smithi031.stdout:8/497: rename d4/d2d/f3a to d4/d7/fa8 0 2024-09-16T08:57:23.683 INFO:tasks.workunit.client.0.smithi031.stdout:4/647: dread d0/d2/faa [0,4194304] 0 2024-09-16T08:57:23.684 INFO:tasks.workunit.client.0.smithi031.stdout:4/648: truncate d0/d2/d16/d2b/f61 1181970 0 2024-09-16T08:57:23.685 INFO:tasks.workunit.client.0.smithi031.stdout:4/649: read d0/d2/d3a/f93 [165213,65380] 0 2024-09-16T08:57:23.686 INFO:tasks.workunit.client.0.smithi031.stdout:4/650: getdents d0 0 2024-09-16T08:57:23.687 INFO:tasks.workunit.client.0.smithi031.stdout:4/651: creat d0/d2/d16/d1e/db9/fcc x:0 0 0 2024-09-16T08:57:23.688 INFO:tasks.workunit.client.0.smithi031.stdout:4/652: write d0/d2/d16/f19 [276902,118987] 0 2024-09-16T08:57:23.694 INFO:tasks.workunit.client.0.smithi031.stdout:8/498: write d4/d7/d21/d35/f5d [394684,77906] 0 2024-09-16T08:57:23.694 INFO:tasks.workunit.client.0.smithi031.stdout:8/499: readlink d4/d9e/d5e/l63 0 2024-09-16T08:57:23.696 INFO:tasks.workunit.client.0.smithi031.stdout:6/711: dwrite d0/f91 [0,4194304] 0 2024-09-16T08:57:23.697 INFO:tasks.workunit.client.0.smithi031.stdout:2/587: dwrite d6/da/f1e [0,4194304] 0 2024-09-16T08:57:23.697 INFO:tasks.workunit.client.0.smithi031.stdout:6/712: dread d0/dd/d74/da6/fc3 [0,4194304] 0 2024-09-16T08:57:23.697 INFO:tasks.workunit.client.0.smithi031.stdout:4/653: truncate d0/f94 520619 0 2024-09-16T08:57:23.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/500: read d4/d9e/d38/f3c [2206043,87153] 0 2024-09-16T08:57:23.703 INFO:tasks.workunit.client.0.smithi031.stdout:8/501: chown d4/d9e/c92 1365 1 2024-09-16T08:57:23.711 INFO:tasks.workunit.client.0.smithi031.stdout:2/588: symlink d6/da/d4b/d4c/d73/lc6 0 2024-09-16T08:57:23.712 INFO:tasks.workunit.client.0.smithi031.stdout:2/589: read d6/d2e/d40/d58/f90 [199791,1574] 0 2024-09-16T08:57:23.712 INFO:tasks.workunit.client.0.smithi031.stdout:2/590: chown d6/d2e/f2f 31 1 2024-09-16T08:57:23.712 INFO:tasks.workunit.client.0.smithi031.stdout:2/591: read d6/d2e/d40/d6c/d2c/d53/d60/f35 [952805,22163] 0 2024-09-16T08:57:23.712 INFO:tasks.workunit.client.0.smithi031.stdout:2/592: fsync d6/d2e/d40/d6c/d2c/d53/d60/f39 0 2024-09-16T08:57:23.713 INFO:tasks.workunit.client.0.smithi031.stdout:4/654: creat d0/d71/fcd x:0 0 0 2024-09-16T08:57:23.714 INFO:tasks.workunit.client.0.smithi031.stdout:8/502: rmdir d4/d9e/d5e 39 2024-09-16T08:57:23.715 INFO:tasks.workunit.client.0.smithi031.stdout:4/655: dread d0/f38 [4194304,4194304] 0 2024-09-16T08:57:23.717 INFO:tasks.workunit.client.0.smithi031.stdout:2/593: link d6/d20/d51/l97 d6/d2e/d40/dc3/lc7 0 2024-09-16T08:57:23.720 INFO:tasks.workunit.client.0.smithi031.stdout:8/503: rename d4/d9e/d5e/f9a to d4/fa9 0 2024-09-16T08:57:23.724 INFO:tasks.workunit.client.0.smithi031.stdout:2/594: getdents d6/da/d37 0 2024-09-16T08:57:23.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/504: mkdir d4/d9e/daa 0 2024-09-16T08:57:23.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/505: creat d4/d2d/d6e/fab x:0 0 0 2024-09-16T08:57:23.729 INFO:tasks.workunit.client.0.smithi031.stdout:4/656: rmdir d0/d8e 39 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:2/595: truncate d6/d20/f4f 6182879 0 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:2/596: creat d6/da/d4b/fc8 x:0 0 0 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:2/597: chown d6/da/d4b/fc8 0 1 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:5/530: sync 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:3/635: sync 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:0/569: sync 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:1/527: sync 2024-09-16T08:57:23.731 INFO:tasks.workunit.client.0.smithi031.stdout:0/570: fsync d6/d12/d47/f7e 0 2024-09-16T08:57:23.733 INFO:tasks.workunit.client.0.smithi031.stdout:7/674: sync 2024-09-16T08:57:23.733 INFO:tasks.workunit.client.0.smithi031.stdout:9/573: sync 2024-09-16T08:57:23.733 INFO:tasks.workunit.client.0.smithi031.stdout:7/675: creat d2/d8/d11/d66/d9a/fde x:0 0 0 2024-09-16T08:57:23.734 INFO:tasks.workunit.client.0.smithi031.stdout:4/657: dread d0/d2/d16/f19 [0,4194304] 0 2024-09-16T08:57:23.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/506: link d4/d7/d21/d37/d46/l5c d4/d2f/lac 0 2024-09-16T08:57:23.739 INFO:tasks.workunit.client.0.smithi031.stdout:9/574: dread d3/d5/da/d34/f77 [0,4194304] 0 2024-09-16T08:57:23.739 INFO:tasks.workunit.client.0.smithi031.stdout:9/575: creat d3/d48/fc8 x:0 0 0 2024-09-16T08:57:23.741 INFO:tasks.workunit.client.0.smithi031.stdout:2/598: link d6/d2e/d40/d6c/d2c/l8a d6/da/d5e/d8d/lc9 0 2024-09-16T08:57:23.741 INFO:tasks.workunit.client.0.smithi031.stdout:2/599: fsync d6/d2e/d40/f54 0 2024-09-16T08:57:23.746 INFO:tasks.workunit.client.0.smithi031.stdout:5/531: unlink d2/f7d 0 2024-09-16T08:57:23.746 INFO:tasks.workunit.client.0.smithi031.stdout:5/532: write d2/dc/d91/f99 [774797,18721] 0 2024-09-16T08:57:23.751 INFO:tasks.workunit.client.0.smithi031.stdout:3/636: truncate d3/d14/d37/d33/f39 666454 0 2024-09-16T08:57:23.751 INFO:tasks.workunit.client.0.smithi031.stdout:3/637: write d3/d14/d37/d33/f3c [1505236,42041] 0 2024-09-16T08:57:23.751 INFO:tasks.workunit.client.0.smithi031.stdout:3/638: chown d3/d14/d37/d1a/d1d/d3b/d3f/d49 3 1 2024-09-16T08:57:23.752 INFO:tasks.workunit.client.0.smithi031.stdout:1/528: truncate d2/d4/d52/f74 3967070 0 2024-09-16T08:57:23.752 INFO:tasks.workunit.client.0.smithi031.stdout:1/529: fdatasync d2/d15/d38/f85 0 2024-09-16T08:57:23.754 INFO:tasks.workunit.client.0.smithi031.stdout:0/571: rename d6/d12/d13/d36/f59 to d6/d12/d13/d3c/fb7 0 2024-09-16T08:57:23.757 INFO:tasks.workunit.client.0.smithi031.stdout:8/507: creat d4/d7/fad x:0 0 0 2024-09-16T08:57:23.758 INFO:tasks.workunit.client.0.smithi031.stdout:3/639: rmdir d3/d14/d37/d1a/d1d/d3b/d3f/d79 39 2024-09-16T08:57:23.760 INFO:tasks.workunit.client.0.smithi031.stdout:1/530: fdatasync d2/d4/f31 0 2024-09-16T08:57:23.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/531: chown d2/d4 1 1 2024-09-16T08:57:23.761 INFO:tasks.workunit.client.0.smithi031.stdout:9/576: dwrite d3/d48/fc8 [0,4194304] 0 2024-09-16T08:57:23.761 INFO:tasks.workunit.client.0.smithi031.stdout:9/577: fsync d3/d5/da/d34/fbe 0 2024-09-16T08:57:23.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/532: dread d2/f53 [0,4194304] 0 2024-09-16T08:57:23.763 INFO:tasks.workunit.client.0.smithi031.stdout:7/676: rename d2/d8/d11/d66/fd6 to d2/d8/d11/d66/d9a/fdf 0 2024-09-16T08:57:23.763 INFO:tasks.workunit.client.0.smithi031.stdout:4/658: rename d0/d2/d16/d59/d84 to d0/d2/d16/d59/d84/d86/d9b/dce 22 2024-09-16T08:57:23.763 INFO:tasks.workunit.client.0.smithi031.stdout:4/659: getdents d0 0 2024-09-16T08:57:23.763 INFO:tasks.workunit.client.0.smithi031.stdout:4/660: readlink d0/d2/d16/l92 0 2024-09-16T08:57:23.763 INFO:tasks.workunit.client.0.smithi031.stdout:4/661: write d0/d2/f90 [4488373,33400] 0 2024-09-16T08:57:23.765 INFO:tasks.workunit.client.0.smithi031.stdout:2/600: dwrite d6/d2e/d40/d6c/f5a [0,4194304] 0 2024-09-16T08:57:23.765 INFO:tasks.workunit.client.0.smithi031.stdout:2/601: write d6/f74 [655085,72702] 0 2024-09-16T08:57:23.775 INFO:tasks.workunit.client.0.smithi031.stdout:0/572: dread d6/f57 [0,4194304] 0 2024-09-16T08:57:23.775 INFO:tasks.workunit.client.0.smithi031.stdout:0/573: write d6/d12/f49 [924746,6182] 0 2024-09-16T08:57:23.779 INFO:tasks.workunit.client.0.smithi031.stdout:7/677: dwrite d2/d2d/d6d/fae [4194304,4194304] 0 2024-09-16T08:57:23.779 INFO:tasks.workunit.client.0.smithi031.stdout:7/678: chown d2/d2d/d6d/fae 1 1 2024-09-16T08:57:23.792 INFO:tasks.workunit.client.0.smithi031.stdout:0/574: dwrite d6/d12/d13/f68 [0,4194304] 0 2024-09-16T08:57:23.822 INFO:tasks.workunit.client.0.smithi031.stdout:3/640: link d3/d14/d37/d33/cd d3/d14/d6f/d7d/cbb 0 2024-09-16T08:57:23.822 INFO:tasks.workunit.client.0.smithi031.stdout:3/641: chown d3/d14/d93/f99 82 1 2024-09-16T08:57:23.822 INFO:tasks.workunit.client.0.smithi031.stdout:3/642: write d3/d14/d37/d1a/d1d/d3b/d3f/d49/faa [101268,26276] 0 2024-09-16T08:57:23.832 INFO:tasks.workunit.client.0.smithi031.stdout:1/533: mknod d2/de/db5/cbd 0 2024-09-16T08:57:23.837 INFO:tasks.workunit.client.0.smithi031.stdout:4/662: mkdir d0/d2/d16/d59/d84/d86/d9b/dcf 0 2024-09-16T08:57:23.837 INFO:tasks.workunit.client.0.smithi031.stdout:4/663: chown d0/d2/d16/d1e/f23 1472 1 2024-09-16T08:57:23.837 INFO:tasks.workunit.client.0.smithi031.stdout:4/664: fdatasync d0/f2e 0 2024-09-16T08:57:23.839 INFO:tasks.workunit.client.0.smithi031.stdout:4/665: dread d0/d2/d16/f4f [0,4194304] 0 2024-09-16T08:57:23.842 INFO:tasks.workunit.client.0.smithi031.stdout:2/602: symlink d6/d2e/d40/d6c/d2c/d53/lca 0 2024-09-16T08:57:23.843 INFO:tasks.workunit.client.0.smithi031.stdout:2/603: chown d6/d2e/d40/c57 910 1 2024-09-16T08:57:23.845 INFO:tasks.workunit.client.0.smithi031.stdout:8/508: rmdir d4/d34 39 2024-09-16T08:57:23.845 INFO:tasks.workunit.client.0.smithi031.stdout:8/509: chown d4/d9e/daa 15311158 1 2024-09-16T08:57:23.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/575: creat d6/d12/d97/d9e/fb8 x:0 0 0 2024-09-16T08:57:23.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/576: stat d6/d12 0 2024-09-16T08:57:23.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/577: fsync d6/d12/d13/f3e 0 2024-09-16T08:57:23.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/578: chown d6/d12/d47/f7e 3739044 1 2024-09-16T08:57:23.916 INFO:tasks.workunit.client.0.smithi031.stdout:3/643: creat d3/d14/d37/d1a/d1d/dac/fbc x:0 0 0 2024-09-16T08:57:23.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/534: creat d2/d4/d1b/d5c/d6b/da9/fbe x:0 0 0 2024-09-16T08:57:23.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/535: chown d2/d15/d5d/c77 7 1 2024-09-16T08:57:23.921 INFO:tasks.workunit.client.0.smithi031.stdout:1/536: fsync d2/d15/d32/f97 0 2024-09-16T08:57:23.921 INFO:tasks.workunit.client.0.smithi031.stdout:1/537: chown d2/d4/d1b/f50 1 1 2024-09-16T08:57:23.931 INFO:tasks.workunit.client.0.smithi031.stdout:5/533: rename d2/dc/d91/f99 to d2/dc/d25/d21/fac 0 2024-09-16T08:57:23.933 INFO:tasks.workunit.client.0.smithi031.stdout:4/666: symlink d0/d2/d16/d59/d84/d86/d9b/dcf/ld0 0 2024-09-16T08:57:23.935 INFO:tasks.workunit.client.0.smithi031.stdout:2/604: truncate d6/da/f1d 297501 0 2024-09-16T08:57:23.935 INFO:tasks.workunit.client.0.smithi031.stdout:2/605: truncate d6/da/d37/d42/fbc 416916 0 2024-09-16T08:57:23.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/510: truncate d4/d7/f15 3025266 0 2024-09-16T08:57:23.941 INFO:tasks.workunit.client.0.smithi031.stdout:0/579: creat d6/d12/d47/fb9 x:0 0 0 2024-09-16T08:57:23.949 INFO:tasks.workunit.client.0.smithi031.stdout:1/538: truncate d2/d4/f31 2810862 0 2024-09-16T08:57:23.952 INFO:tasks.workunit.client.0.smithi031.stdout:9/578: rename d3/d5/da/d34/d3c/d5e/d62/d65/lb5 to d3/d5/da/db/d21/d3b/lc9 0 2024-09-16T08:57:23.953 INFO:tasks.workunit.client.0.smithi031.stdout:0/580: write d6/d12/d13/d3c/f6d [2343108,105636] 0 2024-09-16T08:57:23.953 INFO:tasks.workunit.client.0.smithi031.stdout:2/606: dwrite d6/da/f92 [0,4194304] 0 2024-09-16T08:57:23.954 INFO:tasks.workunit.client.0.smithi031.stdout:9/579: dread d3/d5/da/d34/d3c/d5e/fb1 [0,4194304] 0 2024-09-16T08:57:23.962 INFO:tasks.workunit.client.0.smithi031.stdout:4/667: symlink d0/d2/d16/d2b/daf/ld1 0 2024-09-16T08:57:23.962 INFO:tasks.workunit.client.0.smithi031.stdout:4/668: creat d0/d2/d16/d1e/db9/fd2 x:0 0 0 2024-09-16T08:57:23.967 INFO:tasks.workunit.client.0.smithi031.stdout:8/511: getdents d4/d7/d21/d35 0 2024-09-16T08:57:23.967 INFO:tasks.workunit.client.0.smithi031.stdout:8/512: fsync d4/d7/f51 0 2024-09-16T08:57:23.969 INFO:tasks.workunit.client.0.smithi031.stdout:1/539: link d2/cd d2/d15/d32/d87/cbf 0 2024-09-16T08:57:23.969 INFO:tasks.workunit.client.0.smithi031.stdout:1/540: stat d2/d15/d1f 0 2024-09-16T08:57:23.975 INFO:tasks.workunit.client.0.smithi031.stdout:2/607: dwrite d6/da/d4b/fa4 [0,4194304] 0 2024-09-16T08:57:23.975 INFO:tasks.workunit.client.0.smithi031.stdout:2/608: read - d6/da/d4b/d4c/d73/fb5 zero size 2024-09-16T08:57:23.979 INFO:tasks.workunit.client.0.smithi031.stdout:9/580: creat d3/d5/d73/d81/fca x:0 0 0 2024-09-16T08:57:23.980 INFO:tasks.workunit.client.0.smithi031.stdout:8/513: dread d4/d5b/d7b/f7d [0,4194304] 0 2024-09-16T08:57:23.982 INFO:tasks.workunit.client.0.smithi031.stdout:5/534: rmdir d2/dc/d1d 39 2024-09-16T08:57:23.982 INFO:tasks.workunit.client.0.smithi031.stdout:5/535: write d2/dc/f16 [1788873,17734] 0 2024-09-16T08:57:23.985 INFO:tasks.workunit.client.0.smithi031.stdout:2/609: write d6/da/f27 [3724102,78371] 0 2024-09-16T08:57:23.985 INFO:tasks.workunit.client.0.smithi031.stdout:0/581: dwrite d6/d12/d47/d5b/f5e [0,4194304] 0 2024-09-16T08:57:23.985 INFO:tasks.workunit.client.0.smithi031.stdout:2/610: chown d6/d20/f89 0 1 2024-09-16T08:57:23.985 INFO:tasks.workunit.client.0.smithi031.stdout:0/582: chown d6/d12/d47/f83 3 1 2024-09-16T08:57:23.988 INFO:tasks.workunit.client.0.smithi031.stdout:4/669: creat d0/d2/d16/d59/fd3 x:0 0 0 2024-09-16T08:57:23.988 INFO:tasks.workunit.client.0.smithi031.stdout:4/670: chown d0/d5b/c80 4231347 1 2024-09-16T08:57:23.988 INFO:tasks.workunit.client.0.smithi031.stdout:4/671: chown d0/d2/d16/d2b/cc5 255244 1 2024-09-16T08:57:23.988 INFO:tasks.workunit.client.0.smithi031.stdout:4/672: chown d0/d2/d16/d59/d84/d86/d9b/dcf/ld0 85685 1 2024-09-16T08:57:23.989 INFO:tasks.workunit.client.0.smithi031.stdout:1/541: mkdir d2/d4/d1b/d5c/d6b/da9/dc0 0 2024-09-16T08:57:23.991 INFO:tasks.workunit.client.0.smithi031.stdout:9/581: mkdir d3/d5/da/db/dcb 0 2024-09-16T08:57:23.992 INFO:tasks.workunit.client.0.smithi031.stdout:8/514: rmdir d4/d7/d21/d24 39 2024-09-16T08:57:23.992 INFO:tasks.workunit.client.0.smithi031.stdout:8/515: fsync d4/d7/f51 0 2024-09-16T08:57:23.992 INFO:tasks.workunit.client.0.smithi031.stdout:8/516: creat d4/d77/fae x:0 0 0 2024-09-16T08:57:23.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/673: write d0/d2/f10 [2025929,86380] 0 2024-09-16T08:57:23.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/674: truncate d0/d2/d16/d59/d84/d86/d9b/fb0 982969 0 2024-09-16T08:57:23.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/675: dread - d0/d2/d16/d1e/db9/fcc zero size 2024-09-16T08:57:23.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/676: chown d0/d2/d16/d59/l65 1020 1 2024-09-16T08:57:23.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/677: creat d0/d2/d16/d59/d8d/fd4 x:0 0 0 2024-09-16T08:57:23.998 INFO:tasks.workunit.client.0.smithi031.stdout:4/678: chown d0/d3c/f7f 1039 1 2024-09-16T08:57:23.999 INFO:tasks.workunit.client.0.smithi031.stdout:0/583: mkdir d6/d12/d97/db3/dba 0 2024-09-16T08:57:23.999 INFO:tasks.workunit.client.0.smithi031.stdout:0/584: write d6/d12/d13/f68 [4836676,77774] 0 2024-09-16T08:57:23.999 INFO:tasks.workunit.client.0.smithi031.stdout:0/585: stat d6/d12/d47/d5b/l63 0 2024-09-16T08:57:24.000 INFO:tasks.workunit.client.0.smithi031.stdout:1/542: read - d2/d4/d36/f92 zero size 2024-09-16T08:57:24.001 INFO:tasks.workunit.client.0.smithi031.stdout:4/679: dread d0/fa [0,4194304] 0 2024-09-16T08:57:24.001 INFO:tasks.workunit.client.0.smithi031.stdout:9/582: mknod d3/d5/da/db/d21/d3b/d8b/ccc 0 2024-09-16T08:57:24.003 INFO:tasks.workunit.client.0.smithi031.stdout:8/517: creat d4/d2d/faf x:0 0 0 2024-09-16T08:57:24.007 INFO:tasks.workunit.client.0.smithi031.stdout:4/680: getdents d0/d2/d16/d2b 0 2024-09-16T08:57:24.009 INFO:tasks.workunit.client.0.smithi031.stdout:9/583: mkdir d3/d5/da/d34/d3c/d5e/d9c/dcd 0 2024-09-16T08:57:24.010 INFO:tasks.workunit.client.0.smithi031.stdout:8/518: creat d4/d5b/d8d/fb0 x:0 0 0 2024-09-16T08:57:24.011 INFO:tasks.workunit.client.0.smithi031.stdout:0/586: rmdir d6/d12/d47/d5b/d9b 39 2024-09-16T08:57:24.014 INFO:tasks.workunit.client.0.smithi031.stdout:9/584: creat d3/d5/da/db/dcb/fce x:0 0 0 2024-09-16T08:57:24.016 INFO:tasks.workunit.client.0.smithi031.stdout:9/585: getdents d3/d5/da/d34/d3c 0 2024-09-16T08:57:24.018 INFO:tasks.workunit.client.0.smithi031.stdout:9/586: mkdir d3/d5/da/d34/d3c/d5e/d9c/d95/dcf 0 2024-09-16T08:57:24.019 INFO:tasks.workunit.client.0.smithi031.stdout:1/543: dwrite d2/d15/d1f/d43/f4c [0,4194304] 0 2024-09-16T08:57:24.022 INFO:tasks.workunit.client.0.smithi031.stdout:9/587: mkdir d3/d5/d6a/d74/dd0 0 2024-09-16T08:57:24.024 INFO:tasks.workunit.client.0.smithi031.stdout:1/544: truncate d2/d15/d5d/f23 958066 0 2024-09-16T08:57:24.024 INFO:tasks.workunit.client.0.smithi031.stdout:1/545: chown d2/d15/d38/f9a 1037239 1 2024-09-16T08:57:24.032 INFO:tasks.workunit.client.0.smithi031.stdout:1/546: read d2/de/f2a [1878142,27767] 0 2024-09-16T08:57:24.032 INFO:tasks.workunit.client.0.smithi031.stdout:1/547: chown d2/d15/d4a 28209 1 2024-09-16T08:57:24.032 INFO:tasks.workunit.client.0.smithi031.stdout:1/548: fdatasync d2/f17 0 2024-09-16T08:57:24.032 INFO:tasks.workunit.client.0.smithi031.stdout:4/681: dwrite d0/d2/d16/f6e [0,4194304] 0 2024-09-16T08:57:24.035 INFO:tasks.workunit.client.0.smithi031.stdout:4/682: mkdir d0/d2/d16/d59/dd5 0 2024-09-16T08:57:24.037 INFO:tasks.workunit.client.0.smithi031.stdout:4/683: mknod d0/d2/d16/d59/d84/d86/d9b/cd6 0 2024-09-16T08:57:24.039 INFO:tasks.workunit.client.0.smithi031.stdout:4/684: chown d0/d2/d3a/c53 27792937 1 2024-09-16T08:57:24.039 INFO:tasks.workunit.client.0.smithi031.stdout:4/685: write d0/fa [2202058,29709] 0 2024-09-16T08:57:24.039 INFO:tasks.workunit.client.0.smithi031.stdout:4/686: truncate d0/d2/d16/d1e/db9/fd2 96374 0 2024-09-16T08:57:24.039 INFO:tasks.workunit.client.0.smithi031.stdout:4/687: chown d0/l5 51437 1 2024-09-16T08:57:24.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/688: creat d0/d2/d16/d2b/fd7 x:0 0 0 2024-09-16T08:57:24.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/689: write d0/d2/d16/d1e/db9/fcc [585433,47189] 0 2024-09-16T08:57:24.041 INFO:tasks.workunit.client.0.smithi031.stdout:4/690: symlink d0/d2/d16/d59/d84/d86/d9b/dcf/ld8 0 2024-09-16T08:57:24.044 INFO:tasks.workunit.client.0.smithi031.stdout:0/587: dwrite d6/f33 [0,4194304] 0 2024-09-16T08:57:24.047 INFO:tasks.workunit.client.0.smithi031.stdout:0/588: getdents d6/d61/d91 0 2024-09-16T08:57:24.053 INFO:tasks.workunit.client.0.smithi031.stdout:4/691: dread d0/d2/fe [0,4194304] 0 2024-09-16T08:57:24.053 INFO:tasks.workunit.client.0.smithi031.stdout:9/588: dwrite d3/d5/da/f2b [0,4194304] 0 2024-09-16T08:57:24.053 INFO:tasks.workunit.client.0.smithi031.stdout:9/589: stat d3/d5/da/d34/d3c/d5e/d62/d65/fbd 0 2024-09-16T08:57:24.054 INFO:tasks.workunit.client.0.smithi031.stdout:4/692: mknod d0/d5b/db8/dc8/cd9 0 2024-09-16T08:57:24.055 INFO:tasks.workunit.client.0.smithi031.stdout:9/590: creat d3/d5/da/d34/d3c/d5e/d62/fd1 x:0 0 0 2024-09-16T08:57:24.057 INFO:tasks.workunit.client.0.smithi031.stdout:9/591: dread d3/d5/da/db/fe [0,4194304] 0 2024-09-16T08:57:24.057 INFO:tasks.workunit.client.0.smithi031.stdout:9/592: stat d3 0 2024-09-16T08:57:24.057 INFO:tasks.workunit.client.0.smithi031.stdout:9/593: creat d3/d5/da/d34/d3c/d5e/d9c/d95/fd2 x:0 0 0 2024-09-16T08:57:24.057 INFO:tasks.workunit.client.0.smithi031.stdout:4/693: creat d0/d3c/d6b/fda x:0 0 0 2024-09-16T08:57:24.059 INFO:tasks.workunit.client.0.smithi031.stdout:9/594: mkdir d3/d5/da/db/d21/d3b/dad/dd3 0 2024-09-16T08:57:24.069 INFO:tasks.workunit.client.0.smithi031.stdout:6/713: sync 2024-09-16T08:57:24.069 INFO:tasks.workunit.client.0.smithi031.stdout:6/714: dread - d0/d10/fc7 zero size 2024-09-16T08:57:24.069 INFO:tasks.workunit.client.0.smithi031.stdout:6/715: readlink d0/dd/d74/d67/lcc 0 2024-09-16T08:57:24.070 INFO:tasks.workunit.client.0.smithi031.stdout:6/716: symlink d0/dd/d74/d67/le2 0 2024-09-16T08:57:24.071 INFO:tasks.workunit.client.0.smithi031.stdout:6/717: rmdir d0/dd/d74/dad 39 2024-09-16T08:57:24.071 INFO:tasks.workunit.client.0.smithi031.stdout:6/718: stat d0/dd/d74/c2c 0 2024-09-16T08:57:24.072 INFO:tasks.workunit.client.0.smithi031.stdout:6/719: mkdir d0/d10/dd2/de3 0 2024-09-16T08:57:24.098 INFO:tasks.workunit.client.0.smithi031.stdout:3/644: unlink d3/d14/d37/d1a/d1d/d3b/d3f/f52 0 2024-09-16T08:57:24.102 INFO:tasks.workunit.client.0.smithi031.stdout:6/720: dwrite d0/dd/d74/d5e/d7c/d3f/d92/fc6 [0,4194304] 0 2024-09-16T08:57:24.102 INFO:tasks.workunit.client.0.smithi031.stdout:4/694: dwrite d0/d2/d16/d1e/db9/fd2 [0,4194304] 0 2024-09-16T08:57:24.102 INFO:tasks.workunit.client.0.smithi031.stdout:9/595: dwrite d3/d5/da/d34/f46 [0,4194304] 0 2024-09-16T08:57:24.103 INFO:tasks.workunit.client.0.smithi031.stdout:7/679: rename d2/d2d/f77 to d2/d8/d11/d5d/d7e/fe0 0 2024-09-16T08:57:24.103 INFO:tasks.workunit.client.0.smithi031.stdout:7/680: fdatasync d2/d8/d11/d5d/d7e/db6/fd2 0 2024-09-16T08:57:24.109 INFO:tasks.workunit.client.0.smithi031.stdout:1/549: dread d2/d4/d1b/f50 [0,4194304] 0 2024-09-16T08:57:24.109 INFO:tasks.workunit.client.0.smithi031.stdout:1/550: readlink d2/d15/d9c/l9e 0 2024-09-16T08:57:24.110 INFO:tasks.workunit.client.0.smithi031.stdout:5/536: dwrite d2/dc/f2d [0,4194304] 0 2024-09-16T08:57:24.110 INFO:tasks.workunit.client.0.smithi031.stdout:5/537: chown d2/dc/d36/d76/l9b 0 1 2024-09-16T08:57:24.116 INFO:tasks.workunit.client.0.smithi031.stdout:9/596: unlink d3/d48/f59 0 2024-09-16T08:57:24.118 INFO:tasks.workunit.client.0.smithi031.stdout:4/695: creat d0/d2/fdb x:0 0 0 2024-09-16T08:57:24.118 INFO:tasks.workunit.client.0.smithi031.stdout:3/645: dwrite d3/d14/d37/d1a/d1d/d1e/f80 [0,4194304] 0 2024-09-16T08:57:24.119 INFO:tasks.workunit.client.0.smithi031.stdout:8/519: rename d4/d34 to d4/d7/d21/d37/d46/db1 0 2024-09-16T08:57:24.120 INFO:tasks.workunit.client.0.smithi031.stdout:6/721: dread d0/dd/d74/d5e/d7c/d56/f78 [0,4194304] 0 2024-09-16T08:57:24.120 INFO:tasks.workunit.client.0.smithi031.stdout:6/722: truncate d0/dd/d2b/f31 3105876 0 2024-09-16T08:57:24.121 INFO:tasks.workunit.client.0.smithi031.stdout:1/551: getdents d2/d15/d1f 0 2024-09-16T08:57:24.127 INFO:tasks.workunit.client.0.smithi031.stdout:5/538: dwrite d2/dc/d3d/f42 [0,4194304] 0 2024-09-16T08:57:24.128 INFO:tasks.workunit.client.0.smithi031.stdout:4/696: symlink d0/d2/d16/d59/d84/d86/ldc 0 2024-09-16T08:57:24.128 INFO:tasks.workunit.client.0.smithi031.stdout:4/697: write d0/f2e [9022618,116056] 0 2024-09-16T08:57:24.131 INFO:tasks.workunit.client.0.smithi031.stdout:3/646: mknod d3/d14/d37/d1a/d1d/d1e/cbd 0 2024-09-16T08:57:24.133 INFO:tasks.workunit.client.0.smithi031.stdout:0/589: rename d6/d12/d47/d7c/caa to d6/d12/d97/d9e/cbb 0 2024-09-16T08:57:24.134 INFO:tasks.workunit.client.0.smithi031.stdout:8/520: link d4/d5b/f62 d4/d2d/d97/fb2 0 2024-09-16T08:57:24.135 INFO:tasks.workunit.client.0.smithi031.stdout:8/521: truncate d4/d77/fae 457053 0 2024-09-16T08:57:24.135 INFO:tasks.workunit.client.0.smithi031.stdout:8/522: write d4/d2d/d6e/f74 [567349,597] 0 2024-09-16T08:57:24.139 INFO:tasks.workunit.client.0.smithi031.stdout:6/723: unlink d0/dd/d74/d19/f29 0 2024-09-16T08:57:24.141 INFO:tasks.workunit.client.0.smithi031.stdout:8/523: dread d4/d77/f81 [0,4194304] 0 2024-09-16T08:57:24.141 INFO:tasks.workunit.client.0.smithi031.stdout:8/524: fsync d4/d9e/d38/f73 0 2024-09-16T08:57:24.144 INFO:tasks.workunit.client.0.smithi031.stdout:1/552: mkdir d2/d4/d1b/d5c/d6b/dc1 0 2024-09-16T08:57:24.144 INFO:tasks.workunit.client.0.smithi031.stdout:1/553: dread - d2/d4/d1b/d5c/f94 zero size 2024-09-16T08:57:24.148 INFO:tasks.workunit.client.0.smithi031.stdout:3/647: mknod d3/d14/d37/d1a/d1d/cbe 0 2024-09-16T08:57:24.150 INFO:tasks.workunit.client.0.smithi031.stdout:5/539: dwrite d2/dc/d25/d21/fac [0,4194304] 0 2024-09-16T08:57:24.150 INFO:tasks.workunit.client.0.smithi031.stdout:4/698: dwrite d0/d2/d16/d2b/f61 [0,4194304] 0 2024-09-16T08:57:24.150 INFO:tasks.workunit.client.0.smithi031.stdout:4/699: readlink d0/d2/l8b 0 2024-09-16T08:57:24.151 INFO:tasks.workunit.client.0.smithi031.stdout:4/700: chown d0/c51 4792 1 2024-09-16T08:57:24.152 INFO:tasks.workunit.client.0.smithi031.stdout:7/681: rename d2/l34 to d2/d8/le1 0 2024-09-16T08:57:24.152 INFO:tasks.workunit.client.0.smithi031.stdout:7/682: readlink d2/d8/d11/d5d/lc3 0 2024-09-16T08:57:24.154 INFO:tasks.workunit.client.0.smithi031.stdout:0/590: creat d6/d12/d13/da5/fbc x:0 0 0 2024-09-16T08:57:24.159 INFO:tasks.workunit.client.0.smithi031.stdout:1/554: creat d2/de/db5/fc2 x:0 0 0 2024-09-16T08:57:24.160 INFO:tasks.workunit.client.0.smithi031.stdout:0/591: dread d6/d12/f6c [0,4194304] 0 2024-09-16T08:57:24.160 INFO:tasks.workunit.client.0.smithi031.stdout:0/592: write d6/d12/d13/d3c/f45 [3676051,104769] 0 2024-09-16T08:57:24.161 INFO:tasks.workunit.client.0.smithi031.stdout:0/593: chown d6/caf 49 1 2024-09-16T08:57:24.165 INFO:tasks.workunit.client.0.smithi031.stdout:3/648: creat d3/d14/d37/d1a/d24/fbf x:0 0 0 2024-09-16T08:57:24.170 INFO:tasks.workunit.client.0.smithi031.stdout:1/555: write d2/d15/d1f/d43/f4c [3526980,100007] 0 2024-09-16T08:57:24.171 INFO:tasks.workunit.client.0.smithi031.stdout:1/556: dread - d2/d4/d52/f63 zero size 2024-09-16T08:57:24.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/597: rename d3/d5/d6a/cbf to d3/d5/da/d34/d3c/d5e/d62/d65/cd4 0 2024-09-16T08:57:24.177 INFO:tasks.workunit.client.0.smithi031.stdout:6/724: getdents d0/dd/d74/d19/d54 0 2024-09-16T08:57:24.177 INFO:tasks.workunit.client.0.smithi031.stdout:6/725: stat d0/la5 0 2024-09-16T08:57:24.181 INFO:tasks.workunit.client.0.smithi031.stdout:9/598: fsync d3/d5/d6a/d74/da8/dc3/f7b 0 2024-09-16T08:57:24.182 INFO:tasks.workunit.client.0.smithi031.stdout:3/649: getdents d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87 0 2024-09-16T08:57:24.183 INFO:tasks.workunit.client.0.smithi031.stdout:3/650: write d3/d14/d6f/f72 [1678479,40516] 0 2024-09-16T08:57:24.184 INFO:tasks.workunit.client.0.smithi031.stdout:7/683: dwrite d2/d8/d11/d66/d9a/fde [0,4194304] 0 2024-09-16T08:57:24.184 INFO:tasks.workunit.client.0.smithi031.stdout:9/599: symlink d3/d5/da/db/ld5 0 2024-09-16T08:57:24.186 INFO:tasks.workunit.client.0.smithi031.stdout:5/540: dwrite d2/dc/f88 [0,4194304] 0 2024-09-16T08:57:24.189 INFO:tasks.workunit.client.0.smithi031.stdout:3/651: unlink d3/d14/d93/f99 0 2024-09-16T08:57:24.189 INFO:tasks.workunit.client.0.smithi031.stdout:3/652: chown d3/d14/cab 930330 1 2024-09-16T08:57:24.189 INFO:tasks.workunit.client.0.smithi031.stdout:3/653: fsync d3/d14/d37/d1a/d1d/d3b/d3f/f74 0 2024-09-16T08:57:24.189 INFO:tasks.workunit.client.0.smithi031.stdout:3/654: write d3/d14/d37/d1a/f5c [858528,83294] 0 2024-09-16T08:57:24.191 INFO:tasks.workunit.client.0.smithi031.stdout:7/684: chown d2/d8/d11/c4c 0 1 2024-09-16T08:57:24.191 INFO:tasks.workunit.client.0.smithi031.stdout:7/685: stat d2/d8/d69/d82/fa0 0 2024-09-16T08:57:24.191 INFO:tasks.workunit.client.0.smithi031.stdout:7/686: chown d2/d8/d11/d5d/d7e/l98 49 1 2024-09-16T08:57:24.191 INFO:tasks.workunit.client.0.smithi031.stdout:7/687: dread - d2/d8/d86/f88 zero size 2024-09-16T08:57:24.192 INFO:tasks.workunit.client.0.smithi031.stdout:4/701: dwrite d0/d2/d16/d1e/f9e [0,4194304] 0 2024-09-16T08:57:24.192 INFO:tasks.workunit.client.0.smithi031.stdout:4/702: fsync d0/d2/d16/d1e/fa1 0 2024-09-16T08:57:24.193 INFO:tasks.workunit.client.0.smithi031.stdout:9/600: dread - d3/d48/f9d zero size 2024-09-16T08:57:24.193 INFO:tasks.workunit.client.0.smithi031.stdout:1/557: dwrite f0 [4194304,4194304] 0 2024-09-16T08:57:24.197 INFO:tasks.workunit.client.0.smithi031.stdout:6/726: dwrite d0/dd/d74/d19/f26 [0,4194304] 0 2024-09-16T08:57:24.197 INFO:tasks.workunit.client.0.smithi031.stdout:5/541: mkdir d2/dc/d25/dad 0 2024-09-16T08:57:24.202 INFO:tasks.workunit.client.0.smithi031.stdout:3/655: mkdir d3/d14/d6f/d82/dc0 0 2024-09-16T08:57:24.203 INFO:tasks.workunit.client.0.smithi031.stdout:7/688: getdents d2/d2d 0 2024-09-16T08:57:24.204 INFO:tasks.workunit.client.0.smithi031.stdout:4/703: getdents d0/d5b/db8 0 2024-09-16T08:57:24.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/558: rename d2/d15/d5d/fac to d2/d15/d5d/db4/fc3 0 2024-09-16T08:57:24.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/559: chown d2/d15/d5d/c7a 1606 1 2024-09-16T08:57:24.213 INFO:tasks.workunit.client.0.smithi031.stdout:6/727: symlink d0/dd/d74/d19/de1/le4 0 2024-09-16T08:57:24.213 INFO:tasks.workunit.client.0.smithi031.stdout:6/728: chown d0/f4 2 1 2024-09-16T08:57:24.223 INFO:tasks.workunit.client.0.smithi031.stdout:7/689: creat d2/d2d/d68/da2/fe2 x:0 0 0 2024-09-16T08:57:24.232 INFO:tasks.workunit.client.0.smithi031.stdout:4/704: truncate d0/d2/d3a/f6f 79144 0 2024-09-16T08:57:24.232 INFO:tasks.workunit.client.0.smithi031.stdout:0/594: write d6/f8 [169772,14642] 0 2024-09-16T08:57:24.232 INFO:tasks.workunit.client.0.smithi031.stdout:0/595: fsync d6/d12/d97/db3/fb5 0 2024-09-16T08:57:24.233 INFO:tasks.workunit.client.0.smithi031.stdout:4/705: mkdir d0/d2/d16/ddd 0 2024-09-16T08:57:24.233 INFO:tasks.workunit.client.0.smithi031.stdout:4/706: dread - d0/d2/d16/d2b/fd7 zero size 2024-09-16T08:57:24.234 INFO:tasks.workunit.client.0.smithi031.stdout:5/542: dwrite d2/d4b/f8b [0,4194304] 0 2024-09-16T08:57:24.235 INFO:tasks.workunit.client.0.smithi031.stdout:3/656: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 [0,4194304] 0 2024-09-16T08:57:24.235 INFO:tasks.workunit.client.0.smithi031.stdout:0/596: mknod d6/d12/d13/da5/cbd 0 2024-09-16T08:57:24.235 INFO:tasks.workunit.client.0.smithi031.stdout:0/597: write d6/d12/d13/d3c/f45 [4088900,129327] 0 2024-09-16T08:57:24.237 INFO:tasks.workunit.client.0.smithi031.stdout:4/707: mkdir d0/d2/d16/d59/d84/d86/d9b/dcf/dde 0 2024-09-16T08:57:24.240 INFO:tasks.workunit.client.0.smithi031.stdout:3/657: write d3/d14/f8c [3953628,81459] 0 2024-09-16T08:57:24.240 INFO:tasks.workunit.client.0.smithi031.stdout:3/658: creat d3/d14/d37/d1a/d1d/dac/fc1 x:0 0 0 2024-09-16T08:57:24.240 INFO:tasks.workunit.client.0.smithi031.stdout:3/659: write d3/d14/d37/d1a/f8f [1441089,118289] 0 2024-09-16T08:57:24.240 INFO:tasks.workunit.client.0.smithi031.stdout:5/543: creat d2/dc/d36/fae x:0 0 0 2024-09-16T08:57:24.244 INFO:tasks.workunit.client.0.smithi031.stdout:4/708: creat d0/d2/d16/ddd/fdf x:0 0 0 2024-09-16T08:57:24.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/660: mknod d3/d14/d37/d1a/d1d/dac/cc2 0 2024-09-16T08:57:24.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/661: write d3/d14/f2e [3827173,119017] 0 2024-09-16T08:57:24.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/662: read - d3/d14/d37/d1a/d1d/d3b/d60/fb2 zero size 2024-09-16T08:57:24.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/663: truncate d3/d14/d37/f84 501443 0 2024-09-16T08:57:24.248 INFO:tasks.workunit.client.0.smithi031.stdout:3/664: readlink d3/d14/d37/d1a/d24/l4c 0 2024-09-16T08:57:24.248 INFO:tasks.workunit.client.0.smithi031.stdout:5/544: dread d2/dc/d25/d21/f4e [0,4194304] 0 2024-09-16T08:57:24.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/598: rename d6/c29 to d6/d12/d47/cbe 0 2024-09-16T08:57:24.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/599: write d6/f8 [13713,51266] 0 2024-09-16T08:57:24.252 INFO:tasks.workunit.client.0.smithi031.stdout:1/560: dwrite d2/d15/d1f/d43/fb7 [0,4194304] 0 2024-09-16T08:57:24.253 INFO:tasks.workunit.client.0.smithi031.stdout:6/729: dwrite d0/dd/d74/d19/fdf [0,4194304] 0 2024-09-16T08:57:24.253 INFO:tasks.workunit.client.0.smithi031.stdout:6/730: chown d0/dd/d35/d6c/dac/dba/ld3 794287523 1 2024-09-16T08:57:24.255 INFO:tasks.workunit.client.0.smithi031.stdout:5/545: rmdir d2/dc/d25/d21/d81 39 2024-09-16T08:57:24.256 INFO:tasks.workunit.client.0.smithi031.stdout:7/690: dwrite d2/d8/f8c [0,4194304] 0 2024-09-16T08:57:24.257 INFO:tasks.workunit.client.0.smithi031.stdout:5/546: dread d2/dc/d36/f41 [0,4194304] 0 2024-09-16T08:57:24.257 INFO:tasks.workunit.client.0.smithi031.stdout:4/709: write d0/f1 [3968936,87826] 0 2024-09-16T08:57:24.261 INFO:tasks.workunit.client.0.smithi031.stdout:1/561: link d2/d4/d36/l57 d2/lc4 0 2024-09-16T08:57:24.264 INFO:tasks.workunit.client.0.smithi031.stdout:3/665: dwrite d3/d14/d37/d1a/f76 [0,4194304] 0 2024-09-16T08:57:24.264 INFO:tasks.workunit.client.0.smithi031.stdout:3/666: stat d3/d14/fb6 0 2024-09-16T08:57:24.268 INFO:tasks.workunit.client.0.smithi031.stdout:6/731: rmdir d0/dd/d74/d19/d54/dca 0 2024-09-16T08:57:24.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/600: dwrite d6/d12/d13/f68 [0,4194304] 0 2024-09-16T08:57:24.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/601: chown d6/d12/d47/c8b 37 1 2024-09-16T08:57:24.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/602: chown d6/d12/d13/d36/fa6 128542831 1 2024-09-16T08:57:24.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/603: read - d6/d12/d13/d36/fa6 zero size 2024-09-16T08:57:24.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/604: chown d6/d12/d13/l8e 27741232 1 2024-09-16T08:57:24.272 INFO:tasks.workunit.client.0.smithi031.stdout:0/605: truncate d6/d12/d13/f84 1278649 0 2024-09-16T08:57:24.273 INFO:tasks.workunit.client.0.smithi031.stdout:5/547: mkdir d2/dc/d36/daf 0 2024-09-16T08:57:24.273 INFO:tasks.workunit.client.0.smithi031.stdout:5/548: fdatasync d2/dc/d3d/f86 0 2024-09-16T08:57:24.275 INFO:tasks.workunit.client.0.smithi031.stdout:4/710: symlink d0/d8e/d75/le0 0 2024-09-16T08:57:24.275 INFO:tasks.workunit.client.0.smithi031.stdout:4/711: dread - d0/d2/d16/d59/d84/fc4 zero size 2024-09-16T08:57:24.275 INFO:tasks.workunit.client.0.smithi031.stdout:4/712: write d0/d2/d3a/f49 [344606,84152] 0 2024-09-16T08:57:24.276 INFO:tasks.workunit.client.0.smithi031.stdout:1/562: rename d2/f2c to d2/d15/d5d/fc5 0 2024-09-16T08:57:24.277 INFO:tasks.workunit.client.0.smithi031.stdout:6/732: write d0/f4 [1223655,94297] 0 2024-09-16T08:57:24.279 INFO:tasks.workunit.client.0.smithi031.stdout:3/667: mkdir d3/d14/d37/d33/d97/dc3 0 2024-09-16T08:57:24.281 INFO:tasks.workunit.client.0.smithi031.stdout:7/691: rmdir d2/d2d/d68 39 2024-09-16T08:57:24.294 INFO:tasks.workunit.client.0.smithi031.stdout:5/549: creat d2/dc/d3d/fb0 x:0 0 0 2024-09-16T08:57:24.295 INFO:tasks.workunit.client.0.smithi031.stdout:4/713: symlink d0/d2/d16/d59/d84/le1 0 2024-09-16T08:57:24.298 INFO:tasks.workunit.client.0.smithi031.stdout:1/563: unlink d2/d4/d36/f92 0 2024-09-16T08:57:24.298 INFO:tasks.workunit.client.0.smithi031.stdout:1/564: fdatasync d2/d4/d1b/d5c/f8e 0 2024-09-16T08:57:24.298 INFO:tasks.workunit.client.0.smithi031.stdout:1/565: creat d2/d15/d38/fc6 x:0 0 0 2024-09-16T08:57:24.298 INFO:tasks.workunit.client.0.smithi031.stdout:5/550: mknod d2/dc/d25/d21/d81/cb1 0 2024-09-16T08:57:24.298 INFO:tasks.workunit.client.0.smithi031.stdout:5/551: fsync d2/f2b 0 2024-09-16T08:57:24.300 INFO:tasks.workunit.client.0.smithi031.stdout:4/714: unlink d0/d5b/f89 0 2024-09-16T08:57:24.300 INFO:tasks.workunit.client.0.smithi031.stdout:4/715: write d0/d2/d3a/f49 [5329719,70579] 0 2024-09-16T08:57:24.302 INFO:tasks.workunit.client.0.smithi031.stdout:5/552: unlink d2/f34 0 2024-09-16T08:57:24.302 INFO:tasks.workunit.client.0.smithi031.stdout:4/716: dread d0/d2/d16/d59/d84/d86/fa8 [0,4194304] 0 2024-09-16T08:57:24.305 INFO:tasks.workunit.client.0.smithi031.stdout:1/566: link d2/d4/c8a d2/d15/d5d/cc7 0 2024-09-16T08:57:24.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/553: mkdir d2/dc/d3d/db2 0 2024-09-16T08:57:24.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/554: fsync d2/dc/d25/d21/f85 0 2024-09-16T08:57:24.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/555: chown d2/dc/f88 616 1 2024-09-16T08:57:24.307 INFO:tasks.workunit.client.0.smithi031.stdout:5/556: chown d2/dc/d1d/d97 27 1 2024-09-16T08:57:24.308 INFO:tasks.workunit.client.0.smithi031.stdout:4/717: mknod d0/d2/d16/d59/d84/ce2 0 2024-09-16T08:57:24.311 INFO:tasks.workunit.client.0.smithi031.stdout:1/567: write d2/d4/fa2 [1255671,92289] 0 2024-09-16T08:57:24.311 INFO:tasks.workunit.client.0.smithi031.stdout:1/568: dread - d2/d4/d52/f6f zero size 2024-09-16T08:57:24.313 INFO:tasks.workunit.client.0.smithi031.stdout:5/557: getdents d2/dc/d1d 0 2024-09-16T08:57:24.313 INFO:tasks.workunit.client.0.smithi031.stdout:5/558: write d2/dc/d25/d21/fa3 [345989,59835] 0 2024-09-16T08:57:24.313 INFO:tasks.workunit.client.0.smithi031.stdout:5/559: dread - d2/dc/d1d/f3f zero size 2024-09-16T08:57:24.316 INFO:tasks.workunit.client.0.smithi031.stdout:6/733: dwrite d0/dd/d74/d19/d54/fd6 [0,4194304] 0 2024-09-16T08:57:24.316 INFO:tasks.workunit.client.0.smithi031.stdout:6/734: write d0/d10/f58 [1554344,98288] 0 2024-09-16T08:57:24.316 INFO:tasks.workunit.client.0.smithi031.stdout:4/718: creat d0/d2/d3a/fe3 x:0 0 0 2024-09-16T08:57:24.317 INFO:tasks.workunit.client.0.smithi031.stdout:0/606: dwrite d6/d12/d13/f68 [0,4194304] 0 2024-09-16T08:57:24.317 INFO:tasks.workunit.client.0.smithi031.stdout:0/607: write d6/d12/d47/f7e [152901,13599] 0 2024-09-16T08:57:24.318 INFO:tasks.workunit.client.0.smithi031.stdout:6/735: write d0/dd/d74/d5e/d7c/d3f/d92/fa4 [540592,35031] 0 2024-09-16T08:57:24.318 INFO:tasks.workunit.client.0.smithi031.stdout:3/668: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 [4194304,4194304] 0 2024-09-16T08:57:24.318 INFO:tasks.workunit.client.0.smithi031.stdout:6/736: readlink d0/dd/d74/d5e/d7c/lab 0 2024-09-16T08:57:24.318 INFO:tasks.workunit.client.0.smithi031.stdout:3/669: write d3/d14/d37/d1a/d1d/d3b/d60/f9c [846552,47331] 0 2024-09-16T08:57:24.318 INFO:tasks.workunit.client.0.smithi031.stdout:5/560: write d2/dc/d25/d21/d2e/f5c [1131745,769] 0 2024-09-16T08:57:24.318 INFO:tasks.workunit.client.0.smithi031.stdout:7/692: dwrite d2/f6a [0,4194304] 0 2024-09-16T08:57:24.319 INFO:tasks.workunit.client.0.smithi031.stdout:1/569: symlink d2/de/db5/lc8 0 2024-09-16T08:57:24.324 INFO:tasks.workunit.client.0.smithi031.stdout:4/719: mkdir d0/d2/d16/ddd/de4 0 2024-09-16T08:57:24.324 INFO:tasks.workunit.client.0.smithi031.stdout:0/608: unlink d6/f82 0 2024-09-16T08:57:24.335 INFO:tasks.workunit.client.0.smithi031.stdout:6/737: write d0/dd/f22 [1858165,94422] 0 2024-09-16T08:57:24.335 INFO:tasks.workunit.client.0.smithi031.stdout:6/738: fdatasync d0/dd/d74/d19/f26 0 2024-09-16T08:57:24.339 INFO:tasks.workunit.client.0.smithi031.stdout:5/561: creat d2/dc/d3d/db2/fb3 x:0 0 0 2024-09-16T08:57:24.341 INFO:tasks.workunit.client.0.smithi031.stdout:4/720: dwrite d0/d5b/f91 [0,4194304] 0 2024-09-16T08:57:24.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/721: fdatasync d0/d2/d16/d1e/db9/fcc 0 2024-09-16T08:57:24.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/722: write d0/d2/d16/d59/d84/fa2 [410206,111109] 0 2024-09-16T08:57:24.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/723: write d0/d2/d16/d59/d8d/fd4 [634816,129591] 0 2024-09-16T08:57:24.342 INFO:tasks.workunit.client.0.smithi031.stdout:4/724: creat d0/d3c/d6b/fe5 x:0 0 0 2024-09-16T08:57:24.343 INFO:tasks.workunit.client.0.smithi031.stdout:7/693: rename d2/d8/d11/d66/lac to d2/d8/d86/dd4/le3 0 2024-09-16T08:57:24.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/694: fsync d2/d2d/f2e 0 2024-09-16T08:57:24.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/695: write d2/d2d/d68/da2/fe2 [335834,128826] 0 2024-09-16T08:57:24.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/696: write d2/d8/f2f [4190983,26138] 0 2024-09-16T08:57:24.361 INFO:tasks.workunit.client.0.smithi031.stdout:4/725: creat d0/d2/d16/d59/d84/d86/d9b/dcf/dde/fe6 x:0 0 0 2024-09-16T08:57:24.362 INFO:tasks.workunit.client.0.smithi031.stdout:6/739: rename d0/dd/d74/d19/f26 to d0/dd/d74/d19/d54/fe5 0 2024-09-16T08:57:24.362 INFO:tasks.workunit.client.0.smithi031.stdout:6/740: write d0/dd/d74/d67/fb0 [826187,36118] 0 2024-09-16T08:57:24.363 INFO:tasks.workunit.client.0.smithi031.stdout:0/609: truncate d6/f1a 46626 0 2024-09-16T08:57:24.363 INFO:tasks.workunit.client.0.smithi031.stdout:0/610: fsync d6/d12/d13/d36/f96 0 2024-09-16T08:57:24.364 INFO:tasks.workunit.client.0.smithi031.stdout:4/726: symlink d0/le7 0 2024-09-16T08:57:24.366 INFO:tasks.workunit.client.0.smithi031.stdout:5/562: dwrite d2/dc/d25/d21/f45 [4194304,4194304] 0 2024-09-16T08:57:24.367 INFO:tasks.workunit.client.0.smithi031.stdout:0/611: symlink d6/d61/d91/lbf 0 2024-09-16T08:57:24.367 INFO:tasks.workunit.client.0.smithi031.stdout:0/612: chown d6/d12/d13/ca3 0 1 2024-09-16T08:57:24.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/697: dwrite d2/d8/f4a [0,4194304] 0 2024-09-16T08:57:24.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/698: getdents d2/d2d 0 2024-09-16T08:57:24.368 INFO:tasks.workunit.client.0.smithi031.stdout:7/699: readlink d2/d8/d11/l1f 0 2024-09-16T08:57:24.376 INFO:tasks.workunit.client.0.smithi031.stdout:4/727: dread d0/d3c/f7f [0,4194304] 0 2024-09-16T08:57:24.377 INFO:tasks.workunit.client.0.smithi031.stdout:4/728: mknod d0/d2/d16/d2b/ce8 0 2024-09-16T08:57:24.379 INFO:tasks.workunit.client.0.smithi031.stdout:5/563: dread d2/dc/d25/d21/f6d [0,4194304] 0 2024-09-16T08:57:24.380 INFO:tasks.workunit.client.0.smithi031.stdout:5/564: chown d2/f59 405 1 2024-09-16T08:57:24.380 INFO:tasks.workunit.client.0.smithi031.stdout:5/565: creat d2/dc/d91/fb4 x:0 0 0 2024-09-16T08:57:24.380 INFO:tasks.workunit.client.0.smithi031.stdout:4/729: mknod d0/d5b/db8/ce9 0 2024-09-16T08:57:24.380 INFO:tasks.workunit.client.0.smithi031.stdout:4/730: fsync d0/d2/f33 0 2024-09-16T08:57:24.382 INFO:tasks.workunit.client.0.smithi031.stdout:6/741: dwrite d0/dd/d74/d19/d9a/fc0 [0,4194304] 0 2024-09-16T08:57:24.387 INFO:tasks.workunit.client.0.smithi031.stdout:4/731: mknod d0/d2/d16/d1e/dbe/cea 0 2024-09-16T08:57:24.391 INFO:tasks.workunit.client.0.smithi031.stdout:0/613: dwrite d6/d12/d13/f7f [0,4194304] 0 2024-09-16T08:57:24.394 INFO:tasks.workunit.client.0.smithi031.stdout:5/566: write d2/dc/d25/d21/d2e/f71 [1345435,52148] 0 2024-09-16T08:57:24.396 INFO:tasks.workunit.client.0.smithi031.stdout:4/732: dread d0/d2/d16/d59/d84/d86/d9b/fa4 [0,4194304] 0 2024-09-16T08:57:24.396 INFO:tasks.workunit.client.0.smithi031.stdout:5/567: rename d2/dc/d36/d76/l9b to d2/dc/d3d/d4c/lb5 0 2024-09-16T08:57:24.397 INFO:tasks.workunit.client.0.smithi031.stdout:5/568: read d2/dc/d36/f41 [1362,113917] 0 2024-09-16T08:57:24.397 INFO:tasks.workunit.client.0.smithi031.stdout:4/733: creat d0/db5/feb x:0 0 0 2024-09-16T08:57:24.421 INFO:tasks.workunit.client.0.smithi031.stdout:4/734: dwrite d0/d2/f1a [0,4194304] 0 2024-09-16T08:57:24.421 INFO:tasks.workunit.client.0.smithi031.stdout:5/569: dwrite d2/f7f [0,4194304] 0 2024-09-16T08:57:24.421 INFO:tasks.workunit.client.0.smithi031.stdout:5/570: write d2/dc/d25/d21/d2e/f8f [125970,59017] 0 2024-09-16T08:57:24.422 INFO:tasks.workunit.client.0.smithi031.stdout:4/735: unlink d0/d2/l8b 0 2024-09-16T08:57:24.422 INFO:tasks.workunit.client.0.smithi031.stdout:4/736: fdatasync d0/d2/d16/d1e/fb6 0 2024-09-16T08:57:24.431 INFO:tasks.workunit.client.0.smithi031.stdout:5/571: read d2/f63 [74646,120149] 0 2024-09-16T08:57:24.480 INFO:tasks.workunit.client.0.smithi031.stdout:5/572: symlink d2/dc/lb6 0 2024-09-16T08:57:24.483 INFO:tasks.workunit.client.0.smithi031.stdout:4/737: dwrite d0/f39 [0,4194304] 0 2024-09-16T08:57:24.483 INFO:tasks.workunit.client.0.smithi031.stdout:5/573: fdatasync d2/dc/d25/d21/d2e/f71 0 2024-09-16T08:57:24.485 INFO:tasks.workunit.client.0.smithi031.stdout:4/738: mknod d0/d3c/d6b/d9c/cec 0 2024-09-16T08:57:24.487 INFO:tasks.workunit.client.0.smithi031.stdout:5/574: rename d2/dc/d3d/d4c/l78 to d2/dc/d1d/d97/lb7 0 2024-09-16T08:57:24.489 INFO:tasks.workunit.client.0.smithi031.stdout:4/739: symlink d0/d3c/led 0 2024-09-16T08:57:24.489 INFO:tasks.workunit.client.0.smithi031.stdout:4/740: chown d0/d2/d3a/f49 147427 1 2024-09-16T08:57:24.498 INFO:tasks.workunit.client.0.smithi031.stdout:4/741: dread d0/d2/d3a/f97 [0,4194304] 0 2024-09-16T08:57:24.500 INFO:tasks.workunit.client.0.smithi031.stdout:4/742: creat d0/d3c/d6b/d9c/fee x:0 0 0 2024-09-16T08:57:24.500 INFO:tasks.workunit.client.0.smithi031.stdout:4/743: dread - d0/d2/d16/d59/d84/d86/fc3 zero size 2024-09-16T08:57:24.500 INFO:tasks.workunit.client.0.smithi031.stdout:4/744: write d0/d2/d16/d1e/f3b [922136,90625] 0 2024-09-16T08:57:24.500 INFO:tasks.workunit.client.0.smithi031.stdout:4/745: write d0/d2/d16/f19 [792691,76782] 0 2024-09-16T08:57:24.501 INFO:tasks.workunit.client.0.smithi031.stdout:5/575: dwrite d2/dc/d25/d21/d2e/fa5 [0,4194304] 0 2024-09-16T08:57:24.505 INFO:tasks.workunit.client.0.smithi031.stdout:4/746: dread d0/d2/f10 [0,4194304] 0 2024-09-16T08:57:24.508 INFO:tasks.workunit.client.0.smithi031.stdout:5/576: mknod d2/dc/d91/cb8 0 2024-09-16T08:57:24.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/577: creat d2/dc/d1d/d97/fb9 x:0 0 0 2024-09-16T08:57:24.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/578: creat d2/dc/fba x:0 0 0 2024-09-16T08:57:24.511 INFO:tasks.workunit.client.0.smithi031.stdout:5/579: read - d2/dc/f93 zero size 2024-09-16T08:57:24.517 INFO:tasks.workunit.client.0.smithi031.stdout:5/580: dread d2/dc/f43 [0,4194304] 0 2024-09-16T08:57:24.518 INFO:tasks.workunit.client.0.smithi031.stdout:5/581: mkdir d2/dc/d1d/dbb 0 2024-09-16T08:57:24.521 INFO:tasks.workunit.client.0.smithi031.stdout:5/582: mknod d2/cbc 0 2024-09-16T08:57:24.521 INFO:tasks.workunit.client.0.smithi031.stdout:5/583: readlink d2/dc/d3d/l95 0 2024-09-16T08:57:24.524 INFO:tasks.workunit.client.0.smithi031.stdout:4/747: dwrite d0/db5/fb7 [0,4194304] 0 2024-09-16T08:57:24.535 INFO:tasks.workunit.client.0.smithi031.stdout:4/748: mkdir d0/d2/d16/d1e/dbe/def 0 2024-09-16T08:57:24.536 INFO:tasks.workunit.client.0.smithi031.stdout:4/749: mknod d0/cf0 0 2024-09-16T08:57:24.538 INFO:tasks.workunit.client.0.smithi031.stdout:4/750: creat d0/ff1 x:0 0 0 2024-09-16T08:57:24.539 INFO:tasks.workunit.client.0.smithi031.stdout:4/751: unlink d0/d8e/d75/fc6 0 2024-09-16T08:57:24.557 INFO:tasks.workunit.client.0.smithi031.stdout:4/752: dwrite d0/d5b/f91 [0,4194304] 0 2024-09-16T08:57:24.564 INFO:tasks.workunit.client.0.smithi031.stdout:4/753: mkdir d0/d3c/df2 0 2024-09-16T08:57:24.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/754: unlink d0/d2/d16/l37 0 2024-09-16T08:57:24.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/755: fsync d0/d5b/f7d 0 2024-09-16T08:57:24.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/756: chown d0/d2/d16/d1e/fa1 49407 1 2024-09-16T08:57:24.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/757: stat d0/d2/d16/f22 0 2024-09-16T08:57:24.567 INFO:tasks.workunit.client.0.smithi031.stdout:4/758: symlink d0/d8e/d75/lf3 0 2024-09-16T08:57:24.567 INFO:tasks.workunit.client.0.smithi031.stdout:4/759: dread - d0/d2/d16/d2b/fd7 zero size 2024-09-16T08:57:24.567 INFO:tasks.workunit.client.0.smithi031.stdout:4/760: dread - d0/d2/d16/ddd/fdf zero size 2024-09-16T08:57:24.567 INFO:tasks.workunit.client.0.smithi031.stdout:4/761: mkdir d0/d2/d16/ddd/de4/df4 0 2024-09-16T08:57:24.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/762: stat d0/d2/d3a/fb4 0 2024-09-16T08:57:24.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/763: truncate d0/d2/d16/d59/d84/d86/f8f 4384398 0 2024-09-16T08:57:24.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/764: getdents d0/d5b/db8 0 2024-09-16T08:57:24.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/765: readlink d0/d2/d16/d59/d84/d86/d9b/dcf/ld8 0 2024-09-16T08:57:24.568 INFO:tasks.workunit.client.0.smithi031.stdout:4/766: write d0/d2/d16/d59/d84/d86/fa8 [1221080,103167] 0 2024-09-16T08:57:24.575 INFO:tasks.workunit.client.0.smithi031.stdout:4/767: read d0/d2/d3a/fac [819046,70792] 0 2024-09-16T08:57:24.581 INFO:tasks.workunit.client.0.smithi031.stdout:4/768: getdents d0/d8e/d75 0 2024-09-16T08:57:24.583 INFO:tasks.workunit.client.0.smithi031.stdout:4/769: symlink d0/d8e/lf5 0 2024-09-16T08:57:24.585 INFO:tasks.workunit.client.0.smithi031.stdout:4/770: rename d0/l45 to d0/d2/d3a/lf6 0 2024-09-16T08:57:24.586 INFO:tasks.workunit.client.0.smithi031.stdout:4/771: creat d0/d2/d16/d59/dd5/ff7 x:0 0 0 2024-09-16T08:57:24.587 INFO:tasks.workunit.client.0.smithi031.stdout:4/772: mkdir d0/d5b/db8/df8 0 2024-09-16T08:57:24.589 INFO:tasks.workunit.client.0.smithi031.stdout:4/773: rmdir d0/d3c/d6b/d9c 39 2024-09-16T08:57:24.589 INFO:tasks.workunit.client.0.smithi031.stdout:4/774: truncate d0/d2/d16/d59/d84/d86/fc3 605772 0 2024-09-16T08:57:24.589 INFO:tasks.workunit.client.0.smithi031.stdout:4/775: creat d0/d2/d16/d1e/db9/ff9 x:0 0 0 2024-09-16T08:57:24.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/776: symlink d0/d2/d16/d59/d84/d86/d9b/lfa 0 2024-09-16T08:57:24.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/777: creat d0/d8e/ffb x:0 0 0 2024-09-16T08:57:24.592 INFO:tasks.workunit.client.0.smithi031.stdout:4/778: stat d0/ff1 0 2024-09-16T08:57:24.614 INFO:tasks.workunit.client.0.smithi031.stdout:4/779: dwrite d0/d2/d3a/f97 [0,4194304] 0 2024-09-16T08:57:24.768 INFO:tasks.workunit.client.0.smithi031.stdout:2/611: sync 2024-09-16T08:57:24.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/612: creat d6/da/d5e/d72/fcb x:0 0 0 2024-09-16T08:57:24.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/613: chown d6/da/d37/d8b/dad/laf 19 1 2024-09-16T08:57:24.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/614: fsync d6/d2e/d40/d58/f90 0 2024-09-16T08:57:24.771 INFO:tasks.workunit.client.0.smithi031.stdout:2/615: dread - d6/d2e/d40/f78 zero size 2024-09-16T08:57:24.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/616: mkdir d6/da/d5e/d8d/dcc 0 2024-09-16T08:57:24.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/617: creat d6/d20/fcd x:0 0 0 2024-09-16T08:57:24.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/618: rename d6/l99 to d6/d2e/d40/d6c/d2c/d53/lce 0 2024-09-16T08:57:24.779 INFO:tasks.workunit.client.0.smithi031.stdout:2/619: mkdir d6/d20/da1/dcf 0 2024-09-16T08:57:24.779 INFO:tasks.workunit.client.0.smithi031.stdout:2/620: write d6/d2e/d40/f94 [74594,106602] 0 2024-09-16T08:57:24.796 INFO:tasks.workunit.client.0.smithi031.stdout:2/621: dwrite d6/d2e/d40/f78 [0,4194304] 0 2024-09-16T08:57:24.805 INFO:tasks.workunit.client.0.smithi031.stdout:2/622: getdents d6/d2e/d40/d6c/d2c/d53 0 2024-09-16T08:57:24.807 INFO:tasks.workunit.client.0.smithi031.stdout:2/623: creat d6/d20/da1/dcf/fd0 x:0 0 0 2024-09-16T08:57:24.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/624: mkdir d6/da/d5e/dd1 0 2024-09-16T08:57:24.808 INFO:tasks.workunit.client.0.smithi031.stdout:2/625: rename d6/d2e/d40/d6c to d6/d2e/d40/d6c/d2c/dd2 22 2024-09-16T08:57:24.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/626: mknod d6/da/d4b/d4c/d73/cd3 0 2024-09-16T08:57:24.819 INFO:tasks.workunit.client.0.smithi031.stdout:4/780: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:0/614: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:6/742: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:9/601: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:4/781: dread - d0/d2/d16/ddd/fdf zero size 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:1/570: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:0/615: creat d6/d12/d13/da5/fc0 x:0 0 0 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:3/670: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:0/616: chown d6/d12 841334 1 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:7/700: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/525: sync 2024-09-16T08:57:24.820 INFO:tasks.workunit.client.0.smithi031.stdout:3/671: write d3/d14/d37/d1a/d1d/d3b/d3f/d49/faa [807644,97830] 0 2024-09-16T08:57:24.821 INFO:tasks.workunit.client.0.smithi031.stdout:8/526: read - d4/d2d/d6e/fab zero size 2024-09-16T08:57:24.821 INFO:tasks.workunit.client.0.smithi031.stdout:5/584: sync 2024-09-16T08:57:24.821 INFO:tasks.workunit.client.0.smithi031.stdout:5/585: fsync d2/dc/d1d/f92 0 2024-09-16T08:57:24.821 INFO:tasks.workunit.client.0.smithi031.stdout:5/586: read d2/dc/d36/f94 [2281450,98615] 0 2024-09-16T08:57:24.821 INFO:tasks.workunit.client.0.smithi031.stdout:0/617: write d6/d12/f6c [433306,45739] 0 2024-09-16T08:57:24.821 INFO:tasks.workunit.client.0.smithi031.stdout:9/602: mkdir d3/d5/dd6 0 2024-09-16T08:57:24.822 INFO:tasks.workunit.client.0.smithi031.stdout:2/627: dwrite d6/da/d37/f87 [0,4194304] 0 2024-09-16T08:57:24.822 INFO:tasks.workunit.client.0.smithi031.stdout:2/628: truncate d6/d2e/d40/d6c/d2c/d53/d60/f35 1219274 0 2024-09-16T08:57:24.823 INFO:tasks.workunit.client.0.smithi031.stdout:6/743: rename d0/lb9 to d0/dd/d35/d6c/d83/le6 0 2024-09-16T08:57:24.827 INFO:tasks.workunit.client.0.smithi031.stdout:1/571: write d2/d4/d67/fbc [9399,97530] 0 2024-09-16T08:57:24.830 INFO:tasks.workunit.client.0.smithi031.stdout:8/527: symlink d4/d7/d21/d37/d46/db1/lb3 0 2024-09-16T08:57:24.833 INFO:tasks.workunit.client.0.smithi031.stdout:5/587: creat d2/dc/d3d/d4c/fbd x:0 0 0 2024-09-16T08:57:24.833 INFO:tasks.workunit.client.0.smithi031.stdout:5/588: dread - d2/d4b/f79 zero size 2024-09-16T08:57:24.834 INFO:tasks.workunit.client.0.smithi031.stdout:0/618: creat d6/d12/d47/d7c/fc1 x:0 0 0 2024-09-16T08:57:24.840 INFO:tasks.workunit.client.0.smithi031.stdout:8/528: read d4/d5b/d7b/f8e [1020407,53566] 0 2024-09-16T08:57:24.840 INFO:tasks.workunit.client.0.smithi031.stdout:9/603: mkdir d3/d5/d6a/d74/da8/dc3/dbc/dd7 0 2024-09-16T08:57:24.841 INFO:tasks.workunit.client.0.smithi031.stdout:9/604: dread - d3/d5/d73/d81/fca zero size 2024-09-16T08:57:24.845 INFO:tasks.workunit.client.0.smithi031.stdout:2/629: mknod d6/da/d5e/dd1/cd4 0 2024-09-16T08:57:24.845 INFO:tasks.workunit.client.0.smithi031.stdout:6/744: creat d0/d10/dc8/fe7 x:0 0 0 2024-09-16T08:57:24.846 INFO:tasks.workunit.client.0.smithi031.stdout:6/745: stat d0/d10/f5a 0 2024-09-16T08:57:24.851 INFO:tasks.workunit.client.0.smithi031.stdout:4/782: rmdir d0/d2/d16/d59 39 2024-09-16T08:57:24.851 INFO:tasks.workunit.client.0.smithi031.stdout:4/783: chown d0/d2/d16/d1e/c2f 113 1 2024-09-16T08:57:24.851 INFO:tasks.workunit.client.0.smithi031.stdout:4/784: chown d0/d2/f3d 0 1 2024-09-16T08:57:24.852 INFO:tasks.workunit.client.0.smithi031.stdout:1/572: symlink d2/d4/d1b/lc9 0 2024-09-16T08:57:24.853 INFO:tasks.workunit.client.0.smithi031.stdout:3/672: mkdir d3/d14/d37/d1a/d1d/d3b/dc4 0 2024-09-16T08:57:24.854 INFO:tasks.workunit.client.0.smithi031.stdout:0/619: mknod d6/d12/d97/cc2 0 2024-09-16T08:57:24.854 INFO:tasks.workunit.client.0.smithi031.stdout:9/605: readlink d3/l9e 0 2024-09-16T08:57:24.857 INFO:tasks.workunit.client.0.smithi031.stdout:6/746: creat d0/d8e/fe8 x:0 0 0 2024-09-16T08:57:24.857 INFO:tasks.workunit.client.0.smithi031.stdout:6/747: write d0/dd/d74/d67/fe0 [901785,46416] 0 2024-09-16T08:57:24.859 INFO:tasks.workunit.client.0.smithi031.stdout:4/785: creat d0/d3c/df2/ffc x:0 0 0 2024-09-16T08:57:24.861 INFO:tasks.workunit.client.0.smithi031.stdout:4/786: write d0/d2/d16/d59/d84/d86/fc3 [406723,63183] 0 2024-09-16T08:57:24.861 INFO:tasks.workunit.client.0.smithi031.stdout:3/673: truncate d3/f2a 202300 0 2024-09-16T08:57:24.861 INFO:tasks.workunit.client.0.smithi031.stdout:4/787: fsync d0/d2/d16/d59/fd3 0 2024-09-16T08:57:24.865 INFO:tasks.workunit.client.0.smithi031.stdout:3/674: mknod d3/d14/d37/d1a/d24/cc5 0 2024-09-16T08:57:24.870 INFO:tasks.workunit.client.0.smithi031.stdout:3/675: symlink d3/d14/d37/d33/d97/lc6 0 2024-09-16T08:57:24.871 INFO:tasks.workunit.client.0.smithi031.stdout:3/676: write d3/f4 [3515485,123036] 0 2024-09-16T08:57:24.871 INFO:tasks.workunit.client.0.smithi031.stdout:3/677: stat d3/f5 0 2024-09-16T08:57:24.873 INFO:tasks.workunit.client.0.smithi031.stdout:3/678: dread d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 [0,4194304] 0 2024-09-16T08:57:24.873 INFO:tasks.workunit.client.0.smithi031.stdout:8/529: dwrite d4/f17 [4194304,4194304] 0 2024-09-16T08:57:24.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/589: dwrite d2/d4b/f8a [0,4194304] 0 2024-09-16T08:57:24.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/590: dread - d2/dc/d36/d76/f9a zero size 2024-09-16T08:57:24.877 INFO:tasks.workunit.client.0.smithi031.stdout:1/573: write d2/d15/d32/f55 [143181,14648] 0 2024-09-16T08:57:24.877 INFO:tasks.workunit.client.0.smithi031.stdout:1/574: creat d2/d4/d52/fca x:0 0 0 2024-09-16T08:57:24.879 INFO:tasks.workunit.client.0.smithi031.stdout:3/679: rename d3/d14/d37/c56 to d3/d14/d6f/d82/d96/cc7 0 2024-09-16T08:57:24.883 INFO:tasks.workunit.client.0.smithi031.stdout:0/620: dwrite d6/f1e [0,4194304] 0 2024-09-16T08:57:24.883 INFO:tasks.workunit.client.0.smithi031.stdout:5/591: unlink d2/cbc 0 2024-09-16T08:57:24.883 INFO:tasks.workunit.client.0.smithi031.stdout:5/592: chown d2/d4b/f8b 2 1 2024-09-16T08:57:24.889 INFO:tasks.workunit.client.0.smithi031.stdout:8/530: write d4/d7/d21/d24/f31 [2554462,58817] 0 2024-09-16T08:57:24.891 INFO:tasks.workunit.client.0.smithi031.stdout:0/621: creat d6/d61/d91/fc3 x:0 0 0 2024-09-16T08:57:24.893 INFO:tasks.workunit.client.0.smithi031.stdout:5/593: mknod d2/dc/d91/cbe 0 2024-09-16T08:57:24.893 INFO:tasks.workunit.client.0.smithi031.stdout:4/788: dwrite d0/d71/f81 [0,4194304] 0 2024-09-16T08:57:24.894 INFO:tasks.workunit.client.0.smithi031.stdout:8/531: dread d4/d7/d21/d24/f86 [0,4194304] 0 2024-09-16T08:57:24.894 INFO:tasks.workunit.client.0.smithi031.stdout:9/606: dwrite d3/d48/f9d [0,4194304] 0 2024-09-16T08:57:24.898 INFO:tasks.workunit.client.0.smithi031.stdout:0/622: getdents d6/d12/d13/d3c 0 2024-09-16T08:57:24.898 INFO:tasks.workunit.client.0.smithi031.stdout:6/748: dwrite d0/d10/f3b [4194304,4194304] 0 2024-09-16T08:57:24.898 INFO:tasks.workunit.client.0.smithi031.stdout:5/594: mkdir d2/dc/d25/dbf 0 2024-09-16T08:57:24.904 INFO:tasks.workunit.client.0.smithi031.stdout:5/595: creat d2/dc/d25/dbf/fc0 x:0 0 0 2024-09-16T08:57:24.906 INFO:tasks.workunit.client.0.smithi031.stdout:4/789: getdents d0/d3c 0 2024-09-16T08:57:24.906 INFO:tasks.workunit.client.0.smithi031.stdout:4/790: chown d0/d3c/d6b/fda 59048503 1 2024-09-16T08:57:24.907 INFO:tasks.workunit.client.0.smithi031.stdout:5/596: creat d2/dc/d25/fc1 x:0 0 0 2024-09-16T08:57:24.907 INFO:tasks.workunit.client.0.smithi031.stdout:5/597: creat d2/dc/d3d/fc2 x:0 0 0 2024-09-16T08:57:24.910 INFO:tasks.workunit.client.0.smithi031.stdout:5/598: mkdir d2/dc/d25/d21/d81/dc3 0 2024-09-16T08:57:24.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/791: dread d0/d3c/f7f [0,4194304] 0 2024-09-16T08:57:24.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/792: fdatasync d0/d2/d16/d1e/f52 0 2024-09-16T08:57:24.913 INFO:tasks.workunit.client.0.smithi031.stdout:4/793: dread - d0/d8e/fc2 zero size 2024-09-16T08:57:24.914 INFO:tasks.workunit.client.0.smithi031.stdout:4/794: mkdir d0/d2/d16/d2b/dfd 0 2024-09-16T08:57:24.915 INFO:tasks.workunit.client.0.smithi031.stdout:4/795: write d0/d2/d3a/fb4 [901922,126256] 0 2024-09-16T08:57:24.916 INFO:tasks.workunit.client.0.smithi031.stdout:4/796: mkdir d0/d2/d16/d2b/dfe 0 2024-09-16T08:57:24.919 INFO:tasks.workunit.client.0.smithi031.stdout:4/797: rename d0/d2/d16/d1e/dbe/def to d0/d2/d16/d59/dd5/dff 0 2024-09-16T08:57:24.924 INFO:tasks.workunit.client.0.smithi031.stdout:4/798: mkdir d0/d2/d16/d59/dd5/dff/d100 0 2024-09-16T08:57:24.926 INFO:tasks.workunit.client.0.smithi031.stdout:4/799: creat d0/d2/d16/f101 x:0 0 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:1/575: dwrite d2/de/f2f [0,4194304] 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:3/680: dwrite d3/d14/d37/d1a/f8f [0,4194304] 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:3/681: dread - d3/d14/d37/d1a/f65 zero size 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/532: dwrite d4/d7/f51 [0,4194304] 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:3/682: write d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 [1940778,112786] 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:3/683: write d3/d14/d6f/d82/d96/f98 [613440,55549] 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:3/684: creat d3/d14/d6f/fc8 x:0 0 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:9/607: dwrite d3/d5/da/d34/d3c/d5e/d9c/d95/fd2 [0,4194304] 0 2024-09-16T08:57:24.938 INFO:tasks.workunit.client.0.smithi031.stdout:9/608: write d3/d48/f68 [2475317,73916] 0 2024-09-16T08:57:24.939 INFO:tasks.workunit.client.0.smithi031.stdout:0/623: dwrite d6/d12/f77 [0,4194304] 0 2024-09-16T08:57:24.940 INFO:tasks.workunit.client.0.smithi031.stdout:5/599: dwrite d2/dc/d25/d21/fac [0,4194304] 0 2024-09-16T08:57:24.940 INFO:tasks.workunit.client.0.smithi031.stdout:5/600: chown d2/d4b/f8a 845469461 1 2024-09-16T08:57:24.940 INFO:tasks.workunit.client.0.smithi031.stdout:1/576: mkdir d2/d15/d5d/db4/da3/dcb 0 2024-09-16T08:57:24.944 INFO:tasks.workunit.client.0.smithi031.stdout:8/533: creat d4/fb4 x:0 0 0 2024-09-16T08:57:24.946 INFO:tasks.workunit.client.0.smithi031.stdout:3/685: unlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/f69 0 2024-09-16T08:57:24.955 INFO:tasks.workunit.client.0.smithi031.stdout:9/609: link d3/d5/da/d34/cb9 d3/d5/da/d34/d3c/d5e/d9c/cd8 0 2024-09-16T08:57:24.964 INFO:tasks.workunit.client.0.smithi031.stdout:0/624: mkdir d6/d61/d91/dc4 0 2024-09-16T08:57:24.966 INFO:tasks.workunit.client.0.smithi031.stdout:8/534: truncate d4/f6 2274654 0 2024-09-16T08:57:24.966 INFO:tasks.workunit.client.0.smithi031.stdout:8/535: write d4/d2f/f41 [4373361,91626] 0 2024-09-16T08:57:24.967 INFO:tasks.workunit.client.0.smithi031.stdout:3/686: symlink d3/d14/d37/d1a/d1d/d6a/lc9 0 2024-09-16T08:57:24.970 INFO:tasks.workunit.client.0.smithi031.stdout:9/610: creat d3/d5/d6a/d74/da8/dc3/dbc/dd7/fd9 x:0 0 0 2024-09-16T08:57:24.970 INFO:tasks.workunit.client.0.smithi031.stdout:9/611: fsync d3/d5/da/d34/d3c/d5e/d9c/f9f 0 2024-09-16T08:57:24.970 INFO:tasks.workunit.client.0.smithi031.stdout:9/612: write d3/d5/db3/fb8 [758260,128497] 0 2024-09-16T08:57:24.971 INFO:tasks.workunit.client.0.smithi031.stdout:0/625: mkdir d6/d12/d47/dc5 0 2024-09-16T08:57:24.971 INFO:tasks.workunit.client.0.smithi031.stdout:0/626: chown d6/d12/f37 608 1 2024-09-16T08:57:24.974 INFO:tasks.workunit.client.0.smithi031.stdout:0/627: mkdir d6/d12/d97/dc6 0 2024-09-16T08:57:24.977 INFO:tasks.workunit.client.0.smithi031.stdout:3/687: dread d3/d14/f2e [0,4194304] 0 2024-09-16T08:57:24.979 INFO:tasks.workunit.client.0.smithi031.stdout:3/688: dread d3/d14/d37/d1a/d1d/f92 [0,4194304] 0 2024-09-16T08:57:24.981 INFO:tasks.workunit.client.0.smithi031.stdout:4/800: rename d0/d2/d16/d59 to d0/d102 0 2024-09-16T08:57:24.983 INFO:tasks.workunit.client.0.smithi031.stdout:1/577: rename d2/d15/d32/f55 to d2/d4/d52/fcc 0 2024-09-16T08:57:24.985 INFO:tasks.workunit.client.0.smithi031.stdout:4/801: unlink d0/db5/cbc 0 2024-09-16T08:57:24.987 INFO:tasks.workunit.client.0.smithi031.stdout:8/536: rename d4/d77/fae to d4/fb5 0 2024-09-16T08:57:24.988 INFO:tasks.workunit.client.0.smithi031.stdout:8/537: dread d4/fb5 [0,4194304] 0 2024-09-16T08:57:24.989 INFO:tasks.workunit.client.0.smithi031.stdout:1/578: mknod d2/d4/d1b/d5c/d6b/ccd 0 2024-09-16T08:57:24.992 INFO:tasks.workunit.client.0.smithi031.stdout:8/538: rename d4/d7/d21/d37/d46/db1/fa0 to d4/d7/d99/fb6 0 2024-09-16T08:57:24.995 INFO:tasks.workunit.client.0.smithi031.stdout:8/539: link d4/d7/d21/d37/d46/d8c/f58 d4/fb7 0 2024-09-16T08:57:24.995 INFO:tasks.workunit.client.0.smithi031.stdout:8/540: write d4/d2d/faf [748016,78878] 0 2024-09-16T08:57:24.997 INFO:tasks.workunit.client.0.smithi031.stdout:4/802: read d0/d2/d16/f4e [2661421,114319] 0 2024-09-16T08:57:24.997 INFO:tasks.workunit.client.0.smithi031.stdout:4/803: chown d0/f38 26196 1 2024-09-16T08:57:24.998 INFO:tasks.workunit.client.0.smithi031.stdout:9/613: dwrite d3/d5/da/d34/f5c [4194304,4194304] 0 2024-09-16T08:57:24.998 INFO:tasks.workunit.client.0.smithi031.stdout:0/628: dwrite d6/d12/f16 [0,4194304] 0 2024-09-16T08:57:25.001 INFO:tasks.workunit.client.0.smithi031.stdout:8/541: truncate d4/d9e/d38/f3c 3991863 0 2024-09-16T08:57:25.001 INFO:tasks.workunit.client.0.smithi031.stdout:8/542: write d4/d77/f9b [227552,21069] 0 2024-09-16T08:57:25.003 INFO:tasks.workunit.client.0.smithi031.stdout:4/804: rename d0/d102/d8d/fd4 to d0/d2/d16/ddd/de4/f103 0 2024-09-16T08:57:25.003 INFO:tasks.workunit.client.0.smithi031.stdout:4/805: readlink d0/d2/d16/d2b/daf/ld1 0 2024-09-16T08:57:25.006 INFO:tasks.workunit.client.0.smithi031.stdout:9/614: symlink d3/d5/db3/lda 0 2024-09-16T08:57:25.006 INFO:tasks.workunit.client.0.smithi031.stdout:1/579: write d2/de/f79 [4500137,102001] 0 2024-09-16T08:57:25.007 INFO:tasks.workunit.client.0.smithi031.stdout:8/543: mkdir d4/db8 0 2024-09-16T08:57:25.009 INFO:tasks.workunit.client.0.smithi031.stdout:4/806: mknod d0/d102/d84/d86/d9b/dcf/dde/c104 0 2024-09-16T08:57:25.011 INFO:tasks.workunit.client.0.smithi031.stdout:9/615: link d3/d5/d73/f9b d3/d5/dd6/fdb 0 2024-09-16T08:57:25.011 INFO:tasks.workunit.client.0.smithi031.stdout:9/616: read - d3/d5/f5d zero size 2024-09-16T08:57:25.011 INFO:tasks.workunit.client.0.smithi031.stdout:9/617: fdatasync d3/d5/da/f4f 0 2024-09-16T08:57:25.014 INFO:tasks.workunit.client.0.smithi031.stdout:1/580: fsync d2/d4/f82 0 2024-09-16T08:57:25.014 INFO:tasks.workunit.client.0.smithi031.stdout:1/581: dread - d2/d15/d4a/f5b zero size 2024-09-16T08:57:25.016 INFO:tasks.workunit.client.0.smithi031.stdout:4/807: mkdir d0/d102/dd5/dff/d105 0 2024-09-16T08:57:25.016 INFO:tasks.workunit.client.0.smithi031.stdout:4/808: chown d0/d102/d84/d86/d9b/dcf/ld8 241336242 1 2024-09-16T08:57:25.017 INFO:tasks.workunit.client.0.smithi031.stdout:8/544: dread d4/d9e/d38/f9c [0,4194304] 0 2024-09-16T08:57:25.019 INFO:tasks.workunit.client.0.smithi031.stdout:1/582: truncate d2/d15/d1f/d43/f4c 3298507 0 2024-09-16T08:57:25.019 INFO:tasks.workunit.client.0.smithi031.stdout:1/583: truncate d2/d4/d1b/d5c/f8e 1622272 0 2024-09-16T08:57:25.019 INFO:tasks.workunit.client.0.smithi031.stdout:1/584: dread - d2/d15/d5d/db4/fc3 zero size 2024-09-16T08:57:25.025 INFO:tasks.workunit.client.0.smithi031.stdout:0/629: dwrite d6/d12/f6c [0,4194304] 0 2024-09-16T08:57:25.031 INFO:tasks.workunit.client.0.smithi031.stdout:0/630: dread - d6/f6e zero size 2024-09-16T08:57:25.031 INFO:tasks.workunit.client.0.smithi031.stdout:9/618: dread d3/d5/da/f14 [0,4194304] 0 2024-09-16T08:57:25.031 INFO:tasks.workunit.client.0.smithi031.stdout:9/619: write f2 [1336666,68002] 0 2024-09-16T08:57:25.031 INFO:tasks.workunit.client.0.smithi031.stdout:0/631: write d6/d12/d13/d36/f51 [4285497,5649] 0 2024-09-16T08:57:25.035 INFO:tasks.workunit.client.0.smithi031.stdout:8/545: dwrite d4/d2d/d6e/fab [0,4194304] 0 2024-09-16T08:57:25.038 INFO:tasks.workunit.client.0.smithi031.stdout:1/585: dwrite d2/d4/d1b/d5c/f8e [0,4194304] 0 2024-09-16T08:57:25.039 INFO:tasks.workunit.client.0.smithi031.stdout:1/586: fsync d2/de/db5/fc2 0 2024-09-16T08:57:25.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/809: rename d0/d2/d3a to d0/d3c/df2/d106 0 2024-09-16T08:57:25.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/810: write d0/d2/d16/d1e/f23 [2519529,21471] 0 2024-09-16T08:57:25.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/811: chown d0/d3c/d6b/d9c/lad 1135 1 2024-09-16T08:57:25.045 INFO:tasks.workunit.client.0.smithi031.stdout:9/620: truncate d3/d5/da/db/d21/f41 71090 0 2024-09-16T08:57:25.045 INFO:tasks.workunit.client.0.smithi031.stdout:4/812: write d0/d2/fe [1856429,88403] 0 2024-09-16T08:57:25.045 INFO:tasks.workunit.client.0.smithi031.stdout:4/813: write d0/d3c/f62 [2175232,45802] 0 2024-09-16T08:57:25.048 INFO:tasks.workunit.client.0.smithi031.stdout:0/632: mknod d6/d12/cc7 0 2024-09-16T08:57:25.049 INFO:tasks.workunit.client.0.smithi031.stdout:0/633: fsync d6/d12/d13/f6f 0 2024-09-16T08:57:25.053 INFO:tasks.workunit.client.0.smithi031.stdout:8/546: stat d4/d7/d21/d24/l83 0 2024-09-16T08:57:25.060 INFO:tasks.workunit.client.0.smithi031.stdout:1/587: creat d2/d15/d9c/fce x:0 0 0 2024-09-16T08:57:25.064 INFO:tasks.workunit.client.0.smithi031.stdout:4/814: symlink d0/d3c/df2/l107 0 2024-09-16T08:57:25.066 INFO:tasks.workunit.client.0.smithi031.stdout:0/634: truncate d6/d12/d13/d3c/f6d 3390238 0 2024-09-16T08:57:25.070 INFO:tasks.workunit.client.0.smithi031.stdout:1/588: write d2/d4/f28 [1996585,12438] 0 2024-09-16T08:57:25.070 INFO:tasks.workunit.client.0.smithi031.stdout:1/589: write d2/f3c [4678512,14396] 0 2024-09-16T08:57:25.070 INFO:tasks.workunit.client.0.smithi031.stdout:1/590: fdatasync d2/d4/d1b/f65 0 2024-09-16T08:57:25.070 INFO:tasks.workunit.client.0.smithi031.stdout:1/591: write d2/d4/fa2 [1054569,127841] 0 2024-09-16T08:57:25.072 INFO:tasks.workunit.client.0.smithi031.stdout:9/621: rename d3/d5/da/d34/d3c/d5e/d62/fd1 to d3/fdc 0 2024-09-16T08:57:25.072 INFO:tasks.workunit.client.0.smithi031.stdout:9/622: write d3/d5/da/db/d21/fc1 [833761,61806] 0 2024-09-16T08:57:25.073 INFO:tasks.workunit.client.0.smithi031.stdout:8/547: dwrite d4/d7/d21/d35/d50/f98 [0,4194304] 0 2024-09-16T08:57:25.073 INFO:tasks.workunit.client.0.smithi031.stdout:0/635: dread d6/f8 [0,4194304] 0 2024-09-16T08:57:25.073 INFO:tasks.workunit.client.0.smithi031.stdout:0/636: fdatasync d6/d12/f17 0 2024-09-16T08:57:25.093 INFO:tasks.workunit.client.0.smithi031.stdout:9/623: dread d3/f5b [0,4194304] 0 2024-09-16T08:57:25.093 INFO:tasks.workunit.client.0.smithi031.stdout:9/624: chown d3/d5/d6a 94875676 1 2024-09-16T08:57:25.098 INFO:tasks.workunit.client.0.smithi031.stdout:9/625: dread d3/d5/da/db/d21/fa3 [0,4194304] 0 2024-09-16T08:57:25.099 INFO:tasks.workunit.client.0.smithi031.stdout:9/626: dread d3/d5/da/d34/d3c/d5e/fb1 [0,4194304] 0 2024-09-16T08:57:25.100 INFO:tasks.workunit.client.0.smithi031.stdout:9/627: fdatasync d3/d5/da/d34/d3c/d5e/d62/d65/fbd 0 2024-09-16T08:57:25.101 INFO:tasks.workunit.client.0.smithi031.stdout:9/628: symlink d3/d5/da/db/dcb/ldd 0 2024-09-16T08:57:25.102 INFO:tasks.workunit.client.0.smithi031.stdout:4/815: dwrite d0/d2/d16/f4e [0,4194304] 0 2024-09-16T08:57:25.108 INFO:tasks.workunit.client.0.smithi031.stdout:1/592: dwrite d2/d15/d32/d87/fa0 [0,4194304] 0 2024-09-16T08:57:25.109 INFO:tasks.workunit.client.0.smithi031.stdout:0/637: dwrite d6/f6e [0,4194304] 0 2024-09-16T08:57:25.111 INFO:tasks.workunit.client.0.smithi031.stdout:1/593: creat d2/d4/d1b/d5c/d6b/da9/dc0/fcf x:0 0 0 2024-09-16T08:57:25.111 INFO:tasks.workunit.client.0.smithi031.stdout:1/594: stat d2/d15/d32/d87/l8f 0 2024-09-16T08:57:25.113 INFO:tasks.workunit.client.0.smithi031.stdout:4/816: dread d0/d102/f5d [4194304,4194304] 0 2024-09-16T08:57:25.113 INFO:tasks.workunit.client.0.smithi031.stdout:1/595: mknod d2/d4/d36/cd0 0 2024-09-16T08:57:25.114 INFO:tasks.workunit.client.0.smithi031.stdout:1/596: truncate d2/d4/d36/f7e 4682238 0 2024-09-16T08:57:25.114 INFO:tasks.workunit.client.0.smithi031.stdout:1/597: fdatasync d2/d15/d32/f97 0 2024-09-16T08:57:25.114 INFO:tasks.workunit.client.0.smithi031.stdout:4/817: write d0/d102/d84/fa2 [157973,119967] 0 2024-09-16T08:57:25.114 INFO:tasks.workunit.client.0.smithi031.stdout:0/638: rmdir d6/d12/d47/d7c 39 2024-09-16T08:57:25.115 INFO:tasks.workunit.client.0.smithi031.stdout:1/598: mkdir d2/d4/d1b/d5c/d6b/dc1/dd1 0 2024-09-16T08:57:25.115 INFO:tasks.workunit.client.0.smithi031.stdout:1/599: write d2/d4/f28 [3801863,82743] 0 2024-09-16T08:57:25.115 INFO:tasks.workunit.client.0.smithi031.stdout:4/818: rmdir d0/d102/d84/d86/d9b/dcf/dde 39 2024-09-16T08:57:25.117 INFO:tasks.workunit.client.0.smithi031.stdout:0/639: creat d6/d12/d47/d7c/fc8 x:0 0 0 2024-09-16T08:57:25.118 INFO:tasks.workunit.client.0.smithi031.stdout:1/600: creat d2/d4/d1b/d5c/d6b/dc1/fd2 x:0 0 0 2024-09-16T08:57:25.118 INFO:tasks.workunit.client.0.smithi031.stdout:1/601: creat d2/d4/d52/fd3 x:0 0 0 2024-09-16T08:57:25.119 INFO:tasks.workunit.client.0.smithi031.stdout:8/548: dwrite d4/d7/d21/d35/f5d [0,4194304] 0 2024-09-16T08:57:25.119 INFO:tasks.workunit.client.0.smithi031.stdout:8/549: fsync d4/d77/f81 0 2024-09-16T08:57:25.119 INFO:tasks.workunit.client.0.smithi031.stdout:9/629: dwrite d3/d5/f5d [0,4194304] 0 2024-09-16T08:57:25.120 INFO:tasks.workunit.client.0.smithi031.stdout:4/819: creat d0/d2/d16/d2b/daf/f108 x:0 0 0 2024-09-16T08:57:25.121 INFO:tasks.workunit.client.0.smithi031.stdout:0/640: chown d6/l53 303 1 2024-09-16T08:57:25.124 INFO:tasks.workunit.client.0.smithi031.stdout:1/602: unlink d2/de/ca7 0 2024-09-16T08:57:25.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/550: dread d4/d7/d66/f70 [4194304,4194304] 0 2024-09-16T08:57:25.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/551: chown d4/c9 5754660 1 2024-09-16T08:57:25.129 INFO:tasks.workunit.client.0.smithi031.stdout:9/630: getdents d3/d85 0 2024-09-16T08:57:25.137 INFO:tasks.workunit.client.0.smithi031.stdout:4/820: rename d0/d3c/df2 to d0/d109 0 2024-09-16T08:57:25.137 INFO:tasks.workunit.client.0.smithi031.stdout:4/821: chown d0/d2/fe 1 1 2024-09-16T08:57:25.138 INFO:tasks.workunit.client.0.smithi031.stdout:8/552: read d4/f8 [6415911,14334] 0 2024-09-16T08:57:25.138 INFO:tasks.workunit.client.0.smithi031.stdout:8/553: dread - d4/d7/d21/d35/d79/f8a zero size 2024-09-16T08:57:25.147 INFO:tasks.workunit.client.0.smithi031.stdout:0/641: dread d6/f7b [0,4194304] 0 2024-09-16T08:57:25.150 INFO:tasks.workunit.client.0.smithi031.stdout:0/642: mkdir d6/d12/d47/dc5/dc9 0 2024-09-16T08:57:25.151 INFO:tasks.workunit.client.0.smithi031.stdout:8/554: dread d4/d9e/d38/f73 [0,4194304] 0 2024-09-16T08:57:25.154 INFO:tasks.workunit.client.0.smithi031.stdout:1/603: dwrite d2/d4/fa2 [0,4194304] 0 2024-09-16T08:57:25.155 INFO:tasks.workunit.client.0.smithi031.stdout:9/631: dwrite d3/d5/da/db/fc0 [0,4194304] 0 2024-09-16T08:57:25.156 INFO:tasks.workunit.client.0.smithi031.stdout:8/555: symlink d4/d9e/d38/lb9 0 2024-09-16T08:57:25.158 INFO:tasks.workunit.client.0.smithi031.stdout:9/632: dread d3/d5/db3/fb8 [0,4194304] 0 2024-09-16T08:57:25.158 INFO:tasks.workunit.client.0.smithi031.stdout:0/643: mkdir d6/d12/d13/dca 0 2024-09-16T08:57:25.163 INFO:tasks.workunit.client.0.smithi031.stdout:1/604: rename d2/d4/d52/l60 to d2/d15/d1f/d43/ld4 0 2024-09-16T08:57:25.163 INFO:tasks.workunit.client.0.smithi031.stdout:1/605: fdatasync d2/d4/d1b/d5c/f94 0 2024-09-16T08:57:25.181 INFO:tasks.workunit.client.0.smithi031.stdout:9/633: symlink d3/d5/d6a/d74/da8/dc3/dbc/lde 0 2024-09-16T08:57:25.185 INFO:tasks.workunit.client.0.smithi031.stdout:0/644: read d6/f9 [4275100,100263] 0 2024-09-16T08:57:25.187 INFO:tasks.workunit.client.0.smithi031.stdout:1/606: mknod d2/d15/d4a/cd5 0 2024-09-16T08:57:25.195 INFO:tasks.workunit.client.0.smithi031.stdout:0/645: mknod d6/d12/d97/ccb 0 2024-09-16T08:57:25.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/634: rename d3/d5/db3/fb8 to d3/d5/da/d34/d3c/d5e/d9c/d95/dcf/fdf 0 2024-09-16T08:57:25.196 INFO:tasks.workunit.client.0.smithi031.stdout:1/607: mknod d2/d15/d32/d87/cd6 0 2024-09-16T08:57:25.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/646: mkdir d6/d12/d47/d5b/d9b/da2/dcc 0 2024-09-16T08:57:25.201 INFO:tasks.workunit.client.0.smithi031.stdout:0/647: stat d6/d12/d13/d36 0 2024-09-16T08:57:25.206 INFO:tasks.workunit.client.0.smithi031.stdout:9/635: mkdir d3/d5/dd6/de0 0 2024-09-16T08:57:25.209 INFO:tasks.workunit.client.0.smithi031.stdout:0/648: symlink d6/d12/d47/d5b/d9b/da2/lcd 0 2024-09-16T08:57:25.210 INFO:tasks.workunit.client.0.smithi031.stdout:1/608: getdents d2/d4/d36 0 2024-09-16T08:57:25.213 INFO:tasks.workunit.client.0.smithi031.stdout:9/636: mkdir d3/d5/da/d34/d3c/d5e/de1 0 2024-09-16T08:57:25.213 INFO:tasks.workunit.client.0.smithi031.stdout:9/637: readlink d3/d5/da/d34/d3c/d5e/d9c/d95/l96 0 2024-09-16T08:57:25.214 INFO:tasks.workunit.client.0.smithi031.stdout:0/649: mkdir d6/d61/dce 0 2024-09-16T08:57:25.226 INFO:tasks.workunit.client.0.smithi031.stdout:9/638: write d3/d5/d6a/d74/da8/dc3/f7b [602695,78417] 0 2024-09-16T08:57:25.227 INFO:tasks.workunit.client.0.smithi031.stdout:9/639: symlink d3/d5/da/db/d21/d3b/le2 0 2024-09-16T08:57:25.230 INFO:tasks.workunit.client.0.smithi031.stdout:9/640: mkdir d3/d5/d6a/d74/dd0/de3 0 2024-09-16T08:57:25.231 INFO:tasks.workunit.client.0.smithi031.stdout:0/650: dwrite d6/d12/d13/da5/fc0 [0,4194304] 0 2024-09-16T08:57:25.232 INFO:tasks.workunit.client.0.smithi031.stdout:9/641: creat d3/d5/da/d34/d3c/d5e/d9c/d95/dcf/fe4 x:0 0 0 2024-09-16T08:57:25.235 INFO:tasks.workunit.client.0.smithi031.stdout:1/609: dwrite d2/d4/d1b/d5c/d6b/da9/fbe [0,4194304] 0 2024-09-16T08:57:25.237 INFO:tasks.workunit.client.0.smithi031.stdout:0/651: mknod d6/d12/d97/ccf 0 2024-09-16T08:57:25.246 INFO:tasks.workunit.client.0.smithi031.stdout:1/610: link d2/d15/d5d/l68 d2/d15/d5d/db4/da3/dcb/ld7 0 2024-09-16T08:57:25.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/652: unlink d6/d61/lb0 0 2024-09-16T08:57:25.251 INFO:tasks.workunit.client.0.smithi031.stdout:1/611: rmdir d2/d15/d5d 39 2024-09-16T08:57:25.252 INFO:tasks.workunit.client.0.smithi031.stdout:0/653: symlink d6/d12/d47/dc5/dc9/ld0 0 2024-09-16T08:57:25.253 INFO:tasks.workunit.client.0.smithi031.stdout:0/654: creat d6/d12/d13/d36/fd1 x:0 0 0 2024-09-16T08:57:25.254 INFO:tasks.workunit.client.0.smithi031.stdout:0/655: symlink d6/dac/ld2 0 2024-09-16T08:57:25.263 INFO:tasks.workunit.client.0.smithi031.stdout:9/642: dwrite d3/d82/fac [0,4194304] 0 2024-09-16T08:57:25.265 INFO:tasks.workunit.client.0.smithi031.stdout:9/643: mknod d3/d5/d6a/d74/da8/dc3/dbc/ce5 0 2024-09-16T08:57:25.265 INFO:tasks.workunit.client.0.smithi031.stdout:9/644: chown d3/d5/da/d34/d3c/d5e/d62/d65/l6d 1 1 2024-09-16T08:57:25.276 INFO:tasks.workunit.client.0.smithi031.stdout:1/612: dwrite d2/d4/fa2 [4194304,4194304] 0 2024-09-16T08:57:25.281 INFO:tasks.workunit.client.0.smithi031.stdout:0/656: dwrite d6/d12/d47/d4d/fae [0,4194304] 0 2024-09-16T08:57:25.286 INFO:tasks.workunit.client.0.smithi031.stdout:9/645: dwrite d3/d5/da/db/faa [0,4194304] 0 2024-09-16T08:57:25.290 INFO:tasks.workunit.client.0.smithi031.stdout:1/613: dread d2/de/f2f [0,4194304] 0 2024-09-16T08:57:25.294 INFO:tasks.workunit.client.0.smithi031.stdout:1/614: read d2/d4/d1b/f50 [2436457,68185] 0 2024-09-16T08:57:25.299 INFO:tasks.workunit.client.0.smithi031.stdout:9/646: dread d3/d5/d6a/d74/da8/dc3/f7b [0,4194304] 0 2024-09-16T08:57:25.299 INFO:tasks.workunit.client.0.smithi031.stdout:1/615: rmdir d2/d15/d38 39 2024-09-16T08:57:25.301 INFO:tasks.workunit.client.0.smithi031.stdout:9/647: creat d3/d85/fe6 x:0 0 0 2024-09-16T08:57:25.302 INFO:tasks.workunit.client.0.smithi031.stdout:0/657: dwrite d6/d12/d13/d3c/f45 [0,4194304] 0 2024-09-16T08:57:25.302 INFO:tasks.workunit.client.0.smithi031.stdout:0/658: fdatasync d6/d12/d13/d3c/f70 0 2024-09-16T08:57:25.302 INFO:tasks.workunit.client.0.smithi031.stdout:1/616: stat d2/d4/l11 0 2024-09-16T08:57:25.308 INFO:tasks.workunit.client.0.smithi031.stdout:9/648: symlink d3/d5/da/db/d21/d3b/dad/dd3/le7 0 2024-09-16T08:57:25.318 INFO:tasks.workunit.client.0.smithi031.stdout:1/617: creat d2/d15/d32/d87/fd8 x:0 0 0 2024-09-16T08:57:25.321 INFO:tasks.workunit.client.0.smithi031.stdout:9/649: link d3/d48/f4e d3/d85/fe8 0 2024-09-16T08:57:25.321 INFO:tasks.workunit.client.0.smithi031.stdout:9/650: read - d3/d5/da/db/dcb/fce zero size 2024-09-16T08:57:25.324 INFO:tasks.workunit.client.0.smithi031.stdout:0/659: mkdir d6/dd3 0 2024-09-16T08:57:25.324 INFO:tasks.workunit.client.0.smithi031.stdout:1/618: symlink d2/d15/ld9 0 2024-09-16T08:57:25.324 INFO:tasks.workunit.client.0.smithi031.stdout:1/619: stat d2/d15/d32/d87/da5 0 2024-09-16T08:57:25.330 INFO:tasks.workunit.client.0.smithi031.stdout:0/660: creat d6/dac/fd4 x:0 0 0 2024-09-16T08:57:25.332 INFO:tasks.workunit.client.0.smithi031.stdout:0/661: chown d6/cd 13 1 2024-09-16T08:57:25.333 INFO:tasks.workunit.client.0.smithi031.stdout:9/651: write d3/d5/f6 [3840276,56802] 0 2024-09-16T08:57:25.333 INFO:tasks.workunit.client.0.smithi031.stdout:9/652: chown d3/d5/db3 235 1 2024-09-16T08:57:25.333 INFO:tasks.workunit.client.0.smithi031.stdout:9/653: stat d3/d5/da/d34/f77 0 2024-09-16T08:57:25.334 INFO:tasks.workunit.client.0.smithi031.stdout:9/654: chown d3/d5/d73/d81 202565 1 2024-09-16T08:57:25.334 INFO:tasks.workunit.client.0.smithi031.stdout:9/655: write d3/d85/f91 [4839817,57246] 0 2024-09-16T08:57:25.334 INFO:tasks.workunit.client.0.smithi031.stdout:9/656: chown d3/d5/da/db/d21/d3b/c70 148 1 2024-09-16T08:57:25.334 INFO:tasks.workunit.client.0.smithi031.stdout:9/657: read d3/d5/da/db/d21/d3b/fa5 [596676,35326] 0 2024-09-16T08:57:25.336 INFO:tasks.workunit.client.0.smithi031.stdout:9/658: mknod d3/d5/dd6/de0/ce9 0 2024-09-16T08:57:25.337 INFO:tasks.workunit.client.0.smithi031.stdout:9/659: rename d3/d82 to d3/d5/da/db/d21/d3b/dad/dea 0 2024-09-16T08:57:25.338 INFO:tasks.workunit.client.0.smithi031.stdout:9/660: stat d3/d5/da/d34/d3c/d5e/d62/f84 0 2024-09-16T08:57:25.339 INFO:tasks.workunit.client.0.smithi031.stdout:9/661: mkdir d3/d5/da/d34/d3c/d5e/de1/deb 0 2024-09-16T08:57:25.340 INFO:tasks.workunit.client.0.smithi031.stdout:9/662: mknod d3/d5/da/d34/d4d/cec 0 2024-09-16T08:57:25.340 INFO:tasks.workunit.client.0.smithi031.stdout:9/663: write d3/d5/d73/f86 [342406,98843] 0 2024-09-16T08:57:25.343 INFO:tasks.workunit.client.0.smithi031.stdout:1/620: dwrite d2/f14 [4194304,4194304] 0 2024-09-16T08:57:25.347 INFO:tasks.workunit.client.0.smithi031.stdout:1/621: mkdir d2/d4/d1b/d5c/d6b/da9/dc0/dda 0 2024-09-16T08:57:25.347 INFO:tasks.workunit.client.0.smithi031.stdout:0/662: dread d6/f1e [0,4194304] 0 2024-09-16T08:57:25.348 INFO:tasks.workunit.client.0.smithi031.stdout:1/622: mknod d2/d4/d1b/d5c/d6b/cdb 0 2024-09-16T08:57:25.349 INFO:tasks.workunit.client.0.smithi031.stdout:1/623: creat d2/d4/d1b/fdc x:0 0 0 2024-09-16T08:57:25.349 INFO:tasks.workunit.client.0.smithi031.stdout:1/624: creat d2/d15/d4a/fdd x:0 0 0 2024-09-16T08:57:25.349 INFO:tasks.workunit.client.0.smithi031.stdout:1/625: truncate d2/d4/d52/f6f 395264 0 2024-09-16T08:57:25.351 INFO:tasks.workunit.client.0.smithi031.stdout:1/626: rmdir d2/d4/d1b/d5c/d6b 39 2024-09-16T08:57:25.352 INFO:tasks.workunit.client.0.smithi031.stdout:1/627: mknod d2/d4/d67/cde 0 2024-09-16T08:57:25.353 INFO:tasks.workunit.client.0.smithi031.stdout:1/628: chown d2/d4/l2e 792588 1 2024-09-16T08:57:25.354 INFO:tasks.workunit.client.0.smithi031.stdout:1/629: mknod d2/d15/d32/d87/cdf 0 2024-09-16T08:57:25.354 INFO:tasks.workunit.client.0.smithi031.stdout:1/630: fdatasync d2/d4/d1b/fdc 0 2024-09-16T08:57:25.362 INFO:tasks.workunit.client.0.smithi031.stdout:9/664: dread d3/d5/da/db/d21/d3b/dad/dea/fac [0,4194304] 0 2024-09-16T08:57:25.363 INFO:tasks.workunit.client.0.smithi031.stdout:9/665: symlink d3/d5/da/d34/d3c/d5e/d9c/dcd/led 0 2024-09-16T08:57:25.365 INFO:tasks.workunit.client.0.smithi031.stdout:9/666: rmdir d3/d5/da/d34/d4d 39 2024-09-16T08:57:25.366 INFO:tasks.workunit.client.0.smithi031.stdout:9/667: truncate d3/d5/da/db/fe 370958 0 2024-09-16T08:57:25.374 INFO:tasks.workunit.client.0.smithi031.stdout:0/663: dwrite d6/d12/f22 [0,4194304] 0 2024-09-16T08:57:25.374 INFO:tasks.workunit.client.0.smithi031.stdout:9/668: dread d3/f5b [0,4194304] 0 2024-09-16T08:57:25.375 INFO:tasks.workunit.client.0.smithi031.stdout:1/631: dwrite d2/d4/f54 [0,4194304] 0 2024-09-16T08:57:25.376 INFO:tasks.workunit.client.0.smithi031.stdout:1/632: creat d2/d4/d52/fe0 x:0 0 0 2024-09-16T08:57:25.376 INFO:tasks.workunit.client.0.smithi031.stdout:7/701: sync 2024-09-16T08:57:25.376 INFO:tasks.workunit.client.0.smithi031.stdout:2/630: sync 2024-09-16T08:57:25.376 INFO:tasks.workunit.client.0.smithi031.stdout:7/702: write d2/d8/d86/f88 [786324,119022] 0 2024-09-16T08:57:25.376 INFO:tasks.workunit.client.0.smithi031.stdout:7/703: chown d2/c3c 446863 1 2024-09-16T08:57:25.379 INFO:tasks.workunit.client.0.smithi031.stdout:9/669: truncate d3/d5/d6a/d74/da8/dc3/f7b 64993 0 2024-09-16T08:57:25.380 INFO:tasks.workunit.client.0.smithi031.stdout:0/664: mknod d6/d12/d47/cd5 0 2024-09-16T08:57:25.380 INFO:tasks.workunit.client.0.smithi031.stdout:0/665: fsync d6/d12/d13/d36/f51 0 2024-09-16T08:57:25.380 INFO:tasks.workunit.client.0.smithi031.stdout:0/666: chown d6/d12/d13/c2f 55 1 2024-09-16T08:57:25.387 INFO:tasks.workunit.client.0.smithi031.stdout:9/670: dread d3/d5/da/db/fc0 [0,4194304] 0 2024-09-16T08:57:25.387 INFO:tasks.workunit.client.0.smithi031.stdout:9/671: readlink d3/d5/da/db/l15 0 2024-09-16T08:57:25.389 INFO:tasks.workunit.client.0.smithi031.stdout:1/633: symlink d2/d4/d1b/d5c/d6b/da9/le1 0 2024-09-16T08:57:25.391 INFO:tasks.workunit.client.0.smithi031.stdout:2/631: mknod d6/da/d4b/da0/cd5 0 2024-09-16T08:57:25.392 INFO:tasks.workunit.client.0.smithi031.stdout:7/704: symlink d2/d8/d11/d5d/d7e/le4 0 2024-09-16T08:57:25.395 INFO:tasks.workunit.client.0.smithi031.stdout:0/667: rename d6/f69 to d6/d12/d97/d9e/fd6 0 2024-09-16T08:57:25.395 INFO:tasks.workunit.client.0.smithi031.stdout:0/668: chown d6/d12/d13/d3c 1943 1 2024-09-16T08:57:25.395 INFO:tasks.workunit.client.0.smithi031.stdout:0/669: creat d6/d12/d47/d7c/fd7 x:0 0 0 2024-09-16T08:57:25.399 INFO:tasks.workunit.client.0.smithi031.stdout:7/705: creat d2/d8/fe5 x:0 0 0 2024-09-16T08:57:25.404 INFO:tasks.workunit.client.0.smithi031.stdout:1/634: write d2/d4/f54 [4982492,108111] 0 2024-09-16T08:57:25.408 INFO:tasks.workunit.client.0.smithi031.stdout:0/670: link d6/d12/d13/f24 d6/d12/d13/d36/fd8 0 2024-09-16T08:57:25.410 INFO:tasks.workunit.client.0.smithi031.stdout:7/706: creat d2/d2d/d6d/fe6 x:0 0 0 2024-09-16T08:57:25.410 INFO:tasks.workunit.client.0.smithi031.stdout:7/707: chown d2/d8/l10 53589 1 2024-09-16T08:57:25.410 INFO:tasks.workunit.client.0.smithi031.stdout:7/708: readlink d2/d8/l54 0 2024-09-16T08:57:25.413 INFO:tasks.workunit.client.0.smithi031.stdout:0/671: getdents d6/d12/d47/d5b/d9b 0 2024-09-16T08:57:25.415 INFO:tasks.workunit.client.0.smithi031.stdout:1/635: dread d2/d15/d5d/f3d [0,4194304] 0 2024-09-16T08:57:25.415 INFO:tasks.workunit.client.0.smithi031.stdout:7/709: symlink d2/d8/d11/d66/d9a/le7 0 2024-09-16T08:57:25.416 INFO:tasks.workunit.client.0.smithi031.stdout:0/672: mknod d6/d12/d47/d5b/d9b/da2/dcc/cd9 0 2024-09-16T08:57:25.417 INFO:tasks.workunit.client.0.smithi031.stdout:9/672: dwrite d3/d5/da/d34/d4d/f66 [0,4194304] 0 2024-09-16T08:57:25.418 INFO:tasks.workunit.client.0.smithi031.stdout:0/673: dread d6/f8 [0,4194304] 0 2024-09-16T08:57:25.418 INFO:tasks.workunit.client.0.smithi031.stdout:0/674: fdatasync d6/d12/d13/d36/fd1 0 2024-09-16T08:57:25.419 INFO:tasks.workunit.client.0.smithi031.stdout:2/632: dwrite d6/da/d5e/f70 [0,4194304] 0 2024-09-16T08:57:25.419 INFO:tasks.workunit.client.0.smithi031.stdout:1/636: dread d2/d15/d38/f9a [0,4194304] 0 2024-09-16T08:57:25.420 INFO:tasks.workunit.client.0.smithi031.stdout:1/637: stat d2/d15/d1f/d43/f4c 0 2024-09-16T08:57:25.420 INFO:tasks.workunit.client.0.smithi031.stdout:2/633: readlink d6/d20/d51/l5c 0 2024-09-16T08:57:25.420 INFO:tasks.workunit.client.0.smithi031.stdout:1/638: creat d2/d4/d1b/d5c/d6b/da9/fe2 x:0 0 0 2024-09-16T08:57:25.421 INFO:tasks.workunit.client.0.smithi031.stdout:9/673: link d3/d5/d73/lc2 d3/d5/da/db/d21/lee 0 2024-09-16T08:57:25.421 INFO:tasks.workunit.client.0.smithi031.stdout:9/674: stat d3/d5/f5d 0 2024-09-16T08:57:25.430 INFO:tasks.workunit.client.0.smithi031.stdout:0/675: dread d6/d12/d13/d36/f51 [0,4194304] 0 2024-09-16T08:57:25.431 INFO:tasks.workunit.client.0.smithi031.stdout:1/639: unlink d2/d4/d67/f89 0 2024-09-16T08:57:25.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/676: rename d6/d12/d13/c48 to d6/d61/d91/dc4/cda 0 2024-09-16T08:57:25.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/677: chown d6/d12/d47/d5b/d9b/da2/fab 3297937 1 2024-09-16T08:57:25.436 INFO:tasks.workunit.client.0.smithi031.stdout:0/678: link d6/d12/d47/f7d d6/d12/d13/d3c/fdb 0 2024-09-16T08:57:25.441 INFO:tasks.workunit.client.0.smithi031.stdout:7/710: dwrite d2/d2d/f43 [0,4194304] 0 2024-09-16T08:57:25.445 INFO:tasks.workunit.client.0.smithi031.stdout:7/711: mknod d2/d8/d11/ce8 0 2024-09-16T08:57:25.445 INFO:tasks.workunit.client.0.smithi031.stdout:7/712: getdents d2/d8/d11/dd5 0 2024-09-16T08:57:25.456 INFO:tasks.workunit.client.0.smithi031.stdout:7/713: dread d2/d2d/f2e [4194304,4194304] 0 2024-09-16T08:57:25.461 INFO:tasks.workunit.client.0.smithi031.stdout:9/675: dwrite d3/d5/f6 [0,4194304] 0 2024-09-16T08:57:25.461 INFO:tasks.workunit.client.0.smithi031.stdout:9/676: creat d3/d5/da/fef x:0 0 0 2024-09-16T08:57:25.462 INFO:tasks.workunit.client.0.smithi031.stdout:9/677: write d3/d5/da/d34/f99 [1007660,2235] 0 2024-09-16T08:57:25.464 INFO:tasks.workunit.client.0.smithi031.stdout:9/678: creat d3/d5/dd6/de0/ff0 x:0 0 0 2024-09-16T08:57:25.466 INFO:tasks.workunit.client.0.smithi031.stdout:9/679: rmdir d3/d5/da/d34/d3c/d5e/d9c/d95 39 2024-09-16T08:57:25.467 INFO:tasks.workunit.client.0.smithi031.stdout:9/680: mkdir d3/d5/da/d34/d4d/df1 0 2024-09-16T08:57:25.467 INFO:tasks.workunit.client.0.smithi031.stdout:9/681: stat d3/d5/la1 0 2024-09-16T08:57:25.469 INFO:tasks.workunit.client.0.smithi031.stdout:9/682: creat d3/d5/da/d34/d3c/d5e/d9c/d95/dcf/ff2 x:0 0 0 2024-09-16T08:57:25.471 INFO:tasks.workunit.client.0.smithi031.stdout:9/683: link d3/fdc d3/d5/da/d34/d3c/d5e/d9c/d95/ff3 0 2024-09-16T08:57:25.471 INFO:tasks.workunit.client.0.smithi031.stdout:9/684: truncate d3/d5/da/d34/fbe 200180 0 2024-09-16T08:57:25.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/634: dwrite d6/da/d5e/f70 [0,4194304] 0 2024-09-16T08:57:25.471 INFO:tasks.workunit.client.0.smithi031.stdout:0/679: dwrite d6/d12/f17 [0,4194304] 0 2024-09-16T08:57:25.472 INFO:tasks.workunit.client.0.smithi031.stdout:1/640: dwrite d2/d15/d38/fb2 [0,4194304] 0 2024-09-16T08:57:25.473 INFO:tasks.workunit.client.0.smithi031.stdout:9/685: rmdir d3/d5/d6a/d74 39 2024-09-16T08:57:25.475 INFO:tasks.workunit.client.0.smithi031.stdout:2/635: creat d6/da/d4b/d8f/fd6 x:0 0 0 2024-09-16T08:57:25.476 INFO:tasks.workunit.client.0.smithi031.stdout:7/714: dwrite d2/d8/f3f [0,4194304] 0 2024-09-16T08:57:25.484 INFO:tasks.workunit.client.0.smithi031.stdout:0/680: write d6/d12/d13/f7f [4116123,26939] 0 2024-09-16T08:57:25.490 INFO:tasks.workunit.client.0.smithi031.stdout:9/686: dwrite d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:25.494 INFO:tasks.workunit.client.0.smithi031.stdout:2/636: stat d6/d2e/d40/d6c/d2c/l8a 0 2024-09-16T08:57:25.495 INFO:tasks.workunit.client.0.smithi031.stdout:2/637: fsync d6/d2e/d40/fa5 0 2024-09-16T08:57:25.497 INFO:tasks.workunit.client.0.smithi031.stdout:2/638: dread d6/f74 [0,4194304] 0 2024-09-16T08:57:25.497 INFO:tasks.workunit.client.0.smithi031.stdout:2/639: chown d6/da/d37/d8b/dad 1 1 2024-09-16T08:57:25.497 INFO:tasks.workunit.client.0.smithi031.stdout:2/640: chown d6/d2e/d40/d6c/d2c/d53/d60/f6f 20107 1 2024-09-16T08:57:25.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/715: getdents d2/d8/d86/da1 0 2024-09-16T08:57:25.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/716: chown d2/d8/f4a 116 1 2024-09-16T08:57:25.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/717: stat d2/d8/d69/l79 0 2024-09-16T08:57:25.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/718: fsync d2/d2d/d6d/fb4 0 2024-09-16T08:57:25.498 INFO:tasks.workunit.client.0.smithi031.stdout:7/719: stat d2/d2d/d6d/fe6 0 2024-09-16T08:57:25.499 INFO:tasks.workunit.client.0.smithi031.stdout:0/681: rename d6/d12/d13/d36/f51 to d6/d12/d97/db3/dba/fdc 0 2024-09-16T08:57:25.500 INFO:tasks.workunit.client.0.smithi031.stdout:9/687: fdatasync d3/f2a 0 2024-09-16T08:57:25.501 INFO:tasks.workunit.client.0.smithi031.stdout:7/720: mknod d2/d8/d86/dd4/ce9 0 2024-09-16T08:57:25.505 INFO:tasks.workunit.client.0.smithi031.stdout:2/641: rename d6/da/d4b/d4c/f5f to d6/da/d5e/d72/fd7 0 2024-09-16T08:57:25.505 INFO:tasks.workunit.client.0.smithi031.stdout:2/642: write d6/d2e/d40/fc5 [567915,93970] 0 2024-09-16T08:57:25.505 INFO:tasks.workunit.client.0.smithi031.stdout:2/643: fsync d6/d20/f32 0 2024-09-16T08:57:25.505 INFO:tasks.workunit.client.0.smithi031.stdout:4/822: sync 2024-09-16T08:57:25.505 INFO:tasks.workunit.client.0.smithi031.stdout:6/749: sync 2024-09-16T08:57:25.506 INFO:tasks.workunit.client.0.smithi031.stdout:5/601: sync 2024-09-16T08:57:25.506 INFO:tasks.workunit.client.0.smithi031.stdout:3/689: sync 2024-09-16T08:57:25.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/688: mknod d3/d5/d6a/d74/dd0/cf4 0 2024-09-16T08:57:25.506 INFO:tasks.workunit.client.0.smithi031.stdout:9/689: fdatasync d3/d5/da/f33 0 2024-09-16T08:57:25.508 INFO:tasks.workunit.client.0.smithi031.stdout:8/556: sync 2024-09-16T08:57:25.509 INFO:tasks.workunit.client.0.smithi031.stdout:4/823: mknod d0/d102/d84/d86/d9b/dcf/c10a 0 2024-09-16T08:57:25.514 INFO:tasks.workunit.client.0.smithi031.stdout:7/721: write d2/f42 [196201,97790] 0 2024-09-16T08:57:25.516 INFO:tasks.workunit.client.0.smithi031.stdout:3/690: rename d3/f59 to d3/d14/d37/d1a/d1d/d3b/dc4/fca 0 2024-09-16T08:57:25.517 INFO:tasks.workunit.client.0.smithi031.stdout:8/557: mknod d4/d7/d21/d37/d46/db1/cba 0 2024-09-16T08:57:25.519 INFO:tasks.workunit.client.0.smithi031.stdout:6/750: dread d0/dd/d74/d19/fa1 [0,4194304] 0 2024-09-16T08:57:25.519 INFO:tasks.workunit.client.0.smithi031.stdout:5/602: rmdir d2/dc/d91 39 2024-09-16T08:57:25.522 INFO:tasks.workunit.client.0.smithi031.stdout:7/722: rmdir d2/d8/d11/d66/d9a/d9c/db3 39 2024-09-16T08:57:25.523 INFO:tasks.workunit.client.0.smithi031.stdout:4/824: rename d0/d2/d16/d2b/f74 to d0/d5b/db8/dc8/f10b 0 2024-09-16T08:57:25.524 INFO:tasks.workunit.client.0.smithi031.stdout:6/751: rmdir d0/dd/d2b 39 2024-09-16T08:57:25.527 INFO:tasks.workunit.client.0.smithi031.stdout:5/603: rename d2/dc/d3d/l4d to d2/dc/d3d/lc4 0 2024-09-16T08:57:25.529 INFO:tasks.workunit.client.0.smithi031.stdout:3/691: dread d3/d14/d6f/f73 [0,4194304] 0 2024-09-16T08:57:25.531 INFO:tasks.workunit.client.0.smithi031.stdout:3/692: read d3/d14/d37/d1a/d1d/d3b/d3f/d49/f58 [538865,39427] 0 2024-09-16T08:57:25.533 INFO:tasks.workunit.client.0.smithi031.stdout:6/752: symlink d0/dd/d74/d5e/d7c/d3f/d92/le9 0 2024-09-16T08:57:25.536 INFO:tasks.workunit.client.0.smithi031.stdout:3/693: truncate d3/d14/d37/d1a/d1d/d3b/f43 961863 0 2024-09-16T08:57:25.536 INFO:tasks.workunit.client.0.smithi031.stdout:3/694: write d3/d14/f8c [2906645,68549] 0 2024-09-16T08:57:25.539 INFO:tasks.workunit.client.0.smithi031.stdout:6/753: getdents d0/dd 0 2024-09-16T08:57:25.539 INFO:tasks.workunit.client.0.smithi031.stdout:6/754: chown d0/d8e/cd4 862747 1 2024-09-16T08:57:25.539 INFO:tasks.workunit.client.0.smithi031.stdout:0/682: dwrite d6/d12/d13/d3c/f6d [0,4194304] 0 2024-09-16T08:57:25.543 INFO:tasks.workunit.client.0.smithi031.stdout:6/755: dread d0/dd/f23 [0,4194304] 0 2024-09-16T08:57:25.544 INFO:tasks.workunit.client.0.smithi031.stdout:6/756: write d0/dd/d74/f18 [8648977,100859] 0 2024-09-16T08:57:25.544 INFO:tasks.workunit.client.0.smithi031.stdout:6/757: fsync d0/dd/d74/d67/fb0 0 2024-09-16T08:57:25.551 INFO:tasks.workunit.client.0.smithi031.stdout:3/695: dread d3/d14/d37/d1a/d24/f77 [0,4194304] 0 2024-09-16T08:57:25.551 INFO:tasks.workunit.client.0.smithi031.stdout:3/696: write d3/d14/d37/d1a/d1d/d1e/d67/f6b [1279188,84868] 0 2024-09-16T08:57:25.551 INFO:tasks.workunit.client.0.smithi031.stdout:3/697: chown d3/d14/d37/d1a/f8f 254301938 1 2024-09-16T08:57:25.551 INFO:tasks.workunit.client.0.smithi031.stdout:3/698: readlink d3/d14/d37/l78 0 2024-09-16T08:57:25.551 INFO:tasks.workunit.client.0.smithi031.stdout:3/699: read d3/d14/f2e [3368495,11376] 0 2024-09-16T08:57:25.551 INFO:tasks.workunit.client.0.smithi031.stdout:3/700: mknod d3/d14/d37/d1a/d1d/dac/ccb 0 2024-09-16T08:57:25.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/701: chown d3/d14/d37/d47/f5b 139 1 2024-09-16T08:57:25.552 INFO:tasks.workunit.client.0.smithi031.stdout:3/702: creat d3/d14/d6f/d82/d96/fcc x:0 0 0 2024-09-16T08:57:25.558 INFO:tasks.workunit.client.0.smithi031.stdout:0/683: write d6/f6e [3355013,83923] 0 2024-09-16T08:57:25.558 INFO:tasks.workunit.client.0.smithi031.stdout:0/684: truncate d6/f7b 4632496 0 2024-09-16T08:57:25.558 INFO:tasks.workunit.client.0.smithi031.stdout:0/685: write d6/d12/d47/f55 [2203593,111052] 0 2024-09-16T08:57:25.558 INFO:tasks.workunit.client.0.smithi031.stdout:0/686: write d6/d12/d13/d36/f96 [895408,3050] 0 2024-09-16T08:57:25.558 INFO:tasks.workunit.client.0.smithi031.stdout:0/687: dread - d6/d12/d47/fb9 zero size 2024-09-16T08:57:25.565 INFO:tasks.workunit.client.0.smithi031.stdout:9/690: dwrite d3/d5/d6a/d74/da8/fb2 [4194304,4194304] 0 2024-09-16T08:57:25.566 INFO:tasks.workunit.client.0.smithi031.stdout:5/604: dwrite d2/dc/d25/d21/d2e/f72 [0,4194304] 0 2024-09-16T08:57:25.566 INFO:tasks.workunit.client.0.smithi031.stdout:2/644: dwrite d6/da/f27 [0,4194304] 0 2024-09-16T08:57:25.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/703: fdatasync d3/d14/f8c 0 2024-09-16T08:57:25.566 INFO:tasks.workunit.client.0.smithi031.stdout:4/825: dwrite d0/d2/d16/d1e/f23 [0,4194304] 0 2024-09-16T08:57:25.567 INFO:tasks.workunit.client.0.smithi031.stdout:0/688: write d6/d61/f62 [3954360,34474] 0 2024-09-16T08:57:25.567 INFO:tasks.workunit.client.0.smithi031.stdout:0/689: write d6/d12/d13/d36/fd1 [276401,50197] 0 2024-09-16T08:57:25.569 INFO:tasks.workunit.client.0.smithi031.stdout:9/691: mkdir d3/d5/d73/df5 0 2024-09-16T08:57:25.569 INFO:tasks.workunit.client.0.smithi031.stdout:9/692: creat d3/d5/dd6/de0/ff6 x:0 0 0 2024-09-16T08:57:25.569 INFO:tasks.workunit.client.0.smithi031.stdout:9/693: truncate d3/d85/fe8 23820 0 2024-09-16T08:57:25.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/605: truncate d2/dc/f3b 3241705 0 2024-09-16T08:57:25.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/606: read - d2/dc/f93 zero size 2024-09-16T08:57:25.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/607: write d2/dc/d25/d68/f6b [129653,90914] 0 2024-09-16T08:57:25.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/608: creat d2/dc/d36/fc5 x:0 0 0 2024-09-16T08:57:25.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/609: chown d2/dc/d25/d21/d81 411286811 1 2024-09-16T08:57:25.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/610: chown d2/dc/d25/d21/d2e/f72 1440 1 2024-09-16T08:57:25.573 INFO:tasks.workunit.client.0.smithi031.stdout:5/611: dread - d2/dc/d1d/d97/fb9 zero size 2024-09-16T08:57:25.576 INFO:tasks.workunit.client.0.smithi031.stdout:2/645: write d6/da/fd [88274,14666] 0 2024-09-16T08:57:25.578 INFO:tasks.workunit.client.0.smithi031.stdout:4/826: write d0/db5/fb7 [3833214,116133] 0 2024-09-16T08:57:25.585 INFO:tasks.workunit.client.0.smithi031.stdout:7/723: write d2/d8/d11/d5d/d6e/f71 [1895098,32865] 0 2024-09-16T08:57:25.587 INFO:tasks.workunit.client.0.smithi031.stdout:0/690: rmdir d6/d12/d97 39 2024-09-16T08:57:25.587 INFO:tasks.workunit.client.0.smithi031.stdout:0/691: creat d6/d12/d47/d7c/fdd x:0 0 0 2024-09-16T08:57:25.589 INFO:tasks.workunit.client.0.smithi031.stdout:9/694: creat d3/d5/d6a/d74/dd0/de3/ff7 x:0 0 0 2024-09-16T08:57:25.590 INFO:tasks.workunit.client.0.smithi031.stdout:5/612: creat d2/dc/d25/d21/d81/dc3/fc6 x:0 0 0 2024-09-16T08:57:25.590 INFO:tasks.workunit.client.0.smithi031.stdout:5/613: stat d2/dc/d91/cb8 0 2024-09-16T08:57:25.593 INFO:tasks.workunit.client.0.smithi031.stdout:7/724: mknod d2/d8/d11/d66/d9a/cea 0 2024-09-16T08:57:25.595 INFO:tasks.workunit.client.0.smithi031.stdout:9/695: dread d3/d5/da/db/fe [0,4194304] 0 2024-09-16T08:57:25.596 INFO:tasks.workunit.client.0.smithi031.stdout:5/614: symlink d2/dc/d25/lc7 0 2024-09-16T08:57:25.601 INFO:tasks.workunit.client.0.smithi031.stdout:0/692: mkdir d6/d12/d13/dde 0 2024-09-16T08:57:25.604 INFO:tasks.workunit.client.0.smithi031.stdout:7/725: dread d2/d8/d11/d5d/d6e/f95 [0,4194304] 0 2024-09-16T08:57:25.604 INFO:tasks.workunit.client.0.smithi031.stdout:3/704: dwrite d3/d14/d37/d1a/d1d/d1e/f80 [4194304,4194304] 0 2024-09-16T08:57:25.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/705: fsync d3/d14/d37/d1a/f9b 0 2024-09-16T08:57:25.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/706: creat d3/d14/d37/d47/fcd x:0 0 0 2024-09-16T08:57:25.605 INFO:tasks.workunit.client.0.smithi031.stdout:3/707: chown d3/d14/d37/d1a/d1d/d3b/d3f/d79/d7c 1512671223 1 2024-09-16T08:57:25.607 INFO:tasks.workunit.client.0.smithi031.stdout:3/708: mkdir d3/d14/d6f/d82/dce 0 2024-09-16T08:57:25.608 INFO:tasks.workunit.client.0.smithi031.stdout:3/709: truncate d3/d14/d37/d1a/d1d/dac/fbc 141957 0 2024-09-16T08:57:25.608 INFO:tasks.workunit.client.0.smithi031.stdout:5/615: dread d2/dc/f1f [0,4194304] 0 2024-09-16T08:57:25.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/710: dread d3/f4 [0,4194304] 0 2024-09-16T08:57:25.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/711: write d3/d14/d37/d1a/f65 [94790,87462] 0 2024-09-16T08:57:25.617 INFO:tasks.workunit.client.0.smithi031.stdout:0/693: dread d6/d12/d47/d4d/fae [0,4194304] 0 2024-09-16T08:57:25.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/712: mkdir d3/d14/d37/d1a/d1d/dac/dcf 0 2024-09-16T08:57:25.618 INFO:tasks.workunit.client.0.smithi031.stdout:0/694: rename d6/d12/d13/d36/c65 to d6/d12/d47/d7c/cdf 0 2024-09-16T08:57:25.632 INFO:tasks.workunit.client.0.smithi031.stdout:3/713: dread d3/f5a [0,4194304] 0 2024-09-16T08:57:25.636 INFO:tasks.workunit.client.0.smithi031.stdout:0/695: read d6/d12/d13/d3c/f6d [3212492,22826] 0 2024-09-16T08:57:25.637 INFO:tasks.workunit.client.0.smithi031.stdout:0/696: mknod d6/d61/d91/ce0 0 2024-09-16T08:57:25.640 INFO:tasks.workunit.client.0.smithi031.stdout:4/827: dwrite d0/f38 [4194304,4194304] 0 2024-09-16T08:57:25.644 INFO:tasks.workunit.client.0.smithi031.stdout:4/828: mknod d0/c10c 0 2024-09-16T08:57:25.644 INFO:tasks.workunit.client.0.smithi031.stdout:4/829: stat d0/d109/d106/f6f 0 2024-09-16T08:57:25.644 INFO:tasks.workunit.client.0.smithi031.stdout:4/830: readlink d0/d3c/l4a 0 2024-09-16T08:57:25.645 INFO:tasks.workunit.client.0.smithi031.stdout:4/831: fdatasync d0/d3c/d6b/f5e 0 2024-09-16T08:57:25.645 INFO:tasks.workunit.client.0.smithi031.stdout:9/696: dwrite d3/d5/da/db/d21/d3b/dad/dea/fac [4194304,4194304] 0 2024-09-16T08:57:25.645 INFO:tasks.workunit.client.0.smithi031.stdout:7/726: dwrite d2/d8/d11/d66/d9a/d9c/fba [0,4194304] 0 2024-09-16T08:57:25.645 INFO:tasks.workunit.client.0.smithi031.stdout:7/727: write d2/d8/d11/f44 [790087,108509] 0 2024-09-16T08:57:25.645 INFO:tasks.workunit.client.0.smithi031.stdout:7/728: chown d2/c5a 0 1 2024-09-16T08:57:25.647 INFO:tasks.workunit.client.0.smithi031.stdout:4/832: dread d0/d2/d16/f19 [0,4194304] 0 2024-09-16T08:57:25.647 INFO:tasks.workunit.client.0.smithi031.stdout:4/833: write d0/f1 [2439264,44196] 0 2024-09-16T08:57:25.647 INFO:tasks.workunit.client.0.smithi031.stdout:4/834: stat d0/d2/d16/d2b/dfd 0 2024-09-16T08:57:25.648 INFO:tasks.workunit.client.0.smithi031.stdout:7/729: creat d2/d8/d11/feb x:0 0 0 2024-09-16T08:57:25.650 INFO:tasks.workunit.client.0.smithi031.stdout:4/835: mkdir d0/db5/d10d 0 2024-09-16T08:57:25.651 INFO:tasks.workunit.client.0.smithi031.stdout:9/697: mknod d3/cf8 0 2024-09-16T08:57:25.652 INFO:tasks.workunit.client.0.smithi031.stdout:9/698: mkdir d3/d5/d73/d81/df9 0 2024-09-16T08:57:25.661 INFO:tasks.workunit.client.0.smithi031.stdout:2/646: truncate d6/da/fd 3826302 0 2024-09-16T08:57:25.661 INFO:tasks.workunit.client.0.smithi031.stdout:7/730: dread d2/d8/fd [0,4194304] 0 2024-09-16T08:57:25.670 INFO:tasks.workunit.client.0.smithi031.stdout:2/647: dread d6/f13 [0,4194304] 0 2024-09-16T08:57:25.671 INFO:tasks.workunit.client.0.smithi031.stdout:2/648: creat d6/d2e/d40/d6c/d2c/d53/d60/dbd/fd8 x:0 0 0 2024-09-16T08:57:25.672 INFO:tasks.workunit.client.0.smithi031.stdout:2/649: mkdir d6/da/d4b/d8f/dd9 0 2024-09-16T08:57:25.673 INFO:tasks.workunit.client.0.smithi031.stdout:2/650: mkdir d6/d20/da1/dcf/dda 0 2024-09-16T08:57:25.673 INFO:tasks.workunit.client.0.smithi031.stdout:2/651: stat d6/da/d5e/dd1/cd4 0 2024-09-16T08:57:25.677 INFO:tasks.workunit.client.0.smithi031.stdout:7/731: write d2/d8/f3f [2727708,83351] 0 2024-09-16T08:57:25.685 INFO:tasks.workunit.client.0.smithi031.stdout:3/714: dwrite d3/d14/d6f/fba [0,4194304] 0 2024-09-16T08:57:25.686 INFO:tasks.workunit.client.0.smithi031.stdout:0/697: dwrite d6/d12/d47/d7c/fdd [0,4194304] 0 2024-09-16T08:57:25.689 INFO:tasks.workunit.client.0.smithi031.stdout:2/652: write d6/d2e/d40/d6c/f5a [1002878,124407] 0 2024-09-16T08:57:25.689 INFO:tasks.workunit.client.0.smithi031.stdout:9/699: dwrite d3/fc [0,4194304] 0 2024-09-16T08:57:25.689 INFO:tasks.workunit.client.0.smithi031.stdout:9/700: fdatasync d3/d5/da/d34/f77 0 2024-09-16T08:57:25.690 INFO:tasks.workunit.client.0.smithi031.stdout:3/715: dread d3/d14/d37/d1a/d1d/d3b/d60/f9c [0,4194304] 0 2024-09-16T08:57:25.690 INFO:tasks.workunit.client.0.smithi031.stdout:3/716: chown d3/d14/d6f/d82/dc0 1872486833 1 2024-09-16T08:57:25.692 INFO:tasks.workunit.client.0.smithi031.stdout:0/698: unlink d6/d12/d13/d3c/l89 0 2024-09-16T08:57:25.698 INFO:tasks.workunit.client.0.smithi031.stdout:2/653: mknod d6/da/d37/cdb 0 2024-09-16T08:57:25.700 INFO:tasks.workunit.client.0.smithi031.stdout:9/701: creat d3/d5/da/db/d21/d3b/dad/ffa x:0 0 0 2024-09-16T08:57:25.700 INFO:tasks.workunit.client.0.smithi031.stdout:9/702: chown d3/d5/da/d34/d3c/d5e/d62/d65/cd4 3715520 1 2024-09-16T08:57:25.701 INFO:tasks.workunit.client.0.smithi031.stdout:7/732: dwrite d2/d8/d86/da1/da9/fad [0,4194304] 0 2024-09-16T08:57:25.701 INFO:tasks.workunit.client.0.smithi031.stdout:7/733: read d2/d8/f57 [545272,26589] 0 2024-09-16T08:57:25.703 INFO:tasks.workunit.client.0.smithi031.stdout:0/699: symlink d6/d12/d13/d3c/le1 0 2024-09-16T08:57:25.704 INFO:tasks.workunit.client.0.smithi031.stdout:0/700: dread - d6/d12/d13/d36/f8a zero size 2024-09-16T08:57:25.704 INFO:tasks.workunit.client.0.smithi031.stdout:9/703: dread d3/d5/da/d34/f99 [0,4194304] 0 2024-09-16T08:57:25.704 INFO:tasks.workunit.client.0.smithi031.stdout:3/717: dread d3/d14/d37/d33/f6d [0,4194304] 0 2024-09-16T08:57:25.704 INFO:tasks.workunit.client.0.smithi031.stdout:3/718: fsync d3/d14/d6f/fc8 0 2024-09-16T08:57:25.704 INFO:tasks.workunit.client.0.smithi031.stdout:2/654: mkdir d6/d20/d51/ddc 0 2024-09-16T08:57:25.704 INFO:tasks.workunit.client.0.smithi031.stdout:2/655: fsync d6/da/d4b/d4c/d73/f9b 0 2024-09-16T08:57:25.706 INFO:tasks.workunit.client.0.smithi031.stdout:2/656: dread d6/d2e/d40/d58/f90 [0,4194304] 0 2024-09-16T08:57:25.706 INFO:tasks.workunit.client.0.smithi031.stdout:2/657: chown d6/da/d5e/d8d/lc9 639179 1 2024-09-16T08:57:25.706 INFO:tasks.workunit.client.0.smithi031.stdout:2/658: fdatasync d6/da/d4b/fc8 0 2024-09-16T08:57:25.706 INFO:tasks.workunit.client.0.smithi031.stdout:7/734: unlink d2/f56 0 2024-09-16T08:57:25.707 INFO:tasks.workunit.client.0.smithi031.stdout:0/701: link d6/d12/d13/f84 d6/d12/d47/fe2 0 2024-09-16T08:57:25.709 INFO:tasks.workunit.client.0.smithi031.stdout:9/704: mknod d3/d5/da/d34/d3c/cfb 0 2024-09-16T08:57:25.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/659: creat d6/d20/da1/dcf/dda/fdd x:0 0 0 2024-09-16T08:57:25.713 INFO:tasks.workunit.client.0.smithi031.stdout:2/660: chown d6/d2e/d40/fa5 3781 1 2024-09-16T08:57:25.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/735: creat d2/d8/d11/d5d/fec x:0 0 0 2024-09-16T08:57:25.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/736: fdatasync d2/d8/d86/da1/da9/fd3 0 2024-09-16T08:57:25.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/737: creat d2/d2d/d6d/fed x:0 0 0 2024-09-16T08:57:25.715 INFO:tasks.workunit.client.0.smithi031.stdout:7/738: fsync d2/d8/d11/d66/f72 0 2024-09-16T08:57:25.717 INFO:tasks.workunit.client.0.smithi031.stdout:0/702: unlink d6/d12/l40 0 2024-09-16T08:57:25.717 INFO:tasks.workunit.client.0.smithi031.stdout:0/703: truncate d6/d12/d13/d36/fa9 151344 0 2024-09-16T08:57:25.720 INFO:tasks.workunit.client.0.smithi031.stdout:9/705: unlink d3/d5/d6a/d74/da8/dc3/dbc/lc7 0 2024-09-16T08:57:25.720 INFO:tasks.workunit.client.0.smithi031.stdout:9/706: fdatasync d3/d5/da/db/d21/fa3 0 2024-09-16T08:57:25.724 INFO:tasks.workunit.client.0.smithi031.stdout:2/661: link d6/d2e/d40/d58/fa2 d6/da/d4b/da0/fde 0 2024-09-16T08:57:25.731 INFO:tasks.workunit.client.0.smithi031.stdout:0/704: dread d6/d12/d47/d5b/f67 [0,4194304] 0 2024-09-16T08:57:25.731 INFO:tasks.workunit.client.0.smithi031.stdout:7/739: link d2/d8/f2f d2/fee 0 2024-09-16T08:57:25.736 INFO:tasks.workunit.client.0.smithi031.stdout:9/707: mknod d3/d5/da/db/d21/d3b/dad/cfc 0 2024-09-16T08:57:25.736 INFO:tasks.workunit.client.0.smithi031.stdout:9/708: write d3/d85/fe6 [912249,68059] 0 2024-09-16T08:57:25.741 INFO:tasks.workunit.client.0.smithi031.stdout:3/719: rename d3/d14/d6f to d3/d14/d37/d1a/d1d/dac/dcf/dd0 0 2024-09-16T08:57:25.741 INFO:tasks.workunit.client.0.smithi031.stdout:3/720: truncate d3/d14/d37/d1a/d1d/d9a/fa2 63264 0 2024-09-16T08:57:25.741 INFO:tasks.workunit.client.0.smithi031.stdout:3/721: readlink d3/d14/d37/d1a/d1d/dac/dcf/dd0/d7d/lad 0 2024-09-16T08:57:25.742 INFO:tasks.workunit.client.0.smithi031.stdout:3/722: write d3/d14/fb6 [940489,49539] 0 2024-09-16T08:57:25.743 INFO:tasks.workunit.client.0.smithi031.stdout:7/740: unlink d2/d2d/d6d/cc4 0 2024-09-16T08:57:25.745 INFO:tasks.workunit.client.0.smithi031.stdout:3/723: mkdir d3/d14/d37/d1a/d1d/d3b/d3f/dd1 0 2024-09-16T08:57:25.746 INFO:tasks.workunit.client.0.smithi031.stdout:7/741: stat d2/d2d 0 2024-09-16T08:57:25.750 INFO:tasks.workunit.client.0.smithi031.stdout:7/742: mknod d2/d8/d11/d66/d9a/d9c/db3/cef 0 2024-09-16T08:57:25.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/662: dwrite d6/d2e/d40/d6c/d2c/d53/d60/f93 [0,4194304] 0 2024-09-16T08:57:25.753 INFO:tasks.workunit.client.0.smithi031.stdout:2/663: write d6/d2e/d40/d6c/d2c/d53/d60/f35 [1829902,3692] 0 2024-09-16T08:57:25.755 INFO:tasks.workunit.client.0.smithi031.stdout:2/664: creat d6/da/d4b/d4c/d68/fdf x:0 0 0 2024-09-16T08:57:25.755 INFO:tasks.workunit.client.0.smithi031.stdout:2/665: fsync d6/f36 0 2024-09-16T08:57:25.757 INFO:tasks.workunit.client.0.smithi031.stdout:9/709: dwrite d3/d5/da/d34/d3c/d5e/d9c/d95/dcf/fe4 [0,4194304] 0 2024-09-16T08:57:25.758 INFO:tasks.workunit.client.0.smithi031.stdout:9/710: fsync d3/d5/da/d34/f99 0 2024-09-16T08:57:25.763 INFO:tasks.workunit.client.0.smithi031.stdout:9/711: rename d3/d5/da/db/d21/d3b/l78 to d3/d5/da/d34/d3c/d5e/de1/lfd 0 2024-09-16T08:57:25.764 INFO:tasks.workunit.client.0.smithi031.stdout:9/712: mknod d3/d5/d73/df5/cfe 0 2024-09-16T08:57:25.765 INFO:tasks.workunit.client.0.smithi031.stdout:9/713: symlink d3/d5/da/db/d21/d3b/dbb/lff 0 2024-09-16T08:57:25.766 INFO:tasks.workunit.client.0.smithi031.stdout:9/714: mkdir d3/d5/d6a/d74/da8/d100 0 2024-09-16T08:57:25.767 INFO:tasks.workunit.client.0.smithi031.stdout:3/724: dwrite d3/d14/d37/d1a/d1d/dac/dcf/dd0/fba [4194304,4194304] 0 2024-09-16T08:57:25.767 INFO:tasks.workunit.client.0.smithi031.stdout:9/715: dread d3/d5/da/db/d21/d3b/fa5 [0,4194304] 0 2024-09-16T08:57:25.781 INFO:tasks.workunit.client.0.smithi031.stdout:9/716: dread d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:25.781 INFO:tasks.workunit.client.0.smithi031.stdout:9/717: dread - d3/d5/da/db/dcb/fce zero size 2024-09-16T08:57:25.781 INFO:tasks.workunit.client.0.smithi031.stdout:9/718: truncate d3/d5/da/d34/f77 2541698 0 2024-09-16T08:57:25.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/719: getdents d3/d5/d6a/d74/da8/dc3/dbc 0 2024-09-16T08:57:25.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/720: fdatasync d3/d5/da/d34/d3c/d5e/d9c/f9f 0 2024-09-16T08:57:25.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/721: readlink d3/d5/da/d34/d3c/d5e/d9c/dcd/led 0 2024-09-16T08:57:25.783 INFO:tasks.workunit.client.0.smithi031.stdout:9/722: chown d3/d5/da/db/d21/d3b/dad/dea/fac 14 1 2024-09-16T08:57:25.793 INFO:tasks.workunit.client.0.smithi031.stdout:2/666: dwrite d6/da/d5e/f70 [0,4194304] 0 2024-09-16T08:57:25.793 INFO:tasks.workunit.client.0.smithi031.stdout:2/667: write d6/d2e/f9a [836494,101848] 0 2024-09-16T08:57:25.793 INFO:tasks.workunit.client.0.smithi031.stdout:2/668: chown d6/d20/f32 7593116 1 2024-09-16T08:57:25.793 INFO:tasks.workunit.client.0.smithi031.stdout:2/669: truncate d6/d20/da1/dcf/fd0 150306 0 2024-09-16T08:57:25.799 INFO:tasks.workunit.client.0.smithi031.stdout:9/723: dread d3/d48/f68 [4194304,4194304] 0 2024-09-16T08:57:25.800 INFO:tasks.workunit.client.0.smithi031.stdout:9/724: creat d3/d5/da/db/d21/d3b/dad/dd3/f101 x:0 0 0 2024-09-16T08:57:25.801 INFO:tasks.workunit.client.0.smithi031.stdout:9/725: chown d3/d5/d6a/d74/dd0/de3 23962 1 2024-09-16T08:57:25.801 INFO:tasks.workunit.client.0.smithi031.stdout:9/726: creat d3/d5/da/d34/f102 x:0 0 0 2024-09-16T08:57:25.802 INFO:tasks.workunit.client.0.smithi031.stdout:9/727: creat d3/d5/da/d34/d3c/f103 x:0 0 0 2024-09-16T08:57:25.803 INFO:tasks.workunit.client.0.smithi031.stdout:9/728: creat d3/d5/d73/d81/df9/f104 x:0 0 0 2024-09-16T08:57:25.806 INFO:tasks.workunit.client.0.smithi031.stdout:2/670: write d6/d2e/f2f [3660852,85677] 0 2024-09-16T08:57:25.806 INFO:tasks.workunit.client.0.smithi031.stdout:2/671: readlink d6/l22 0 2024-09-16T08:57:25.807 INFO:tasks.workunit.client.0.smithi031.stdout:2/672: mkdir d6/da/d5e/de0 0 2024-09-16T08:57:25.809 INFO:tasks.workunit.client.0.smithi031.stdout:7/743: dwrite d2/d2d/fd8 [0,4194304] 0 2024-09-16T08:57:25.815 INFO:tasks.workunit.client.0.smithi031.stdout:3/725: dwrite d3/d14/d37/d1a/d1d/d3b/dc4/fca [0,4194304] 0 2024-09-16T08:57:25.815 INFO:tasks.workunit.client.0.smithi031.stdout:3/726: write d3/f4 [740417,105849] 0 2024-09-16T08:57:25.815 INFO:tasks.workunit.client.0.smithi031.stdout:3/727: chown d3/d14/d37/d1a/d1d/dac/dcf/dd0/d7d/f9f 819 1 2024-09-16T08:57:25.821 INFO:tasks.workunit.client.0.smithi031.stdout:7/744: mknod d2/d8/d86/da1/da9/cf0 0 2024-09-16T08:57:25.825 INFO:tasks.workunit.client.0.smithi031.stdout:3/728: rmdir d3/d14/d37/d1a/d1d/d3b/d60/db1 0 2024-09-16T08:57:25.825 INFO:tasks.workunit.client.0.smithi031.stdout:3/729: chown d3/d14/d37/d1a/d1d/d3b/d3f/c41 0 1 2024-09-16T08:57:25.828 INFO:tasks.workunit.client.0.smithi031.stdout:3/730: symlink d3/d14/d37/d1a/d1d/dac/dcf/dd0/ld2 0 2024-09-16T08:57:25.828 INFO:tasks.workunit.client.0.smithi031.stdout:2/673: dwrite d6/d2e/d40/d6c/f5a [0,4194304] 0 2024-09-16T08:57:25.829 INFO:tasks.workunit.client.0.smithi031.stdout:2/674: write d6/d2e/d40/d58/dae/db1/fc4 [706525,96136] 0 2024-09-16T08:57:25.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/675: rename d6/d2e/d40/f54 to d6/d2e/d40/d6c/d2c/d53/d60/dbd/fe1 0 2024-09-16T08:57:25.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/676: creat d6/da/d5e/d72/fe2 x:0 0 0 2024-09-16T08:57:25.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/677: dread - d6/da/d37/f63 zero size 2024-09-16T08:57:25.833 INFO:tasks.workunit.client.0.smithi031.stdout:2/678: chown d6/d2e/d40/d6c 64511 1 2024-09-16T08:57:25.833 INFO:tasks.workunit.client.0.smithi031.stdout:7/745: write d2/d8/d69/f6c [3853051,130424] 0 2024-09-16T08:57:25.834 INFO:tasks.workunit.client.0.smithi031.stdout:9/729: fsync d3/d5/d73/d81/df9/f104 0 2024-09-16T08:57:25.834 INFO:tasks.workunit.client.0.smithi031.stdout:9/730: chown d3/d5/da/d34/d3c/d5e/d62/d65/f93 25 1 2024-09-16T08:57:25.834 INFO:tasks.workunit.client.0.smithi031.stdout:9/731: readlink d3/d5/da/d34/d3c/d5e/de1/lfd 0 2024-09-16T08:57:25.842 INFO:tasks.workunit.client.0.smithi031.stdout:9/732: link d3/d5/d6a/d74/dd0/de3/ff7 d3/d5/da/d34/d4d/f105 0 2024-09-16T08:57:25.842 INFO:tasks.workunit.client.0.smithi031.stdout:7/746: dread d2/d8/d11/d66/d9a/fca [0,4194304] 0 2024-09-16T08:57:25.843 INFO:tasks.workunit.client.0.smithi031.stdout:2/679: rmdir d6/da/d37/d42 39 2024-09-16T08:57:25.845 INFO:tasks.workunit.client.0.smithi031.stdout:7/747: mkdir d2/d8/d86/dd4/df1 0 2024-09-16T08:57:25.845 INFO:tasks.workunit.client.0.smithi031.stdout:7/748: getdents d2/d8/d11/d66/d9a/d9c 0 2024-09-16T08:57:25.846 INFO:tasks.workunit.client.0.smithi031.stdout:9/733: write d3/d5/da/db/d21/f2f [4174323,3831] 0 2024-09-16T08:57:25.846 INFO:tasks.workunit.client.0.smithi031.stdout:9/734: write d3/d5/da/f33 [743733,100746] 0 2024-09-16T08:57:25.846 INFO:tasks.workunit.client.0.smithi031.stdout:9/735: truncate d3/d5/da/d34/fbe 1078361 0 2024-09-16T08:57:25.848 INFO:tasks.workunit.client.0.smithi031.stdout:7/749: mknod d2/d8/d86/dd4/cf2 0 2024-09-16T08:57:25.850 INFO:tasks.workunit.client.0.smithi031.stdout:3/731: dwrite d3/d14/d37/d1a/d1d/d1e/f80 [4194304,4194304] 0 2024-09-16T08:57:25.851 INFO:tasks.workunit.client.0.smithi031.stdout:3/732: read - d3/d14/d37/d1a/d1d/d1e/d67/d7f/f9e zero size 2024-09-16T08:57:25.851 INFO:tasks.workunit.client.0.smithi031.stdout:9/736: mknod d3/d5/d6a/d74/da8/dc3/dbc/dd7/c106 0 2024-09-16T08:57:25.853 INFO:tasks.workunit.client.0.smithi031.stdout:7/750: symlink d2/d2d/d6d/lf3 0 2024-09-16T08:57:25.858 INFO:tasks.workunit.client.0.smithi031.stdout:2/680: getdents d6/d2e/d40 0 2024-09-16T08:57:25.860 INFO:tasks.workunit.client.0.smithi031.stdout:6/758: sync 2024-09-16T08:57:25.860 INFO:tasks.workunit.client.0.smithi031.stdout:1/641: sync 2024-09-16T08:57:25.861 INFO:tasks.workunit.client.0.smithi031.stdout:1/642: chown d2/de 805727 1 2024-09-16T08:57:25.862 INFO:tasks.workunit.client.0.smithi031.stdout:6/759: read d0/dd/d2b/f45 [446769,52316] 0 2024-09-16T08:57:25.862 INFO:tasks.workunit.client.0.smithi031.stdout:6/760: chown d0/c46 0 1 2024-09-16T08:57:25.866 INFO:tasks.workunit.client.0.smithi031.stdout:3/733: mkdir d3/d14/d37/d1a/d1d/d3b/d3f/dd1/dd3 0 2024-09-16T08:57:25.867 INFO:tasks.workunit.client.0.smithi031.stdout:9/737: rename d3/fdc to d3/d5/dd6/f107 0 2024-09-16T08:57:25.870 INFO:tasks.workunit.client.0.smithi031.stdout:6/761: mknod d0/dd/d74/d67/dbb/cea 0 2024-09-16T08:57:25.871 INFO:tasks.workunit.client.0.smithi031.stdout:6/762: write d0/dd/d74/d5e/fb1 [740550,88829] 0 2024-09-16T08:57:25.871 INFO:tasks.workunit.client.0.smithi031.stdout:3/734: symlink d3/d14/d37/d1a/d1d/d3b/dc4/ld4 0 2024-09-16T08:57:25.872 INFO:tasks.workunit.client.0.smithi031.stdout:9/738: link d3/l9e d3/d5/d6a/d74/dd0/l108 0 2024-09-16T08:57:25.873 INFO:tasks.workunit.client.0.smithi031.stdout:1/643: dread d2/de/f2a [0,4194304] 0 2024-09-16T08:57:25.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/739: mkdir d3/d5/da/d109 0 2024-09-16T08:57:25.874 INFO:tasks.workunit.client.0.smithi031.stdout:9/740: chown d3/d5/da/d34/d3c/f103 237 1 2024-09-16T08:57:25.876 INFO:tasks.workunit.client.0.smithi031.stdout:8/558: sync 2024-09-16T08:57:25.879 INFO:tasks.workunit.client.0.smithi031.stdout:5/616: sync 2024-09-16T08:57:25.879 INFO:tasks.workunit.client.0.smithi031.stdout:5/617: truncate d2/dc/f61 625528 0 2024-09-16T08:57:25.881 INFO:tasks.workunit.client.0.smithi031.stdout:2/681: dwrite d6/da/d37/d42/fbc [0,4194304] 0 2024-09-16T08:57:25.881 INFO:tasks.workunit.client.0.smithi031.stdout:2/682: fdatasync d6/da/d5e/d72/fe2 0 2024-09-16T08:57:25.882 INFO:tasks.workunit.client.0.smithi031.stdout:5/618: mkdir d2/dc/d25/dc8 0 2024-09-16T08:57:25.884 INFO:tasks.workunit.client.0.smithi031.stdout:4/836: sync 2024-09-16T08:57:25.884 INFO:tasks.workunit.client.0.smithi031.stdout:0/705: sync 2024-09-16T08:57:25.884 INFO:tasks.workunit.client.0.smithi031.stdout:4/837: write d0/fa [4360365,46280] 0 2024-09-16T08:57:25.887 INFO:tasks.workunit.client.0.smithi031.stdout:2/683: unlink d6/d20/f66 0 2024-09-16T08:57:25.891 INFO:tasks.workunit.client.0.smithi031.stdout:1/644: write d2/f14 [2578348,72696] 0 2024-09-16T08:57:25.895 INFO:tasks.workunit.client.0.smithi031.stdout:5/619: rename d2/dc/d25/f2f to d2/dc/d8c/fc9 0 2024-09-16T08:57:25.895 INFO:tasks.workunit.client.0.smithi031.stdout:5/620: chown d2/d4b/f79 479 1 2024-09-16T08:57:25.895 INFO:tasks.workunit.client.0.smithi031.stdout:5/621: chown d2/dc/d36/c9c 15977 1 2024-09-16T08:57:25.897 INFO:tasks.workunit.client.0.smithi031.stdout:1/645: mkdir d2/d15/d5d/db4/de3 0 2024-09-16T08:57:25.899 INFO:tasks.workunit.client.0.smithi031.stdout:5/622: dread d2/dc/f61 [0,4194304] 0 2024-09-16T08:57:25.899 INFO:tasks.workunit.client.0.smithi031.stdout:5/623: creat d2/dc/d36/fca x:0 0 0 2024-09-16T08:57:25.913 INFO:tasks.workunit.client.0.smithi031.stdout:6/763: dwrite d0/d10/dc8/f90 [0,4194304] 0 2024-09-16T08:57:25.913 INFO:tasks.workunit.client.0.smithi031.stdout:8/559: dwrite d4/d9e/d38/f9c [4194304,4194304] 0 2024-09-16T08:57:25.925 INFO:tasks.workunit.client.0.smithi031.stdout:3/735: dwrite d3/d14/d37/d1a/f26 [0,4194304] 0 2024-09-16T08:57:25.925 INFO:tasks.workunit.client.0.smithi031.stdout:4/838: dwrite d0/f18 [0,4194304] 0 2024-09-16T08:57:25.925 INFO:tasks.workunit.client.0.smithi031.stdout:4/839: chown d0/d2/d16/f99 90244047 1 2024-09-16T08:57:25.925 INFO:tasks.workunit.client.0.smithi031.stdout:4/840: chown d0/d3c/d6b 1 1 2024-09-16T08:57:25.925 INFO:tasks.workunit.client.0.smithi031.stdout:4/841: chown d0/fa 3 1 2024-09-16T08:57:25.925 INFO:tasks.workunit.client.0.smithi031.stdout:0/706: dwrite d6/dac/fd4 [0,4194304] 0 2024-09-16T08:57:25.926 INFO:tasks.workunit.client.0.smithi031.stdout:0/707: write d6/d12/d13/d36/fd8 [1485868,55212] 0 2024-09-16T08:57:25.986 INFO:tasks.workunit.client.0.smithi031.stdout:1/646: truncate d2/d4/d52/fcc 3898896 0 2024-09-16T08:57:25.986 INFO:tasks.workunit.client.0.smithi031.stdout:1/647: write d2/d4/d1b/d5c/d6b/dc1/fd2 [286273,89011] 0 2024-09-16T08:57:25.991 INFO:tasks.workunit.client.0.smithi031.stdout:5/624: link d2/dc/fba d2/dc/d1d/dbb/fcb 0 2024-09-16T08:57:25.999 INFO:tasks.workunit.client.0.smithi031.stdout:8/560: rename d4/d7/d21/d24/l93 to d4/d2d/d97/lbb 0 2024-09-16T08:57:25.999 INFO:tasks.workunit.client.0.smithi031.stdout:8/561: chown d4/d9e/d5e/l63 95 1 2024-09-16T08:57:26.003 INFO:tasks.workunit.client.0.smithi031.stdout:3/736: truncate d3/d14/d37/d47/f5b 3455113 0 2024-09-16T08:57:26.003 INFO:tasks.workunit.client.0.smithi031.stdout:3/737: truncate d3/d14/d37/d47/fcd 890910 0 2024-09-16T08:57:26.004 INFO:tasks.workunit.client.0.smithi031.stdout:3/738: write d3/d14/d37/d1a/d1d/d1e/d67/f6b [271782,83716] 0 2024-09-16T08:57:26.006 INFO:tasks.workunit.client.0.smithi031.stdout:4/842: mkdir d0/d2/d16/d2b/d10e 0 2024-09-16T08:57:26.014 INFO:tasks.workunit.client.0.smithi031.stdout:3/739: write d3/d14/d37/d1a/d1d/d1e/f80 [6738763,36954] 0 2024-09-16T08:57:26.014 INFO:tasks.workunit.client.0.smithi031.stdout:1/648: mkdir d2/d4/d1b/de4 0 2024-09-16T08:57:26.015 INFO:tasks.workunit.client.0.smithi031.stdout:1/649: fsync d2/d4/d1b/fb3 0 2024-09-16T08:57:26.017 INFO:tasks.workunit.client.0.smithi031.stdout:5/625: creat d2/dc/d25/d68/fcc x:0 0 0 2024-09-16T08:57:26.019 INFO:tasks.workunit.client.0.smithi031.stdout:6/764: rename d0/d10/f58 to d0/dd/d35/d6c/dac/dba/feb 0 2024-09-16T08:57:26.020 INFO:tasks.workunit.client.0.smithi031.stdout:6/765: write d0/dd/d74/d19/d54/fd6 [4756105,7394] 0 2024-09-16T08:57:26.020 INFO:tasks.workunit.client.0.smithi031.stdout:6/766: creat d0/d10/dc8/fec x:0 0 0 2024-09-16T08:57:26.020 INFO:tasks.workunit.client.0.smithi031.stdout:6/767: fsync d0/f15 0 2024-09-16T08:57:26.021 INFO:tasks.workunit.client.0.smithi031.stdout:8/562: truncate d4/d9e/d38/f3c 3571258 0 2024-09-16T08:57:26.030 INFO:tasks.workunit.client.0.smithi031.stdout:6/768: dread d0/f7f [0,4194304] 0 2024-09-16T08:57:26.039 INFO:tasks.workunit.client.0.smithi031.stdout:5/626: creat d2/dc/d36/daf/fcd x:0 0 0 2024-09-16T08:57:26.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/563: dwrite d4/d7/d99/fb6 [0,4194304] 0 2024-09-16T08:57:26.043 INFO:tasks.workunit.client.0.smithi031.stdout:3/740: rename d3/d14/d37/d1a/d1d/dac/ccb to d3/d14/d37/d1a/d1d/dac/dcf/dd0/d82/d96/cd5 0 2024-09-16T08:57:26.043 INFO:tasks.workunit.client.0.smithi031.stdout:3/741: dread - d3/d14/d37/d33/d97/fb5 zero size 2024-09-16T08:57:26.046 INFO:tasks.workunit.client.0.smithi031.stdout:6/769: creat d0/dd/d74/d5e/fed x:0 0 0 2024-09-16T08:57:26.047 INFO:tasks.workunit.client.0.smithi031.stdout:6/770: stat d0/dd/d74/d67/fbf 0 2024-09-16T08:57:26.047 INFO:tasks.workunit.client.0.smithi031.stdout:6/771: chown d0/dd/d74/d5e/f77 2538 1 2024-09-16T08:57:26.052 INFO:tasks.workunit.client.0.smithi031.stdout:8/564: dread d4/f8 [4194304,4194304] 0 2024-09-16T08:57:26.052 INFO:tasks.workunit.client.0.smithi031.stdout:8/565: chown d4/d7/d21/d35/d50/f98 2 1 2024-09-16T08:57:26.053 INFO:tasks.workunit.client.0.smithi031.stdout:0/708: dwrite d6/d12/d13/f39 [0,4194304] 0 2024-09-16T08:57:26.060 INFO:tasks.workunit.client.0.smithi031.stdout:5/627: fsync d2/dc/d36/daf/fcd 0 2024-09-16T08:57:26.060 INFO:tasks.workunit.client.0.smithi031.stdout:5/628: fsync d2/dc/d1d/f92 0 2024-09-16T08:57:26.060 INFO:tasks.workunit.client.0.smithi031.stdout:5/629: fsync d2/dc/f1f 0 2024-09-16T08:57:26.062 INFO:tasks.workunit.client.0.smithi031.stdout:0/709: dread d6/f1e [0,4194304] 0 2024-09-16T08:57:26.062 INFO:tasks.workunit.client.0.smithi031.stdout:1/650: rename d2/ca1 to d2/d4/d1b/d5c/d6b/dc1/ce5 0 2024-09-16T08:57:26.062 INFO:tasks.workunit.client.0.smithi031.stdout:1/651: read d2/d4/d52/fcc [2758054,38194] 0 2024-09-16T08:57:26.063 INFO:tasks.workunit.client.0.smithi031.stdout:6/772: unlink d0/d10/f5a 0 2024-09-16T08:57:26.068 INFO:tasks.workunit.client.0.smithi031.stdout:1/652: dread d2/d15/d5d/f23 [0,4194304] 0 2024-09-16T08:57:26.068 INFO:tasks.workunit.client.0.smithi031.stdout:0/710: symlink d6/d61/d91/dc4/le3 0 2024-09-16T08:57:26.068 INFO:tasks.workunit.client.0.smithi031.stdout:0/711: write d6/d12/d47/fb9 [214822,8533] 0 2024-09-16T08:57:26.069 INFO:tasks.workunit.client.0.smithi031.stdout:0/712: chown d6/d12/f37 251201 1 2024-09-16T08:57:26.070 INFO:tasks.workunit.client.0.smithi031.stdout:3/742: rename d3/d14/d37/d1a/d1d/dac/dcf to d3/d14/d37/d1a/d1d/d1e/d67/dd6 0 2024-09-16T08:57:26.070 INFO:tasks.workunit.client.0.smithi031.stdout:3/743: stat d3/f5a 0 2024-09-16T08:57:26.070 INFO:tasks.workunit.client.0.smithi031.stdout:3/744: chown d3/d14/cab 17887075 1 2024-09-16T08:57:26.070 INFO:tasks.workunit.client.0.smithi031.stdout:3/745: fsync d3/d14/d37/d1a/d1d/d3b/f8b 0 2024-09-16T08:57:26.071 INFO:tasks.workunit.client.0.smithi031.stdout:8/566: dwrite d4/fb7 [0,4194304] 0 2024-09-16T08:57:26.072 INFO:tasks.workunit.client.0.smithi031.stdout:6/773: creat d0/dd/d35/d6c/d83/fee x:0 0 0 2024-09-16T08:57:26.073 INFO:tasks.workunit.client.0.smithi031.stdout:5/630: dread d2/f7f [0,4194304] 0 2024-09-16T08:57:26.073 INFO:tasks.workunit.client.0.smithi031.stdout:5/631: fsync d2/dc/fba 0 2024-09-16T08:57:26.073 INFO:tasks.workunit.client.0.smithi031.stdout:5/632: stat d2/dc/d36/daf 0 2024-09-16T08:57:26.075 INFO:tasks.workunit.client.0.smithi031.stdout:3/746: dread d3/d14/d37/d1a/d1d/f75 [0,4194304] 0 2024-09-16T08:57:26.075 INFO:tasks.workunit.client.0.smithi031.stdout:3/747: fsync d3/d14/d37/d47/fcd 0 2024-09-16T08:57:26.075 INFO:tasks.workunit.client.0.smithi031.stdout:1/653: creat d2/dae/db9/fe6 x:0 0 0 2024-09-16T08:57:26.078 INFO:tasks.workunit.client.0.smithi031.stdout:0/713: write d6/d12/d47/d7c/fdd [2858972,32304] 0 2024-09-16T08:57:26.078 INFO:tasks.workunit.client.0.smithi031.stdout:0/714: fdatasync d6/d12/d13/f3e 0 2024-09-16T08:57:26.079 INFO:tasks.workunit.client.0.smithi031.stdout:8/567: truncate d4/d7/d21/d37/d46/db1/f6a 274022 0 2024-09-16T08:57:26.079 INFO:tasks.workunit.client.0.smithi031.stdout:8/568: creat d4/d2f/fbc x:0 0 0 2024-09-16T08:57:26.080 INFO:tasks.workunit.client.0.smithi031.stdout:8/569: write d4/d5b/d7b/f7d [4749765,66913] 0 2024-09-16T08:57:26.080 INFO:tasks.workunit.client.0.smithi031.stdout:6/774: creat d0/dd/d74/d5e/d7c/d3f/d65/fef x:0 0 0 2024-09-16T08:57:26.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/633: symlink d2/dc/d3d/db2/lce 0 2024-09-16T08:57:26.085 INFO:tasks.workunit.client.0.smithi031.stdout:3/748: dread d3/d14/d37/d1a/d1d/d3b/d3f/f74 [0,4194304] 0 2024-09-16T08:57:26.088 INFO:tasks.workunit.client.0.smithi031.stdout:6/775: symlink d0/dd/d74/d19/d54/lf0 0 2024-09-16T08:57:26.088 INFO:tasks.workunit.client.0.smithi031.stdout:6/776: getdents d0/dd/d35/d6c/dac 0 2024-09-16T08:57:26.088 INFO:tasks.workunit.client.0.smithi031.stdout:3/749: symlink d3/d14/d37/d33/d97/ld7 0 2024-09-16T08:57:26.089 INFO:tasks.workunit.client.0.smithi031.stdout:3/750: fsync d3/d14/d37/d1a/d1d/d3b/dc4/fca 0 2024-09-16T08:57:26.089 INFO:tasks.workunit.client.0.smithi031.stdout:1/654: symlink d2/de/db5/le7 0 2024-09-16T08:57:26.089 INFO:tasks.workunit.client.0.smithi031.stdout:6/777: mkdir d0/df1 0 2024-09-16T08:57:26.089 INFO:tasks.workunit.client.0.smithi031.stdout:6/778: fdatasync d0/dd/d74/d19/f61 0 2024-09-16T08:57:26.089 INFO:tasks.workunit.client.0.smithi031.stdout:6/779: truncate d0/fd7 461583 0 2024-09-16T08:57:26.091 INFO:tasks.workunit.client.0.smithi031.stdout:3/751: link d3/d14/c53 d3/d14/d37/d1a/d1d/d1e/d67/dd6/dd0/cd8 0 2024-09-16T08:57:26.094 INFO:tasks.workunit.client.0.smithi031.stdout:3/752: truncate d3/d14/d37/d33/f3c 656253 0 2024-09-16T08:57:26.094 INFO:tasks.workunit.client.0.smithi031.stdout:3/753: write d3/d14/d37/d1a/d1d/d1e/d67/dd6/dd0/faf [73054,10569] 0 2024-09-16T08:57:26.097 INFO:tasks.workunit.client.0.smithi031.stdout:3/754: creat d3/d14/d37/d33/fd9 x:0 0 0 2024-09-16T08:57:26.098 INFO:tasks.workunit.client.0.smithi031.stdout:0/715: dwrite d6/d61/d91/fc3 [0,4194304] 0 2024-09-16T08:57:26.103 INFO:tasks.workunit.client.0.smithi031.stdout:3/755: dread d3/d14/fb6 [0,4194304] 0 2024-09-16T08:57:26.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:25 smithi031 ceph-mon[27844]: pgmap v14: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 328 MiB/s rd, 421 MiB/s wr, 591 op/s 2024-09-16T08:57:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:25 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:25 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:26.127 INFO:tasks.workunit.client.0.smithi031.stdout:1/655: dwrite d2/d4/d67/fbc [0,4194304] 0 2024-09-16T08:57:26.127 INFO:tasks.workunit.client.0.smithi031.stdout:6/780: dwrite d0/dd/d74/d5e/d7c/d3f/f4b [0,4194304] 0 2024-09-16T08:57:26.128 INFO:tasks.workunit.client.0.smithi031.stdout:3/756: dwrite d3/d14/d37/d1a/d1d/d1e/d67/dd6/dd0/fc8 [0,4194304] 0 2024-09-16T08:57:26.129 INFO:tasks.workunit.client.0.smithi031.stdout:1/656: getdents d2/d15/d5d/db4/da3 0 2024-09-16T08:57:26.129 INFO:tasks.workunit.client.0.smithi031.stdout:1/657: chown d2/d4/d52/fe0 25 1 2024-09-16T08:57:26.129 INFO:tasks.workunit.client.0.smithi031.stdout:0/716: dwrite d6/d12/d97/db3/fb5 [0,4194304] 0 2024-09-16T08:57:26.132 INFO:tasks.workunit.client.0.smithi031.stdout:1/658: creat d2/d15/d5d/db4/da3/fe8 x:0 0 0 2024-09-16T08:57:26.138 INFO:tasks.workunit.client.0.smithi031.stdout:6/781: write d0/dd/d74/d19/f61 [1755932,4028] 0 2024-09-16T08:57:26.138 INFO:tasks.workunit.client.0.smithi031.stdout:6/782: stat d0/dd/d74/d5e/d7c/c3d 0 2024-09-16T08:57:26.142 INFO:tasks.workunit.client.0.smithi031.stdout:3/757: dread d3/d14/d37/d1a/d1d/d3b/f4f [0,4194304] 0 2024-09-16T08:57:26.142 INFO:tasks.workunit.client.0.smithi031.stdout:3/758: stat d3/d14/d37/d1a/d1d/d1e/d67/dd6/dd0/d7d/l89 0 2024-09-16T08:57:26.142 INFO:tasks.workunit.client.0.smithi031.stdout:0/717: unlink d6/d12/d47/d4d/f7a 0 2024-09-16T08:57:26.142 INFO:tasks.workunit.client.0.smithi031.stdout:1/659: dread d2/d4/f54 [0,4194304] 0 2024-09-16T08:57:26.142 INFO:tasks.workunit.client.0.smithi031.stdout:6/783: dread d0/dd/d74/d5e/d7c/d56/f78 [0,4194304] 0 2024-09-16T08:57:26.142 INFO:tasks.workunit.client.0.smithi031.stdout:6/784: fsync d0/dd/d74/d19/f85 0 2024-09-16T08:57:26.143 INFO:tasks.workunit.client.0.smithi031.stdout:6/785: chown d0/fd7 1017680 1 2024-09-16T08:57:26.144 INFO:tasks.workunit.client.0.smithi031.stdout:0/718: mknod d6/dd3/ce4 0 2024-09-16T08:57:26.147 INFO:tasks.workunit.client.0.smithi031.stdout:0/719: link d6/d61/c75 d6/d12/d47/d4d/ce5 0 2024-09-16T08:57:26.158 INFO:tasks.workunit.client.0.smithi031.stdout:0/720: dread d6/d12/f16 [0,4194304] 0 2024-09-16T08:57:26.159 INFO:tasks.workunit.client.0.smithi031.stdout:1/660: rename d2/d4 to d2/d15/d1f/de9 0 2024-09-16T08:57:26.161 INFO:tasks.workunit.client.0.smithi031.stdout:3/759: dwrite d3/d14/d37/d1a/d1d/d1e/d67/dd6/dd0/faf [0,4194304] 0 2024-09-16T08:57:26.162 INFO:tasks.workunit.client.0.smithi031.stdout:9/741: sync 2024-09-16T08:57:26.162 INFO:tasks.workunit.client.0.smithi031.stdout:7/751: sync 2024-09-16T08:57:26.162 INFO:tasks.workunit.client.0.smithi031.stdout:7/752: chown d2/d8/d11/d5d/lc3 455514132 1 2024-09-16T08:57:26.162 INFO:tasks.workunit.client.0.smithi031.stdout:1/661: truncate d2/f14 3179489 0 2024-09-16T08:57:26.162 INFO:tasks.workunit.client.0.smithi031.stdout:1/662: stat d2/l2d 0 2024-09-16T08:57:26.167 INFO:tasks.workunit.client.0.smithi031.stdout:7/753: creat d2/d2d/ff4 x:0 0 0 2024-09-16T08:57:26.168 INFO:tasks.workunit.client.0.smithi031.stdout:7/754: chown d2/d8/d69/d82/fa0 5992109 1 2024-09-16T08:57:26.168 INFO:tasks.workunit.client.0.smithi031.stdout:7/755: fdatasync d2/d8/d11/d66/f72 0 2024-09-16T08:57:26.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/760: write d3/d14/d37/d1a/d1d/d3b/dc4/fca [2039108,50578] 0 2024-09-16T08:57:26.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/761: read - d3/d14/d37/d1a/d1d/d3b/f8b zero size 2024-09-16T08:57:26.173 INFO:tasks.workunit.client.0.smithi031.stdout:7/756: stat d2/d8/d11/l19 0 2024-09-16T08:57:26.173 INFO:tasks.workunit.client.0.smithi031.stdout:0/721: write d6/d12/d13/f39 [2373568,34181] 0 2024-09-16T08:57:26.174 INFO:tasks.workunit.client.0.smithi031.stdout:9/742: dread d3/d5/da/f3e [0,4194304] 0 2024-09-16T08:57:26.178 INFO:tasks.workunit.client.0.smithi031.stdout:3/762: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d79/d7c/lda 0 2024-09-16T08:57:26.178 INFO:tasks.workunit.client.0.smithi031.stdout:3/763: write d3/d14/d37/d1a/d24/f40 [922011,27013] 0 2024-09-16T08:57:26.181 INFO:tasks.workunit.client.0.smithi031.stdout:9/743: dread d3/d48/f68 [4194304,4194304] 0 2024-09-16T08:57:26.181 INFO:tasks.workunit.client.0.smithi031.stdout:0/722: truncate d6/dac/fd4 804333 0 2024-09-16T08:57:26.183 INFO:tasks.workunit.client.0.smithi031.stdout:0/723: mkdir d6/d12/d47/dc5/de6 0 2024-09-16T08:57:26.183 INFO:tasks.workunit.client.0.smithi031.stdout:0/724: chown d6/d12/d47/c8d 330 1 2024-09-16T08:57:26.184 INFO:tasks.workunit.client.0.smithi031.stdout:1/663: dwrite d2/de/f79 [4194304,4194304] 0 2024-09-16T08:57:26.184 INFO:tasks.workunit.client.0.smithi031.stdout:1/664: chown d2/d15/d1f/de9/d1b/de4 8 1 2024-09-16T08:57:26.184 INFO:tasks.workunit.client.0.smithi031.stdout:0/725: write d6/d12/d13/f84 [385911,127862] 0 2024-09-16T08:57:26.185 INFO:tasks.workunit.client.0.smithi031.stdout:1/665: rmdir d2/de/d6c 39 2024-09-16T08:57:26.185 INFO:tasks.workunit.client.0.smithi031.stdout:1/666: readlink d2/de/db5/lc8 0 2024-09-16T08:57:26.186 INFO:tasks.workunit.client.0.smithi031.stdout:0/726: truncate d6/d12/d13/d36/f56 2407801 0 2024-09-16T08:57:26.188 INFO:tasks.workunit.client.0.smithi031.stdout:9/744: write d3/d5/da/db/d21/f41 [365833,32687] 0 2024-09-16T08:57:26.188 INFO:tasks.workunit.client.0.smithi031.stdout:7/757: dread d2/d8/d11/d5d/d7e/fe0 [0,4194304] 0 2024-09-16T08:57:26.188 INFO:tasks.workunit.client.0.smithi031.stdout:1/667: rename d2/d15/d1f/de9/d36 to d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea 0 2024-09-16T08:57:26.190 INFO:tasks.workunit.client.0.smithi031.stdout:1/668: truncate d2/d15/d1f/de9/d52/fcc 3865383 0 2024-09-16T08:57:26.194 INFO:tasks.workunit.client.0.smithi031.stdout:1/669: symlink d2/leb 0 2024-09-16T08:57:26.195 INFO:tasks.workunit.client.0.smithi031.stdout:9/745: dread d3/d5/da/f89 [0,4194304] 0 2024-09-16T08:57:26.198 INFO:tasks.workunit.client.0.smithi031.stdout:1/670: mknod d2/d15/d1f/cec 0 2024-09-16T08:57:26.200 INFO:tasks.workunit.client.0.smithi031.stdout:9/746: creat d3/d5/dd6/f10a x:0 0 0 2024-09-16T08:57:26.205 INFO:tasks.workunit.client.0.smithi031.stdout:9/747: creat d3/d5/da/d34/d3c/d5e/d9c/d95/f10b x:0 0 0 2024-09-16T08:57:26.205 INFO:tasks.workunit.client.0.smithi031.stdout:1/671: symlink d2/d15/d5d/db4/da3/led 0 2024-09-16T08:57:26.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/672: creat d2/d15/d5d/db4/da3/dcb/fee x:0 0 0 2024-09-16T08:57:26.209 INFO:tasks.workunit.client.0.smithi031.stdout:1/673: truncate d2/d15/d1f/de9/d1b/f65 1397748 0 2024-09-16T08:57:26.211 INFO:tasks.workunit.client.0.smithi031.stdout:1/674: mkdir d2/d15/d5d/def 0 2024-09-16T08:57:26.211 INFO:tasks.workunit.client.0.smithi031.stdout:1/675: truncate d2/d15/d1f/de9/d1b/f65 2084779 0 2024-09-16T08:57:26.212 INFO:tasks.workunit.client.0.smithi031.stdout:3/764: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/d49/faa [0,4194304] 0 2024-09-16T08:57:26.213 INFO:tasks.workunit.client.0.smithi031.stdout:1/676: dread d2/d15/d1f/de9/d1b/f2b [0,4194304] 0 2024-09-16T08:57:26.213 INFO:tasks.workunit.client.0.smithi031.stdout:0/727: dwrite d6/f8 [0,4194304] 0 2024-09-16T08:57:26.213 INFO:tasks.workunit.client.0.smithi031.stdout:7/758: dwrite d2/d8/d11/d66/d9a/fde [4194304,4194304] 0 2024-09-16T08:57:26.219 INFO:tasks.workunit.client.0.smithi031.stdout:3/765: symlink d3/d14/d37/d1a/d1d/d1e/ldb 0 2024-09-16T08:57:26.220 INFO:tasks.workunit.client.0.smithi031.stdout:3/766: readlink d3/d14/d37/d1a/d1d/l86 0 2024-09-16T08:57:26.220 INFO:tasks.workunit.client.0.smithi031.stdout:3/767: creat d3/d14/d37/d1a/d1d/dac/fdc x:0 0 0 2024-09-16T08:57:26.220 INFO:tasks.workunit.client.0.smithi031.stdout:7/759: dread d2/d8/d69/f6c [0,4194304] 0 2024-09-16T08:57:26.220 INFO:tasks.workunit.client.0.smithi031.stdout:7/760: dread - d2/d8/f87 zero size 2024-09-16T08:57:26.223 INFO:tasks.workunit.client.0.smithi031.stdout:0/728: mknod d6/d12/d47/ce7 0 2024-09-16T08:57:26.237 INFO:tasks.workunit.client.0.smithi031.stdout:0/729: mknod d6/d12/d97/ce8 0 2024-09-16T08:57:26.239 INFO:tasks.workunit.client.0.smithi031.stdout:0/730: creat d6/d61/fe9 x:0 0 0 2024-09-16T08:57:26.242 INFO:tasks.workunit.client.0.smithi031.stdout:1/677: dwrite d2/d15/d1f/de9/d1b/f50 [0,4194304] 0 2024-09-16T08:57:26.243 INFO:tasks.workunit.client.0.smithi031.stdout:0/731: dread d6/d12/d47/d5b/f67 [0,4194304] 0 2024-09-16T08:57:26.244 INFO:tasks.workunit.client.0.smithi031.stdout:1/678: truncate d2/d15/d4a/f6a 3241591 0 2024-09-16T08:57:26.245 INFO:tasks.workunit.client.0.smithi031.stdout:0/732: truncate d6/d12/d97/db3/dba/fdc 83120 0 2024-09-16T08:57:26.246 INFO:tasks.workunit.client.0.smithi031.stdout:1/679: mkdir d2/d15/d1f/de9/d1b/d5c/d6b/dc1/df0 0 2024-09-16T08:57:26.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/768: dwrite d3/d14/d37/d33/f7a [0,4194304] 0 2024-09-16T08:57:26.247 INFO:tasks.workunit.client.0.smithi031.stdout:3/769: chown d3/d14/fb6 7240 1 2024-09-16T08:57:26.248 INFO:tasks.workunit.client.0.smithi031.stdout:7/761: dwrite d2/d2d/ff4 [0,4194304] 0 2024-09-16T08:57:26.248 INFO:tasks.workunit.client.0.smithi031.stdout:7/762: write d2/d8/f3f [5095444,15050] 0 2024-09-16T08:57:26.249 INFO:tasks.workunit.client.0.smithi031.stdout:0/733: dread d6/d12/d13/f46 [0,4194304] 0 2024-09-16T08:57:26.252 INFO:tasks.workunit.client.0.smithi031.stdout:3/770: write d3/d14/d37/d1a/d1d/d3b/f4f [1438620,71733] 0 2024-09-16T08:57:26.254 INFO:tasks.workunit.client.0.smithi031.stdout:0/734: dread d6/d12/d13/d36/fd8 [0,4194304] 0 2024-09-16T08:57:26.254 INFO:tasks.workunit.client.0.smithi031.stdout:0/735: stat d6/d12/d47/fe2 0 2024-09-16T08:57:26.258 INFO:tasks.workunit.client.0.smithi031.stdout:0/736: dread d6/f7b [0,4194304] 0 2024-09-16T08:57:26.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:25 smithi097 ceph-mon[35478]: pgmap v14: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 328 MiB/s rd, 421 MiB/s wr, 591 op/s 2024-09-16T08:57:26.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:25 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:26.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:25 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:26.262 INFO:tasks.workunit.client.0.smithi031.stdout:0/737: dread d6/d12/d13/f24 [0,4194304] 0 2024-09-16T08:57:26.269 INFO:tasks.workunit.client.0.smithi031.stdout:0/738: truncate d6/dac/fd4 481471 0 2024-09-16T08:57:26.269 INFO:tasks.workunit.client.0.smithi031.stdout:0/739: chown d6/d12/d13/f7f 1507 1 2024-09-16T08:57:26.272 INFO:tasks.workunit.client.0.smithi031.stdout:7/763: write d2/f5b [403882,116254] 0 2024-09-16T08:57:26.272 INFO:tasks.workunit.client.0.smithi031.stdout:7/764: chown d2/d8/d11/d66/d9a/fc9 1 1 2024-09-16T08:57:26.274 INFO:tasks.workunit.client.0.smithi031.stdout:3/771: dwrite d3/d14/d37/d1a/d1d/d1e/f80 [0,4194304] 0 2024-09-16T08:57:26.276 INFO:tasks.workunit.client.0.smithi031.stdout:7/765: truncate d2/d2d/f4b 35514 0 2024-09-16T08:57:26.276 INFO:tasks.workunit.client.0.smithi031.stdout:7/766: stat d2/d8/d69 0 2024-09-16T08:57:26.276 INFO:tasks.workunit.client.0.smithi031.stdout:7/767: fsync d2/d8/d69/d82/fa0 0 2024-09-16T08:57:26.276 INFO:tasks.workunit.client.0.smithi031.stdout:7/768: creat d2/d2d/d6d/ff5 x:0 0 0 2024-09-16T08:57:26.278 INFO:tasks.workunit.client.0.smithi031.stdout:0/740: mknod d6/d12/d13/cea 0 2024-09-16T08:57:26.344 INFO:tasks.workunit.client.0.smithi031.stdout:3/772: unlink d3/d14/d37/d1a/d1d/d1e/f80 0 2024-09-16T08:57:26.345 INFO:tasks.workunit.client.0.smithi031.stdout:3/773: dread d3/d14/d37/d1a/d1d/d9a/fa2 [0,4194304] 0 2024-09-16T08:57:26.357 INFO:tasks.workunit.client.0.smithi031.stdout:0/741: mkdir d6/d12/d13/dca/deb 0 2024-09-16T08:57:26.358 INFO:tasks.workunit.client.0.smithi031.stdout:3/774: stat d3/d14/d37/d1a/d24/c36 0 2024-09-16T08:57:26.364 INFO:tasks.workunit.client.0.smithi031.stdout:0/742: creat d6/d12/d47/d5b/d9b/fec x:0 0 0 2024-09-16T08:57:26.364 INFO:tasks.workunit.client.0.smithi031.stdout:0/743: chown d6/dac/ld2 1 1 2024-09-16T08:57:26.365 INFO:tasks.workunit.client.0.smithi031.stdout:3/775: getdents d3/d14/d37/d1a/d1d/d3b/d94 0 2024-09-16T08:57:26.365 INFO:tasks.workunit.client.0.smithi031.stdout:3/776: chown d3/l5e 6609994 1 2024-09-16T08:57:26.366 INFO:tasks.workunit.client.0.smithi031.stdout:0/744: mknod d6/dd3/ced 0 2024-09-16T08:57:26.366 INFO:tasks.workunit.client.0.smithi031.stdout:0/745: fdatasync d6/d12/d13/d36/fd1 0 2024-09-16T08:57:26.367 INFO:tasks.workunit.client.0.smithi031.stdout:3/777: mkdir d3/d14/d37/d1a/d1d/d1e/d67/dd6/dd0/d82/ddd 0 2024-09-16T08:57:26.369 INFO:tasks.workunit.client.0.smithi031.stdout:0/746: creat d6/d12/d47/d5b/fee x:0 0 0 2024-09-16T08:57:26.370 INFO:tasks.workunit.client.0.smithi031.stdout:0/747: mknod d6/d12/d47/cef 0 2024-09-16T08:57:26.370 INFO:tasks.workunit.client.0.smithi031.stdout:0/748: fdatasync d6/d12/d47/d5b/d9b/fec 0 2024-09-16T08:57:26.371 INFO:tasks.workunit.client.0.smithi031.stdout:0/749: symlink d6/d61/lf0 0 2024-09-16T08:57:26.371 INFO:tasks.workunit.client.0.smithi031.stdout:7/769: dwrite d2/d8/d11/d5d/d6e/f95 [0,4194304] 0 2024-09-16T08:57:26.371 INFO:tasks.workunit.client.0.smithi031.stdout:7/770: chown d2/d8/d11 214465 1 2024-09-16T08:57:26.374 INFO:tasks.workunit.client.0.smithi031.stdout:7/771: readlink d2/d8/d11/l16 0 2024-09-16T08:57:26.382 INFO:tasks.workunit.client.0.smithi031.stdout:0/750: read d6/f4f [107778,126468] 0 2024-09-16T08:57:26.384 INFO:tasks.workunit.client.0.smithi031.stdout:7/772: dread d2/d8/f60 [0,4194304] 0 2024-09-16T08:57:26.387 INFO:tasks.workunit.client.0.smithi031.stdout:0/751: symlink d6/lf1 0 2024-09-16T08:57:26.388 INFO:tasks.workunit.client.0.smithi031.stdout:2/684: sync 2024-09-16T08:57:26.388 INFO:tasks.workunit.client.0.smithi031.stdout:4/843: sync 2024-09-16T08:57:26.391 INFO:tasks.workunit.client.0.smithi031.stdout:3/778: dwrite d3/d14/d37/d33/f39 [0,4194304] 0 2024-09-16T08:57:26.394 INFO:tasks.workunit.client.0.smithi031.stdout:2/685: getdents d6/d2e/d40/dc3 0 2024-09-16T08:57:26.395 INFO:tasks.workunit.client.0.smithi031.stdout:2/686: write d6/da/f1d [59022,83328] 0 2024-09-16T08:57:26.401 INFO:tasks.workunit.client.0.smithi031.stdout:7/773: dread d2/f1a [0,4194304] 0 2024-09-16T08:57:26.403 INFO:tasks.workunit.client.0.smithi031.stdout:7/774: creat d2/d2d/d68/ff6 x:0 0 0 2024-09-16T08:57:26.404 INFO:tasks.workunit.client.0.smithi031.stdout:3/779: dread d3/d14/d37/d1a/d1d/d1e/d67/dd6/dd0/fc8 [0,4194304] 0 2024-09-16T08:57:26.405 INFO:tasks.workunit.client.0.smithi031.stdout:7/775: creat d2/d8/d69/ff7 x:0 0 0 2024-09-16T08:57:26.405 INFO:tasks.workunit.client.0.smithi031.stdout:3/780: dread d3/d14/d37/f84 [0,4194304] 0 2024-09-16T08:57:26.406 INFO:tasks.workunit.client.0.smithi031.stdout:2/687: dread d6/da/d4b/d4c/d68/fa8 [0,4194304] 0 2024-09-16T08:57:26.407 INFO:tasks.workunit.client.0.smithi031.stdout:7/776: creat d2/ff8 x:0 0 0 2024-09-16T08:57:26.407 INFO:tasks.workunit.client.0.smithi031.stdout:7/777: readlink d2/d2d/l38 0 2024-09-16T08:57:26.409 INFO:tasks.workunit.client.0.smithi031.stdout:0/752: dwrite d6/f7 [0,4194304] 0 2024-09-16T08:57:26.410 INFO:tasks.workunit.client.0.smithi031.stdout:4/844: dwrite d0/d3c/d6b/fe5 [0,4194304] 0 2024-09-16T08:57:26.413 INFO:tasks.workunit.client.0.smithi031.stdout:2/688: mkdir d6/d20/da1/dcf/dda/de3 0 2024-09-16T08:57:26.413 INFO:tasks.workunit.client.0.smithi031.stdout:2/689: readlink d6/d2e/d40/d6c/d2c/d53/lce 0 2024-09-16T08:57:26.415 INFO:tasks.workunit.client.0.smithi031.stdout:3/781: rename d3/d14/d37/d1a/d1d/d1e/d67 to d3/d14/d37/d1a/d1d/d9a/dde 0 2024-09-16T08:57:26.419 INFO:tasks.workunit.client.0.smithi031.stdout:0/753: unlink d6/d12/d13/f68 0 2024-09-16T08:57:26.419 INFO:tasks.workunit.client.0.smithi031.stdout:0/754: creat d6/d12/d97/db3/ff2 x:0 0 0 2024-09-16T08:57:26.421 INFO:tasks.workunit.client.0.smithi031.stdout:4/845: write d0/f18 [1394339,124419] 0 2024-09-16T08:57:26.426 INFO:tasks.workunit.client.0.smithi031.stdout:3/782: creat d3/d14/d37/d1a/d1d/d6a/fdf x:0 0 0 2024-09-16T08:57:26.430 INFO:tasks.workunit.client.0.smithi031.stdout:0/755: creat d6/d12/d13/ff3 x:0 0 0 2024-09-16T08:57:26.431 INFO:tasks.workunit.client.0.smithi031.stdout:8/570: sync 2024-09-16T08:57:26.431 INFO:tasks.workunit.client.0.smithi031.stdout:8/571: write d4/d7/d21/d35/d79/f8a [248817,1311] 0 2024-09-16T08:57:26.432 INFO:tasks.workunit.client.0.smithi031.stdout:2/690: dwrite d6/da/d4b/da0/fde [0,4194304] 0 2024-09-16T08:57:26.432 INFO:tasks.workunit.client.0.smithi031.stdout:2/691: stat d6/da/d4b/d4c 0 2024-09-16T08:57:26.433 INFO:tasks.workunit.client.0.smithi031.stdout:4/846: creat d0/d102/d84/d86/d9b/f10f x:0 0 0 2024-09-16T08:57:26.434 INFO:tasks.workunit.client.0.smithi031.stdout:0/756: symlink d6/d61/lf4 0 2024-09-16T08:57:26.437 INFO:tasks.workunit.client.0.smithi031.stdout:7/778: dwrite d2/d8/d11/d5d/d7e/db6/fd2 [0,4194304] 0 2024-09-16T08:57:26.437 INFO:tasks.workunit.client.0.smithi031.stdout:8/572: unlink d4/d9e/l53 0 2024-09-16T08:57:26.441 INFO:tasks.workunit.client.0.smithi031.stdout:4/847: dread d0/f1 [0,4194304] 0 2024-09-16T08:57:26.442 INFO:tasks.workunit.client.0.smithi031.stdout:2/692: dread d6/da/d37/f87 [0,4194304] 0 2024-09-16T08:57:26.444 INFO:tasks.workunit.client.0.smithi031.stdout:0/757: rename d6/d12/d47/fb9 to d6/d12/d47/d5b/d9b/da2/ff5 0 2024-09-16T08:57:26.448 INFO:tasks.workunit.client.0.smithi031.stdout:8/573: dread d4/d7/d21/d35/f5d [0,4194304] 0 2024-09-16T08:57:26.451 INFO:tasks.workunit.client.0.smithi031.stdout:3/783: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 [4194304,4194304] 0 2024-09-16T08:57:26.471 INFO:tasks.workunit.client.0.smithi031.stdout:2/693: symlink d6/d20/da1/le4 0 2024-09-16T08:57:26.473 INFO:tasks.workunit.client.0.smithi031.stdout:7/779: rename d2/d2d/d68/ff6 to d2/d8/d86/ff9 0 2024-09-16T08:57:26.473 INFO:tasks.workunit.client.0.smithi031.stdout:7/780: stat d2/d2d/d68/da2/lbd 0 2024-09-16T08:57:26.476 INFO:tasks.workunit.client.0.smithi031.stdout:0/758: symlink d6/d61/lf6 0 2024-09-16T08:57:26.479 INFO:tasks.workunit.client.0.smithi031.stdout:2/694: write d6/da/d37/d8b/f8c [2442216,60560] 0 2024-09-16T08:57:26.490 INFO:tasks.workunit.client.0.smithi031.stdout:3/784: mknod d3/ce0 0 2024-09-16T08:57:26.492 INFO:tasks.workunit.client.0.smithi031.stdout:7/781: creat d2/d8/d11/d66/d9a/d9c/ffa x:0 0 0 2024-09-16T08:57:26.492 INFO:tasks.workunit.client.0.smithi031.stdout:7/782: fdatasync d2/d2d/d6d/fc6 0 2024-09-16T08:57:26.492 INFO:tasks.workunit.client.0.smithi031.stdout:7/783: creat d2/d2d/d6d/ffb x:0 0 0 2024-09-16T08:57:26.495 INFO:tasks.workunit.client.0.smithi031.stdout:0/759: mknod d6/dac/cf7 0 2024-09-16T08:57:26.499 INFO:tasks.workunit.client.0.smithi031.stdout:2/695: creat d6/da/d37/fe5 x:0 0 0 2024-09-16T08:57:26.500 INFO:tasks.workunit.client.0.smithi031.stdout:5/634: sync 2024-09-16T08:57:26.500 INFO:tasks.workunit.client.0.smithi031.stdout:6/786: sync 2024-09-16T08:57:26.500 INFO:tasks.workunit.client.0.smithi031.stdout:6/787: chown d0/dd/d74/dad 1 1 2024-09-16T08:57:26.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/574: getdents d4/d77 0 2024-09-16T08:57:26.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/575: write d4/d7/fad [556245,55612] 0 2024-09-16T08:57:26.501 INFO:tasks.workunit.client.0.smithi031.stdout:8/576: creat d4/d2f/fbd x:0 0 0 2024-09-16T08:57:26.507 INFO:tasks.workunit.client.0.smithi031.stdout:2/696: creat d6/d20/d51/ddc/fe6 x:0 0 0 2024-09-16T08:57:26.508 INFO:tasks.workunit.client.0.smithi031.stdout:6/788: mknod d0/dd/d35/d6c/d83/cf2 0 2024-09-16T08:57:26.509 INFO:tasks.workunit.client.0.smithi031.stdout:8/577: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:26.510 INFO:tasks.workunit.client.0.smithi031.stdout:7/784: link d2/d8/fd d2/ffc 0 2024-09-16T08:57:26.511 INFO:tasks.workunit.client.0.smithi031.stdout:6/789: mkdir d0/dd/df3 0 2024-09-16T08:57:26.511 INFO:tasks.workunit.client.0.smithi031.stdout:6/790: creat d0/dd/d74/d5e/d7c/d3f/d92/ff4 x:0 0 0 2024-09-16T08:57:26.512 INFO:tasks.workunit.client.0.smithi031.stdout:8/578: creat d4/d7/d21/d35/fbe x:0 0 0 2024-09-16T08:57:26.515 INFO:tasks.workunit.client.0.smithi031.stdout:6/791: mknod d0/dd/d74/cf5 0 2024-09-16T08:57:26.515 INFO:tasks.workunit.client.0.smithi031.stdout:6/792: write d0/dd/d74/dad/fc9 [3344162,69193] 0 2024-09-16T08:57:26.516 INFO:tasks.workunit.client.0.smithi031.stdout:8/579: symlink d4/d7/d21/d35/d50/da3/lbf 0 2024-09-16T08:57:26.516 INFO:tasks.workunit.client.0.smithi031.stdout:8/580: fsync d4/d7/d21/d35/d50/f98 0 2024-09-16T08:57:26.516 INFO:tasks.workunit.client.0.smithi031.stdout:5/635: write d2/f7f [4056418,90589] 0 2024-09-16T08:57:26.517 INFO:tasks.workunit.client.0.smithi031.stdout:5/636: fsync d2/dc/d25/d21/d81/dc3/fc6 0 2024-09-16T08:57:26.517 INFO:tasks.workunit.client.0.smithi031.stdout:0/760: dwrite d6/d12/d97/db3/dba/fdc [0,4194304] 0 2024-09-16T08:57:26.517 INFO:tasks.workunit.client.0.smithi031.stdout:8/581: getdents d4/d7/d99 0 2024-09-16T08:57:26.518 INFO:tasks.workunit.client.0.smithi031.stdout:8/582: stat d4/d7/d21/d35/d50/l5f 0 2024-09-16T08:57:26.518 INFO:tasks.workunit.client.0.smithi031.stdout:6/793: dread d0/dd/d74/d5e/d7c/d3f/d92/fa4 [0,4194304] 0 2024-09-16T08:57:26.519 INFO:tasks.workunit.client.0.smithi031.stdout:0/761: symlink d6/d61/lf8 0 2024-09-16T08:57:26.520 INFO:tasks.workunit.client.0.smithi031.stdout:0/762: creat d6/d61/d91/ff9 x:0 0 0 2024-09-16T08:57:26.525 INFO:tasks.workunit.client.0.smithi031.stdout:6/794: mkdir d0/d10/dc8/df6 0 2024-09-16T08:57:26.525 INFO:tasks.workunit.client.0.smithi031.stdout:6/795: truncate d0/dd/d74/d5e/fed 195286 0 2024-09-16T08:57:26.531 INFO:tasks.workunit.client.0.smithi031.stdout:0/763: truncate d6/d61/d91/fc3 2552993 0 2024-09-16T08:57:26.533 INFO:tasks.workunit.client.0.smithi031.stdout:3/785: truncate d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 6421735 0 2024-09-16T08:57:26.534 INFO:tasks.workunit.client.0.smithi031.stdout:8/583: rmdir d4/d2f 39 2024-09-16T08:57:26.536 INFO:tasks.workunit.client.0.smithi031.stdout:6/796: dread d0/d10/f60 [0,4194304] 0 2024-09-16T08:57:26.537 INFO:tasks.workunit.client.0.smithi031.stdout:6/797: write d0/dd/d74/d19/d54/f9c [606342,56700] 0 2024-09-16T08:57:26.537 INFO:tasks.workunit.client.0.smithi031.stdout:0/764: mkdir d6/d12/d13/da5/dfa 0 2024-09-16T08:57:26.537 INFO:tasks.workunit.client.0.smithi031.stdout:0/765: chown d6/d12/d97/d9e/fb8 196254 1 2024-09-16T08:57:26.537 INFO:tasks.workunit.client.0.smithi031.stdout:2/697: dwrite d6/da/d5e/d72/fcb [0,4194304] 0 2024-09-16T08:57:26.537 INFO:tasks.workunit.client.0.smithi031.stdout:2/698: write d6/d2e/d40/d6c/d2c/d53/d60/f35 [493914,92318] 0 2024-09-16T08:57:26.538 INFO:tasks.workunit.client.0.smithi031.stdout:7/785: dwrite d2/d2d/d6d/fed [0,4194304] 0 2024-09-16T08:57:26.538 INFO:tasks.workunit.client.0.smithi031.stdout:7/786: readlink d2/d8/d11/l1f 0 2024-09-16T08:57:26.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/786: link d3/d14/d37/d33/d97/ld7 d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/le1 0 2024-09-16T08:57:26.538 INFO:tasks.workunit.client.0.smithi031.stdout:3/787: fsync d3/d14/d37/d47/f66 0 2024-09-16T08:57:26.539 INFO:tasks.workunit.client.0.smithi031.stdout:6/798: dread d0/dd/d2b/f45 [0,4194304] 0 2024-09-16T08:57:26.540 INFO:tasks.workunit.client.0.smithi031.stdout:0/766: mknod d6/d12/d13/da5/cfb 0 2024-09-16T08:57:26.541 INFO:tasks.workunit.client.0.smithi031.stdout:5/637: dwrite d2/dc/d3d/d4c/f7a [0,4194304] 0 2024-09-16T08:57:26.545 INFO:tasks.workunit.client.0.smithi031.stdout:2/699: rename d6/f36 to d6/da/d4b/d4c/fe7 0 2024-09-16T08:57:26.551 INFO:tasks.workunit.client.0.smithi031.stdout:2/700: dread d6/da/f6a [0,4194304] 0 2024-09-16T08:57:26.554 INFO:tasks.workunit.client.0.smithi031.stdout:3/788: symlink d3/d14/d37/d1a/d1d/d9a/dde/d7f/le2 0 2024-09-16T08:57:26.557 INFO:tasks.workunit.client.0.smithi031.stdout:6/799: symlink d0/d10/dc8/lf7 0 2024-09-16T08:57:26.558 INFO:tasks.workunit.client.0.smithi031.stdout:0/767: readlink d6/l25 0 2024-09-16T08:57:26.558 INFO:tasks.workunit.client.0.smithi031.stdout:0/768: dread - d6/d12/d13/d36/fa6 zero size 2024-09-16T08:57:26.559 INFO:tasks.workunit.client.0.smithi031.stdout:0/769: write d6/d12/d97/d9e/fd6 [746671,94033] 0 2024-09-16T08:57:26.559 INFO:tasks.workunit.client.0.smithi031.stdout:2/701: dread d6/da/f27 [0,4194304] 0 2024-09-16T08:57:26.560 INFO:tasks.workunit.client.0.smithi031.stdout:2/702: chown d6/da/c21 21448224 1 2024-09-16T08:57:26.568 INFO:tasks.workunit.client.0.smithi031.stdout:7/787: dwrite d2/f18 [0,4194304] 0 2024-09-16T08:57:26.576 INFO:tasks.workunit.client.0.smithi031.stdout:0/770: unlink d6/d12/d47/d7c/c86 0 2024-09-16T08:57:26.576 INFO:tasks.workunit.client.0.smithi031.stdout:0/771: truncate d6/d12/d13/da5/fc0 5037375 0 2024-09-16T08:57:26.576 INFO:tasks.workunit.client.0.smithi031.stdout:2/703: creat d6/d20/da1/dcf/dda/de3/fe8 x:0 0 0 2024-09-16T08:57:26.580 INFO:tasks.workunit.client.0.smithi031.stdout:7/788: rename d2/d8/d11/d66/d9a/d9c/lbb to d2/d8/d11/d5d/lfd 0 2024-09-16T08:57:26.580 INFO:tasks.workunit.client.0.smithi031.stdout:7/789: creat d2/d8/d11/d81/ffe x:0 0 0 2024-09-16T08:57:26.583 INFO:tasks.workunit.client.0.smithi031.stdout:0/772: mkdir d6/d12/d13/dca/dfc 0 2024-09-16T08:57:26.586 INFO:tasks.workunit.client.0.smithi031.stdout:2/704: rename d6/d2e/d40/f78 to d6/fe9 0 2024-09-16T08:57:26.586 INFO:tasks.workunit.client.0.smithi031.stdout:9/748: sync 2024-09-16T08:57:26.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/680: sync 2024-09-16T08:57:26.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/681: write d2/f14 [2156841,53958] 0 2024-09-16T08:57:26.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/790: dread d2/f18 [0,4194304] 0 2024-09-16T08:57:26.592 INFO:tasks.workunit.client.0.smithi031.stdout:7/791: dread - d2/d8/d11/d66/d9a/d9c/db3/fb0 zero size 2024-09-16T08:57:26.600 INFO:tasks.workunit.client.0.smithi031.stdout:2/705: rename d6/d20/da1/fb0 to d6/d20/da1/dcf/dda/fea 0 2024-09-16T08:57:26.602 INFO:tasks.workunit.client.0.smithi031.stdout:9/749: rmdir d3/d5/da/d34/d3c/d5e 39 2024-09-16T08:57:26.603 INFO:tasks.workunit.client.0.smithi031.stdout:0/773: dwrite d6/f6e [0,4194304] 0 2024-09-16T08:57:26.603 INFO:tasks.workunit.client.0.smithi031.stdout:0/774: chown d6/d12/d13/d3c 410705356 1 2024-09-16T08:57:26.603 INFO:tasks.workunit.client.0.smithi031.stdout:7/792: write d2/d8/d86/da1/da9/fad [4014443,116403] 0 2024-09-16T08:57:26.612 INFO:tasks.workunit.client.0.smithi031.stdout:2/706: unlink d6/fa3 0 2024-09-16T08:57:26.612 INFO:tasks.workunit.client.0.smithi031.stdout:2/707: truncate d6/da/d5e/d8d/f9e 4389345 0 2024-09-16T08:57:26.615 INFO:tasks.workunit.client.0.smithi031.stdout:7/793: getdents d2/d2d/d68/da2 0 2024-09-16T08:57:26.616 INFO:tasks.workunit.client.0.smithi031.stdout:2/708: mkdir d6/da/d37/deb 0 2024-09-16T08:57:26.616 INFO:tasks.workunit.client.0.smithi031.stdout:9/750: dread d3/d5/da/f14 [0,4194304] 0 2024-09-16T08:57:26.616 INFO:tasks.workunit.client.0.smithi031.stdout:9/751: creat d3/d5/da/db/d21/d3b/dad/dd3/f10c x:0 0 0 2024-09-16T08:57:26.618 INFO:tasks.workunit.client.0.smithi031.stdout:7/794: creat d2/d8/d86/da1/fff x:0 0 0 2024-09-16T08:57:26.619 INFO:tasks.workunit.client.0.smithi031.stdout:9/752: creat d3/d5/da/d34/d4d/df1/f10d x:0 0 0 2024-09-16T08:57:26.623 INFO:tasks.workunit.client.0.smithi031.stdout:9/753: truncate d3/d5/da/d34/d3c/d5e/d9c/f9f 3954767 0 2024-09-16T08:57:26.623 INFO:tasks.workunit.client.0.smithi031.stdout:3/789: dwrite d3/d14/d37/d1a/d1d/d3b/d60/f7e [0,4194304] 0 2024-09-16T08:57:26.623 INFO:tasks.workunit.client.0.smithi031.stdout:1/682: dwrite d2/d15/d1f/de9/d52/f74 [0,4194304] 0 2024-09-16T08:57:26.623 INFO:tasks.workunit.client.0.smithi031.stdout:1/683: chown d2/d15/d1f/l99 2081459779 1 2024-09-16T08:57:26.623 INFO:tasks.workunit.client.0.smithi031.stdout:1/684: truncate d2/d15/d1f/de9/d52/fca 137295 0 2024-09-16T08:57:26.627 INFO:tasks.workunit.client.0.smithi031.stdout:9/754: rename d3/d5/da/db/d21/d3b/d8b to d3/d5/d6a/d74/da8/dc3/dbc/dd7/d10e 0 2024-09-16T08:57:26.627 INFO:tasks.workunit.client.0.smithi031.stdout:9/755: write d3/d5/da/d34/fbe [1647888,7330] 0 2024-09-16T08:57:26.629 INFO:tasks.workunit.client.0.smithi031.stdout:3/790: dread d3/d14/d37/d1a/d1d/d9a/dde/f6b [0,4194304] 0 2024-09-16T08:57:26.629 INFO:tasks.workunit.client.0.smithi031.stdout:1/685: rename d2/d15/d5d/l68 to d2/d15/d1f/de9/d1b/d5c/d6b/dc1/dd1/lf1 0 2024-09-16T08:57:26.631 INFO:tasks.workunit.client.0.smithi031.stdout:9/756: link d3/d5/da/d34/d3c/d5e/d9c/d95/c4c d3/d5/d6a/d74/da8/dc3/c10f 0 2024-09-16T08:57:26.632 INFO:tasks.workunit.client.0.smithi031.stdout:3/791: symlink d3/d14/d37/d1a/d1d/d3b/d94/le3 0 2024-09-16T08:57:26.634 INFO:tasks.workunit.client.0.smithi031.stdout:1/686: creat d2/d15/d1f/de9/d1b/d5c/ff2 x:0 0 0 2024-09-16T08:57:26.635 INFO:tasks.workunit.client.0.smithi031.stdout:9/757: symlink d3/l110 0 2024-09-16T08:57:26.636 INFO:tasks.workunit.client.0.smithi031.stdout:9/758: mknod d3/d5/da/d34/d3c/d5e/d62/d65/c111 0 2024-09-16T08:57:26.636 INFO:tasks.workunit.client.0.smithi031.stdout:9/759: dread - d3/d5/da/d34/d3c/d5e/d9c/d95/f10b zero size 2024-09-16T08:57:26.637 INFO:tasks.workunit.client.0.smithi031.stdout:9/760: chown d3/d48/fc8 54 1 2024-09-16T08:57:26.637 INFO:tasks.workunit.client.0.smithi031.stdout:9/761: link d3/d5/da/db/d21/fc1 d3/d5/d6a/d74/da8/d100/f112 0 2024-09-16T08:57:26.638 INFO:tasks.workunit.client.0.smithi031.stdout:9/762: readlink d3/l4a 0 2024-09-16T08:57:26.638 INFO:tasks.workunit.client.0.smithi031.stdout:9/763: read d3/d85/fe6 [953153,114604] 0 2024-09-16T08:57:26.639 INFO:tasks.workunit.client.0.smithi031.stdout:9/764: mkdir d3/d5/da/db/d113 0 2024-09-16T08:57:26.639 INFO:tasks.workunit.client.0.smithi031.stdout:9/765: read - d3/d5/da/d34/d4d/f105 zero size 2024-09-16T08:57:26.639 INFO:tasks.workunit.client.0.smithi031.stdout:9/766: read - d3/d5/da/db/d21/d3b/dad/dd3/f10c zero size 2024-09-16T08:57:26.640 INFO:tasks.workunit.client.0.smithi031.stdout:9/767: truncate d3/d48/f49 180341 0 2024-09-16T08:57:26.641 INFO:tasks.workunit.client.0.smithi031.stdout:0/775: dwrite d6/f74 [0,4194304] 0 2024-09-16T08:57:26.641 INFO:tasks.workunit.client.0.smithi031.stdout:0/776: write d6/d12/d13/d36/fd1 [335246,90083] 0 2024-09-16T08:57:26.642 INFO:tasks.workunit.client.0.smithi031.stdout:9/768: mknod d3/d5/da/d34/d3c/d5e/d62/d65/c114 0 2024-09-16T08:57:26.649 INFO:tasks.workunit.client.0.smithi031.stdout:9/769: dread d3/d5/da/d34/d3c/d5e/fb1 [0,4194304] 0 2024-09-16T08:57:26.649 INFO:tasks.workunit.client.0.smithi031.stdout:9/770: chown d3/d5/da/f2b 8377 1 2024-09-16T08:57:26.657 INFO:tasks.workunit.client.0.smithi031.stdout:2/709: dwrite d6/d2e/d40/d6c/f5a [0,4194304] 0 2024-09-16T08:57:26.657 INFO:tasks.workunit.client.0.smithi031.stdout:2/710: creat d6/d20/da1/dcf/fec x:0 0 0 2024-09-16T08:57:26.661 INFO:tasks.workunit.client.0.smithi031.stdout:2/711: symlink d6/d20/d51/led 0 2024-09-16T08:57:26.661 INFO:tasks.workunit.client.0.smithi031.stdout:2/712: write d6/da/d4b/d4c/d68/fa8 [605952,39144] 0 2024-09-16T08:57:26.661 INFO:tasks.workunit.client.0.smithi031.stdout:7/795: dwrite d2/d8/d86/ff9 [0,4194304] 0 2024-09-16T08:57:26.662 INFO:tasks.workunit.client.0.smithi031.stdout:0/777: dwrite d6/f1a [0,4194304] 0 2024-09-16T08:57:26.662 INFO:tasks.workunit.client.0.smithi031.stdout:3/792: dwrite d3/d14/d37/d1a/d24/f40 [0,4194304] 0 2024-09-16T08:57:26.664 INFO:tasks.workunit.client.0.smithi031.stdout:7/796: truncate d2/d8/f3f 2032011 0 2024-09-16T08:57:26.664 INFO:tasks.workunit.client.0.smithi031.stdout:7/797: write d2/d8/d11/d5d/d7e/faf [782774,129675] 0 2024-09-16T08:57:26.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/798: write d2/d8/d69/d82/f96 [79678,73140] 0 2024-09-16T08:57:26.666 INFO:tasks.workunit.client.0.smithi031.stdout:7/799: readlink d2/d8/d11/l16 0 2024-09-16T08:57:26.666 INFO:tasks.workunit.client.0.smithi031.stdout:0/778: creat d6/d12/d13/dca/ffd x:0 0 0 2024-09-16T08:57:26.673 INFO:tasks.workunit.client.0.smithi031.stdout:2/713: dread f4 [4194304,4194304] 0 2024-09-16T08:57:26.673 INFO:tasks.workunit.client.0.smithi031.stdout:2/714: creat d6/d20/da1/dcf/dda/fee x:0 0 0 2024-09-16T08:57:26.674 INFO:tasks.workunit.client.0.smithi031.stdout:3/793: creat d3/d14/d85/fe4 x:0 0 0 2024-09-16T08:57:26.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/715: dread d6/d20/f4f [4194304,4194304] 0 2024-09-16T08:57:26.678 INFO:tasks.workunit.client.0.smithi031.stdout:7/800: rmdir d2/d8/d86 39 2024-09-16T08:57:26.678 INFO:tasks.workunit.client.0.smithi031.stdout:9/771: getdents d3 0 2024-09-16T08:57:26.680 INFO:tasks.workunit.client.0.smithi031.stdout:0/779: symlink d6/d12/d13/d36/lfe 0 2024-09-16T08:57:26.680 INFO:tasks.workunit.client.0.smithi031.stdout:0/780: write d6/d12/d47/d5b/d9b/fec [426814,80716] 0 2024-09-16T08:57:26.680 INFO:tasks.workunit.client.0.smithi031.stdout:0/781: chown d6/d12/d47/d5b/l5c 979343354 1 2024-09-16T08:57:26.686 INFO:tasks.workunit.client.0.smithi031.stdout:2/716: link d6/d2e/d40/d6c/d2c/d53/d60/f93 d6/d20/d51/ddc/fef 0 2024-09-16T08:57:26.687 INFO:tasks.workunit.client.0.smithi031.stdout:9/772: stat d3/d5/d6a/d74/da8/dc3/c10f 0 2024-09-16T08:57:26.691 INFO:tasks.workunit.client.0.smithi031.stdout:2/717: link d6/da/d37/f63 d6/d2e/d40/d58/dae/ff0 0 2024-09-16T08:57:26.692 INFO:tasks.workunit.client.0.smithi031.stdout:7/801: symlink d2/l100 0 2024-09-16T08:57:26.694 INFO:tasks.workunit.client.0.smithi031.stdout:2/718: dread d6/d2e/f9a [0,4194304] 0 2024-09-16T08:57:26.697 INFO:tasks.workunit.client.0.smithi031.stdout:2/719: unlink d6/da/d5e/d8d/la7 0 2024-09-16T08:57:26.697 INFO:tasks.workunit.client.0.smithi031.stdout:2/720: chown d6/d2e/d40/d6c/d2c/l65 18 1 2024-09-16T08:57:26.704 INFO:tasks.workunit.client.0.smithi031.stdout:0/782: rmdir d6/d12/d13/dca 39 2024-09-16T08:57:26.704 INFO:tasks.workunit.client.0.smithi031.stdout:0/783: read - d6/d12/d13/d36/f8a zero size 2024-09-16T08:57:26.705 INFO:tasks.workunit.client.0.smithi031.stdout:0/784: truncate d6/d12/d13/f41 1402659 0 2024-09-16T08:57:26.707 INFO:tasks.workunit.client.0.smithi031.stdout:7/802: dread d2/fee [0,4194304] 0 2024-09-16T08:57:26.707 INFO:tasks.workunit.client.0.smithi031.stdout:3/794: truncate d3/d14/d37/d1a/d24/f40 1167114 0 2024-09-16T08:57:26.711 INFO:tasks.workunit.client.0.smithi031.stdout:7/803: symlink d2/d8/d11/d66/d9a/d9c/db3/l101 0 2024-09-16T08:57:26.711 INFO:tasks.workunit.client.0.smithi031.stdout:7/804: fsync d2/d8/fd 0 2024-09-16T08:57:26.712 INFO:tasks.workunit.client.0.smithi031.stdout:9/773: dwrite d3/d5/da/d34/d4d/f105 [0,4194304] 0 2024-09-16T08:57:26.716 INFO:tasks.workunit.client.0.smithi031.stdout:0/785: dread d6/f8 [0,4194304] 0 2024-09-16T08:57:26.718 INFO:tasks.workunit.client.0.smithi031.stdout:2/721: dread d6/f13 [0,4194304] 0 2024-09-16T08:57:26.720 INFO:tasks.workunit.client.0.smithi031.stdout:9/774: mkdir d3/d5/da/d34/d3c/d5e/de1/d115 0 2024-09-16T08:57:26.721 INFO:tasks.workunit.client.0.smithi031.stdout:7/805: mknod d2/c102 0 2024-09-16T08:57:26.731 INFO:tasks.workunit.client.0.smithi031.stdout:3/795: dwrite d3/d14/d37/d1a/d1d/d9a/dde/f6b [0,4194304] 0 2024-09-16T08:57:26.731 INFO:tasks.workunit.client.0.smithi031.stdout:3/796: readlink d3/le 0 2024-09-16T08:57:26.732 INFO:tasks.workunit.client.0.smithi031.stdout:0/786: fsync d6/f34 0 2024-09-16T08:57:26.735 INFO:tasks.workunit.client.0.smithi031.stdout:2/722: mknod d6/d2e/d40/d58/dae/cf1 0 2024-09-16T08:57:26.737 INFO:tasks.workunit.client.0.smithi031.stdout:7/806: creat d2/d8/d11/d66/f103 x:0 0 0 2024-09-16T08:57:26.738 INFO:tasks.workunit.client.0.smithi031.stdout:9/775: dwrite d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:26.744 INFO:tasks.workunit.client.0.smithi031.stdout:3/797: creat d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/fe5 x:0 0 0 2024-09-16T08:57:26.745 INFO:tasks.workunit.client.0.smithi031.stdout:0/787: symlink d6/d12/d47/dc5/de6/lff 0 2024-09-16T08:57:26.746 INFO:tasks.workunit.client.0.smithi031.stdout:2/723: rename d6/d2e/d40/d6c/f3b to d6/da/d4b/ff2 0 2024-09-16T08:57:26.747 INFO:tasks.workunit.client.0.smithi031.stdout:7/807: link d2/d8/d86/dd4/cf2 d2/d8/d86/da1/da9/c104 0 2024-09-16T08:57:26.747 INFO:tasks.workunit.client.0.smithi031.stdout:7/808: read d2/d8/d11/d5d/d7e/faf [825277,77430] 0 2024-09-16T08:57:26.749 INFO:tasks.workunit.client.0.smithi031.stdout:9/776: mkdir d3/d5/dd6/d116 0 2024-09-16T08:57:26.749 INFO:tasks.workunit.client.0.smithi031.stdout:9/777: write d3/d5/da/db/d21/fa3 [886708,87856] 0 2024-09-16T08:57:26.750 INFO:tasks.workunit.client.0.smithi031.stdout:0/788: symlink d6/d12/d47/dc5/l100 0 2024-09-16T08:57:26.750 INFO:tasks.workunit.client.0.smithi031.stdout:0/789: stat d6/d12/d13/d36/c8c 0 2024-09-16T08:57:26.753 INFO:tasks.workunit.client.0.smithi031.stdout:7/809: mknod d2/d8/d11/d5d/d7e/db6/c105 0 2024-09-16T08:57:26.756 INFO:tasks.workunit.client.0.smithi031.stdout:9/778: creat d3/d5/db3/f117 x:0 0 0 2024-09-16T08:57:26.756 INFO:tasks.workunit.client.0.smithi031.stdout:9/779: fdatasync d3/d5/d73/f9b 0 2024-09-16T08:57:26.756 INFO:tasks.workunit.client.0.smithi031.stdout:9/780: readlink d3/l4a 0 2024-09-16T08:57:26.756 INFO:tasks.workunit.client.0.smithi031.stdout:9/781: read f2 [3661659,90318] 0 2024-09-16T08:57:26.758 INFO:tasks.workunit.client.0.smithi031.stdout:2/724: creat d6/d2e/d40/d6c/d2c/d53/d60/d91/ff3 x:0 0 0 2024-09-16T08:57:26.760 INFO:tasks.workunit.client.0.smithi031.stdout:0/790: write d6/f1a [1688238,114561] 0 2024-09-16T08:57:26.760 INFO:tasks.workunit.client.0.smithi031.stdout:0/791: readlink d6/dac/ld2 0 2024-09-16T08:57:26.760 INFO:tasks.workunit.client.0.smithi031.stdout:0/792: fsync d6/d12/d13/da5/fc0 0 2024-09-16T08:57:26.760 INFO:tasks.workunit.client.0.smithi031.stdout:0/793: dread - d6/d12/d13/d36/fa6 zero size 2024-09-16T08:57:26.760 INFO:tasks.workunit.client.0.smithi031.stdout:0/794: readlink d6/d12/d47/d5b/l63 0 2024-09-16T08:57:26.760 INFO:tasks.workunit.client.0.smithi031.stdout:0/795: write d6/d12/d13/d3c/fdb [508792,11618] 0 2024-09-16T08:57:26.761 INFO:tasks.workunit.client.0.smithi031.stdout:7/810: rename d2/f18 to d2/d8/d11/d5d/f106 0 2024-09-16T08:57:26.765 INFO:tasks.workunit.client.0.smithi031.stdout:0/796: link d6/d12/d47/ce7 d6/d12/d47/d5b/c101 0 2024-09-16T08:57:26.767 INFO:tasks.workunit.client.0.smithi031.stdout:2/725: rename d6/da/fd to d6/da/d4b/d4c/d68/ff4 0 2024-09-16T08:57:26.767 INFO:tasks.workunit.client.0.smithi031.stdout:2/726: creat d6/ff5 x:0 0 0 2024-09-16T08:57:26.767 INFO:tasks.workunit.client.0.smithi031.stdout:2/727: readlink d6/l59 0 2024-09-16T08:57:26.767 INFO:tasks.workunit.client.0.smithi031.stdout:2/728: write d6/d20/da1/dcf/dda/fee [1001632,75567] 0 2024-09-16T08:57:26.768 INFO:tasks.workunit.client.0.smithi031.stdout:2/729: write d6/d20/f32 [395331,8910] 0 2024-09-16T08:57:26.768 INFO:tasks.workunit.client.0.smithi031.stdout:2/730: chown d6/d20/da1/dcf/dda/de3 0 1 2024-09-16T08:57:26.768 INFO:tasks.workunit.client.0.smithi031.stdout:2/731: creat d6/d2e/d40/d6c/d2c/d53/d60/dbd/ff6 x:0 0 0 2024-09-16T08:57:26.771 INFO:tasks.workunit.client.0.smithi031.stdout:7/811: write d2/d8/f50 [1276667,81143] 0 2024-09-16T08:57:26.772 INFO:tasks.workunit.client.0.smithi031.stdout:0/797: read d6/d61/d91/fc3 [1425353,122431] 0 2024-09-16T08:57:26.774 INFO:tasks.workunit.client.0.smithi031.stdout:2/732: write d6/da/d37/d42/f71 [1483828,7388] 0 2024-09-16T08:57:26.774 INFO:tasks.workunit.client.0.smithi031.stdout:2/733: chown d6/d2e/d40/d6c/d2c/d53/d60/f93 216 1 2024-09-16T08:57:26.774 INFO:tasks.workunit.client.0.smithi031.stdout:2/734: chown f1 33059680 1 2024-09-16T08:57:26.774 INFO:tasks.workunit.client.0.smithi031.stdout:0/798: creat d6/d61/dce/f102 x:0 0 0 2024-09-16T08:57:26.779 INFO:tasks.workunit.client.0.smithi031.stdout:9/782: dwrite d3/d5/da/d34/d3c/d5e/d9c/d95/fd2 [4194304,4194304] 0 2024-09-16T08:57:26.786 INFO:tasks.workunit.client.0.smithi031.stdout:7/812: rmdir d2/d2d 39 2024-09-16T08:57:26.788 INFO:tasks.workunit.client.0.smithi031.stdout:0/799: unlink d6/d12/d47/c9d 0 2024-09-16T08:57:26.790 INFO:tasks.workunit.client.0.smithi031.stdout:9/783: mknod d3/d5/da/d34/d3c/d5e/d9c/c118 0 2024-09-16T08:57:26.791 INFO:tasks.workunit.client.0.smithi031.stdout:2/735: dwrite d6/d2e/d40/d6c/d2c/d53/d60/d91/ff3 [0,4194304] 0 2024-09-16T08:57:26.792 INFO:tasks.workunit.client.0.smithi031.stdout:7/813: mknod d2/d8/d11/d5d/c107 0 2024-09-16T08:57:26.795 INFO:tasks.workunit.client.0.smithi031.stdout:0/800: link d6/d12/d47/d5b/l87 d6/dd3/l103 0 2024-09-16T08:57:26.795 INFO:tasks.workunit.client.0.smithi031.stdout:0/801: write d6/d12/d47/d4d/fa1 [909577,14108] 0 2024-09-16T08:57:26.795 INFO:tasks.workunit.client.0.smithi031.stdout:0/802: write d6/d12/d47/f7d [1349981,30350] 0 2024-09-16T08:57:26.797 INFO:tasks.workunit.client.0.smithi031.stdout:7/814: dread d2/d8/d11/d66/d9a/fc9 [0,4194304] 0 2024-09-16T08:57:26.797 INFO:tasks.workunit.client.0.smithi031.stdout:7/815: creat d2/d8/d86/da1/da9/f108 x:0 0 0 2024-09-16T08:57:26.797 INFO:tasks.workunit.client.0.smithi031.stdout:7/816: creat d2/d8/d11/d5d/f109 x:0 0 0 2024-09-16T08:57:26.798 INFO:tasks.workunit.client.0.smithi031.stdout:9/784: symlink d3/d5/da/d34/l119 0 2024-09-16T08:57:26.803 INFO:tasks.workunit.client.0.smithi031.stdout:9/785: stat d3/d5/da/d34/d3c/d5e/d9c/cd8 0 2024-09-16T08:57:26.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/803: dread d6/f6e [0,4194304] 0 2024-09-16T08:57:26.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/804: write d6/d12/d47/d5b/d9b/fa8 [898577,125407] 0 2024-09-16T08:57:26.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/805: creat d6/dac/f104 x:0 0 0 2024-09-16T08:57:26.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/806: readlink d6/d61/lf0 0 2024-09-16T08:57:26.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/807: truncate d6/d12/d13/f46 1319384 0 2024-09-16T08:57:26.805 INFO:tasks.workunit.client.0.smithi031.stdout:7/817: rmdir d2/d8/d69 39 2024-09-16T08:57:26.807 INFO:tasks.workunit.client.0.smithi031.stdout:0/808: truncate d6/f1a 3753225 0 2024-09-16T08:57:26.808 INFO:tasks.workunit.client.0.smithi031.stdout:7/818: mknod d2/d8/c10a 0 2024-09-16T08:57:26.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/736: dwrite d6/da/d4b/fa4 [0,4194304] 0 2024-09-16T08:57:26.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/737: write d6/da/d4b/d4c/d73/fb5 [148419,88484] 0 2024-09-16T08:57:26.809 INFO:tasks.workunit.client.0.smithi031.stdout:2/738: write d6/da/d4b/f83 [90644,58534] 0 2024-09-16T08:57:26.809 INFO:tasks.workunit.client.0.smithi031.stdout:0/809: mknod d6/d12/d47/d5b/c105 0 2024-09-16T08:57:26.809 INFO:tasks.workunit.client.0.smithi031.stdout:0/810: fsync d6/f3b 0 2024-09-16T08:57:26.810 INFO:tasks.workunit.client.0.smithi031.stdout:7/819: mkdir d2/d8/d11/d81/d10b 0 2024-09-16T08:57:26.811 INFO:tasks.workunit.client.0.smithi031.stdout:2/739: link d6/d20/f89 d6/da/d5e/d72/ff7 0 2024-09-16T08:57:26.812 INFO:tasks.workunit.client.0.smithi031.stdout:2/740: dread - d6/ff5 zero size 2024-09-16T08:57:26.812 INFO:tasks.workunit.client.0.smithi031.stdout:2/741: fdatasync d6/d20/d51/ddc/fef 0 2024-09-16T08:57:26.812 INFO:tasks.workunit.client.0.smithi031.stdout:2/742: fdatasync d6/d20/f89 0 2024-09-16T08:57:26.815 INFO:tasks.workunit.client.0.smithi031.stdout:2/743: mknod d6/d20/da1/cf8 0 2024-09-16T08:57:26.818 INFO:tasks.workunit.client.0.smithi031.stdout:9/786: read d3/d48/f9d [3627384,19850] 0 2024-09-16T08:57:26.825 INFO:tasks.workunit.client.0.smithi031.stdout:0/811: dread d6/f3b [0,4194304] 0 2024-09-16T08:57:26.825 INFO:tasks.workunit.client.0.smithi031.stdout:7/820: rmdir d2/d8/d11/d81 39 2024-09-16T08:57:26.825 INFO:tasks.workunit.client.0.smithi031.stdout:2/744: dread d6/f2d [0,4194304] 0 2024-09-16T08:57:26.829 INFO:tasks.workunit.client.0.smithi031.stdout:9/787: rmdir d3/d5/da/d34/d3c 39 2024-09-16T08:57:26.834 INFO:tasks.workunit.client.0.smithi031.stdout:2/745: truncate d6/d20/f4f 339944 0 2024-09-16T08:57:26.838 INFO:tasks.workunit.client.0.smithi031.stdout:9/788: creat d3/d5/da/d34/d3c/d5e/d9c/f11a x:0 0 0 2024-09-16T08:57:26.838 INFO:tasks.workunit.client.0.smithi031.stdout:9/789: chown d3/d5/d73/d81/c8c 16624473 1 2024-09-16T08:57:26.843 INFO:tasks.workunit.client.0.smithi031.stdout:7/821: link d2/c47 d2/d8/d86/da1/c10c 0 2024-09-16T08:57:26.843 INFO:tasks.workunit.client.0.smithi031.stdout:7/822: dread - d2/d8/d69/d82/fd0 zero size 2024-09-16T08:57:26.848 INFO:tasks.workunit.client.0.smithi031.stdout:7/823: unlink d2/ffc 0 2024-09-16T08:57:26.849 INFO:tasks.workunit.client.0.smithi031.stdout:9/790: read d3/d5/da/db/fe [2026570,113909] 0 2024-09-16T08:57:26.856 INFO:tasks.workunit.client.0.smithi031.stdout:9/791: symlink d3/d5/da/db/l11b 0 2024-09-16T08:57:26.856 INFO:tasks.workunit.client.0.smithi031.stdout:9/792: write d3/d5/d73/f86 [404385,7146] 0 2024-09-16T08:57:26.860 INFO:tasks.workunit.client.0.smithi031.stdout:2/746: rename d6/d2e/d40/d6c/d2c/d53/d60/dbd to d6/da/d4b/d4c/df9 0 2024-09-16T08:57:26.864 INFO:tasks.workunit.client.0.smithi031.stdout:7/824: write d2/d2d/f43 [3062304,38457] 0 2024-09-16T08:57:26.865 INFO:tasks.workunit.client.0.smithi031.stdout:2/747: creat d6/d2e/ffa x:0 0 0 2024-09-16T08:57:26.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/748: rename d6/d20/da1/dcf/fd0 to d6/da/d5e/d8d/dcc/ffb 0 2024-09-16T08:57:26.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/749: stat d6/da/d5e/d8d/f9e 0 2024-09-16T08:57:26.867 INFO:tasks.workunit.client.0.smithi031.stdout:2/750: fdatasync d6/da/d4b/da0/fde 0 2024-09-16T08:57:26.869 INFO:tasks.workunit.client.0.smithi031.stdout:2/751: readlink d6/d2e/d40/d6c/d2c/l4e 0 2024-09-16T08:57:26.871 INFO:tasks.workunit.client.0.smithi031.stdout:2/752: creat d6/da/d37/d8b/dab/ffc x:0 0 0 2024-09-16T08:57:26.871 INFO:tasks.workunit.client.0.smithi031.stdout:2/753: read - d6/d20/fac zero size 2024-09-16T08:57:26.872 INFO:tasks.workunit.client.0.smithi031.stdout:9/793: dwrite d3/d5/da/db/d21/fc1 [0,4194304] 0 2024-09-16T08:57:26.875 INFO:tasks.workunit.client.0.smithi031.stdout:7/825: write d2/d8/f8c [1438476,128957] 0 2024-09-16T08:57:26.879 INFO:tasks.workunit.client.0.smithi031.stdout:9/794: rmdir d3/d5/d6a/d74/da8 39 2024-09-16T08:57:26.879 INFO:tasks.workunit.client.0.smithi031.stdout:9/795: readlink d3/l4a 0 2024-09-16T08:57:26.879 INFO:tasks.workunit.client.0.smithi031.stdout:9/796: readlink d3/d5/d6a/d74/da8/dc3/dbc/lde 0 2024-09-16T08:57:26.879 INFO:tasks.workunit.client.0.smithi031.stdout:9/797: truncate d3/d5/da/f6b 5443647 0 2024-09-16T08:57:26.880 INFO:tasks.workunit.client.0.smithi031.stdout:9/798: write d3/d5/d73/f9b [754880,64972] 0 2024-09-16T08:57:26.882 INFO:tasks.workunit.client.0.smithi031.stdout:9/799: write d3/d5/da/d34/f99 [718648,46758] 0 2024-09-16T08:57:26.882 INFO:tasks.workunit.client.0.smithi031.stdout:9/800: chown d3/d5/d6a/d74/dd0/l108 95974 1 2024-09-16T08:57:26.885 INFO:tasks.workunit.client.0.smithi031.stdout:7/826: read d2/d8/d11/d5d/fc2 [3202336,33822] 0 2024-09-16T08:57:26.886 INFO:tasks.workunit.client.0.smithi031.stdout:9/801: mkdir d3/d5/da/db/dcb/d11c 0 2024-09-16T08:57:26.888 INFO:tasks.workunit.client.0.smithi031.stdout:7/827: mkdir d2/d8/d11/d66/d9a/d9c/db3/d10d 0 2024-09-16T08:57:26.888 INFO:tasks.workunit.client.0.smithi031.stdout:7/828: chown d2/d8/d69/d82/cd9 113 1 2024-09-16T08:57:26.889 INFO:tasks.workunit.client.0.smithi031.stdout:9/802: unlink d3/d5/da/d34/d3c/d5e/d9c/d95/dcf/ff2 0 2024-09-16T08:57:26.890 INFO:tasks.workunit.client.0.smithi031.stdout:2/754: dwrite d6/da/d4b/d4c/df9/fe1 [0,4194304] 0 2024-09-16T08:57:26.890 INFO:tasks.workunit.client.0.smithi031.stdout:7/829: rename d2/d8/f50 to d2/d8/d11/d66/f10e 0 2024-09-16T08:57:26.891 INFO:tasks.workunit.client.0.smithi031.stdout:9/803: getdents d3/d5/da/d34/d3c/d5e 0 2024-09-16T08:57:26.892 INFO:tasks.workunit.client.0.smithi031.stdout:2/755: dread d6/d20/f32 [0,4194304] 0 2024-09-16T08:57:26.893 INFO:tasks.workunit.client.0.smithi031.stdout:7/830: creat d2/d8/d69/d82/f10f x:0 0 0 2024-09-16T08:57:26.893 INFO:tasks.workunit.client.0.smithi031.stdout:7/831: stat d2/d8/d86/dd4/fdd 0 2024-09-16T08:57:26.893 INFO:tasks.workunit.client.0.smithi031.stdout:7/832: chown d2/c47 60198 1 2024-09-16T08:57:26.894 INFO:tasks.workunit.client.0.smithi031.stdout:2/756: creat d6/da/d5e/d8d/dcc/ffd x:0 0 0 2024-09-16T08:57:26.894 INFO:tasks.workunit.client.0.smithi031.stdout:2/757: chown d6/da/d4b/d4c/d73/f9b 718350 1 2024-09-16T08:57:26.895 INFO:tasks.workunit.client.0.smithi031.stdout:7/833: mkdir d2/d8/d11/d81/d110 0 2024-09-16T08:57:26.895 INFO:tasks.workunit.client.0.smithi031.stdout:7/834: truncate d2/d2d/d6d/ff5 1005886 0 2024-09-16T08:57:26.897 INFO:tasks.workunit.client.0.smithi031.stdout:2/758: readlink d6/d2e/d40/d6c/d2c/l3c 0 2024-09-16T08:57:26.897 INFO:tasks.workunit.client.0.smithi031.stdout:2/759: stat d6/d2e/f9a 0 2024-09-16T08:57:26.897 INFO:tasks.workunit.client.0.smithi031.stdout:2/760: dread - d6/d20/d51/f62 zero size 2024-09-16T08:57:26.898 INFO:tasks.workunit.client.0.smithi031.stdout:7/835: creat d2/d8/d11/d5d/d6e/f111 x:0 0 0 2024-09-16T08:57:26.901 INFO:tasks.workunit.client.0.smithi031.stdout:2/761: truncate f4 1085279 0 2024-09-16T08:57:26.903 INFO:tasks.workunit.client.0.smithi031.stdout:2/762: mknod d6/da/d37/deb/cfe 0 2024-09-16T08:57:26.906 INFO:tasks.workunit.client.0.smithi031.stdout:2/763: creat d6/da/d4b/d4c/df9/fff x:0 0 0 2024-09-16T08:57:26.906 INFO:tasks.workunit.client.0.smithi031.stdout:2/764: dread - d6/da/d4b/fc8 zero size 2024-09-16T08:57:26.906 INFO:tasks.workunit.client.0.smithi031.stdout:2/765: chown d6/d2e/d40/dc3 3 1 2024-09-16T08:57:26.907 INFO:tasks.workunit.client.0.smithi031.stdout:2/766: chown d6/da/d37/d42/l50 1 1 2024-09-16T08:57:26.909 INFO:tasks.workunit.client.0.smithi031.stdout:2/767: link d6/c1f d6/d2e/d40/c100 0 2024-09-16T08:57:26.911 INFO:tasks.workunit.client.0.smithi031.stdout:2/768: link d6/c10 d6/d2e/d40/c101 0 2024-09-16T08:57:26.912 INFO:tasks.workunit.client.0.smithi031.stdout:2/769: mknod d6/da/d5e/dd1/c102 0 2024-09-16T08:57:26.912 INFO:tasks.workunit.client.0.smithi031.stdout:9/804: dwrite d3/d5/da/db/d21/d3b/dad/dea/fac [0,4194304] 0 2024-09-16T08:57:26.913 INFO:tasks.workunit.client.0.smithi031.stdout:2/770: rmdir d6/d2e/d40/d58/dae/db1 39 2024-09-16T08:57:26.913 INFO:tasks.workunit.client.0.smithi031.stdout:2/771: chown d6/d2e/d40/d6c/d2c/d53/d60/f39 683 1 2024-09-16T08:57:26.913 INFO:tasks.workunit.client.0.smithi031.stdout:2/772: chown d6/f16 4585 1 2024-09-16T08:57:26.914 INFO:tasks.workunit.client.0.smithi031.stdout:9/805: rename d3/d48/f4e to d3/d5/da/db/d21/d3b/f11d 0 2024-09-16T08:57:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/773: dread d6/da/d4b/f83 [0,4194304] 0 2024-09-16T08:57:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/774: write d6/da/d5e/d8d/dcc/ffd [414803,76249] 0 2024-09-16T08:57:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/775: write d6/d20/da1/dcf/dda/fea [457157,2787] 0 2024-09-16T08:57:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:7/836: dwrite d2/d8/f63 [0,4194304] 0 2024-09-16T08:57:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:9/806: fdatasync d3/d5/d73/f9b 0 2024-09-16T08:57:26.918 INFO:tasks.workunit.client.0.smithi031.stdout:2/776: mknod d6/da/d4b/d4c/d68/c103 0 2024-09-16T08:57:26.919 INFO:tasks.workunit.client.0.smithi031.stdout:2/777: chown d6/d20/da1/le4 4600 1 2024-09-16T08:57:26.919 INFO:tasks.workunit.client.0.smithi031.stdout:2/778: chown d6/da/d5e/d8d/cc2 1862131 1 2024-09-16T08:57:26.920 INFO:tasks.workunit.client.0.smithi031.stdout:9/807: write d3/d5/da/d34/fbe [1147446,4351] 0 2024-09-16T08:57:26.920 INFO:tasks.workunit.client.0.smithi031.stdout:9/808: truncate d3/d5/da/db/d21/d3b/dad/dd3/f10c 712972 0 2024-09-16T08:57:26.925 INFO:tasks.workunit.client.0.smithi031.stdout:9/809: dread d3/d48/f9d [0,4194304] 0 2024-09-16T08:57:26.925 INFO:tasks.workunit.client.0.smithi031.stdout:9/810: fsync d3/d5/da/db/fc0 0 2024-09-16T08:57:26.930 INFO:tasks.workunit.client.0.smithi031.stdout:7/837: creat d2/d2d/d68/f112 x:0 0 0 2024-09-16T08:57:26.933 INFO:tasks.workunit.client.0.smithi031.stdout:9/811: dread d3/d85/fe8 [0,4194304] 0 2024-09-16T08:57:26.934 INFO:tasks.workunit.client.0.smithi031.stdout:7/838: mkdir d2/d2d/d68/d113 0 2024-09-16T08:57:26.943 INFO:tasks.workunit.client.0.smithi031.stdout:9/812: write d3/d5/d6a/d74/dd0/de3/ff7 [2372618,41006] 0 2024-09-16T08:57:26.947 INFO:tasks.workunit.client.0.smithi031.stdout:7/839: write d2/d8/d11/d66/d9a/fca [3222363,29248] 0 2024-09-16T08:57:26.947 INFO:tasks.workunit.client.0.smithi031.stdout:7/840: write d2/d2d/f43 [2607037,18409] 0 2024-09-16T08:57:26.947 INFO:tasks.workunit.client.0.smithi031.stdout:7/841: read - d2/d8/d86/da1/fff zero size 2024-09-16T08:57:26.948 INFO:tasks.workunit.client.0.smithi031.stdout:8/584: sync 2024-09-16T08:57:26.948 INFO:tasks.workunit.client.0.smithi031.stdout:6/800: sync 2024-09-16T08:57:26.948 INFO:tasks.workunit.client.0.smithi031.stdout:5/638: sync 2024-09-16T08:57:26.950 INFO:tasks.workunit.client.0.smithi031.stdout:4/848: sync 2024-09-16T08:57:26.952 INFO:tasks.workunit.client.0.smithi031.stdout:7/842: mkdir d2/d2d/d68/d113/d114 0 2024-09-16T08:57:26.952 INFO:tasks.workunit.client.0.smithi031.stdout:7/843: creat d2/d8/d11/d66/f115 x:0 0 0 2024-09-16T08:57:26.953 INFO:tasks.workunit.client.0.smithi031.stdout:8/585: link d4/d5b/d8d/c94 d4/d2d/cc0 0 2024-09-16T08:57:26.953 INFO:tasks.workunit.client.0.smithi031.stdout:8/586: chown d4/d7/d21/d37/d46/l9d 132069284 1 2024-09-16T08:57:26.953 INFO:tasks.workunit.client.0.smithi031.stdout:8/587: write d4/d77/f81 [1785387,31122] 0 2024-09-16T08:57:26.955 INFO:tasks.workunit.client.0.smithi031.stdout:5/639: rename d2/dc/d1d/f3f to d2/dc/d25/fcf 0 2024-09-16T08:57:26.957 INFO:tasks.workunit.client.0.smithi031.stdout:6/801: dread d0/dd/d74/d67/fbf [0,4194304] 0 2024-09-16T08:57:26.959 INFO:tasks.workunit.client.0.smithi031.stdout:7/844: unlink d2/d2d/d6d/fc6 0 2024-09-16T08:57:26.959 INFO:tasks.workunit.client.0.smithi031.stdout:7/845: creat d2/d8/f116 x:0 0 0 2024-09-16T08:57:26.960 INFO:tasks.workunit.client.0.smithi031.stdout:8/588: link d4/d7/d99/fb6 d4/d7/d99/fc1 0 2024-09-16T08:57:26.965 INFO:tasks.workunit.client.0.smithi031.stdout:5/640: unlink d2/dc/f16 0 2024-09-16T08:57:26.968 INFO:tasks.workunit.client.0.smithi031.stdout:6/802: truncate d0/d10/f3b 1831515 0 2024-09-16T08:57:26.974 INFO:tasks.workunit.client.0.smithi031.stdout:5/641: rmdir d2/dc/d1d/dbb 39 2024-09-16T08:57:26.975 INFO:tasks.workunit.client.0.smithi031.stdout:6/803: link d0/d10/dc8/lf7 d0/d10/dc8/lf8 0 2024-09-16T08:57:26.976 INFO:tasks.workunit.client.0.smithi031.stdout:9/813: dwrite d3/d5/da/db/dcb/fce [0,4194304] 0 2024-09-16T08:57:26.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/589: getdents d4/d7/d21/d37/d46 0 2024-09-16T08:57:26.977 INFO:tasks.workunit.client.0.smithi031.stdout:4/849: dwrite d0/f2e [8388608,4194304] 0 2024-09-16T08:57:26.977 INFO:tasks.workunit.client.0.smithi031.stdout:9/814: dread d3/d5/d6a/d74/da8/dc3/f7b [0,4194304] 0 2024-09-16T08:57:26.979 INFO:tasks.workunit.client.0.smithi031.stdout:6/804: mkdir d0/dd/d74/d19/df9 0 2024-09-16T08:57:26.979 INFO:tasks.workunit.client.0.smithi031.stdout:6/805: write d0/dd/d74/d5e/d7c/d56/fce [30978,44055] 0 2024-09-16T08:57:26.989 INFO:tasks.workunit.client.0.smithi031.stdout:5/642: dwrite d2/dc/d36/f4a [0,4194304] 0 2024-09-16T08:57:26.993 INFO:tasks.workunit.client.0.smithi031.stdout:8/590: rename d4/d77/f8f to d4/d9e/d38/fc2 0 2024-09-16T08:57:26.993 INFO:tasks.workunit.client.0.smithi031.stdout:8/591: chown d4/d9e/d5e 8781 1 2024-09-16T08:57:26.996 INFO:tasks.workunit.client.0.smithi031.stdout:6/806: dwrite d0/dd/d74/d5e/d7c/d3f/f4b [0,4194304] 0 2024-09-16T08:57:26.997 INFO:tasks.workunit.client.0.smithi031.stdout:4/850: creat d0/d3c/d6b/d9c/f110 x:0 0 0 2024-09-16T08:57:27.000 INFO:tasks.workunit.client.0.smithi031.stdout:5/643: stat d2/dc/d3d/lc4 0 2024-09-16T08:57:27.000 INFO:tasks.workunit.client.0.smithi031.stdout:5/644: readlink d2/l80 0 2024-09-16T08:57:27.002 INFO:tasks.workunit.client.0.smithi031.stdout:9/815: rename d3/d85/fe8 to d3/d5/da/d34/f11e 0 2024-09-16T08:57:27.002 INFO:tasks.workunit.client.0.smithi031.stdout:9/816: chown d3/d5/da/db/d21/d3b/dad/dea/cba 19 1 2024-09-16T08:57:27.003 INFO:tasks.workunit.client.0.smithi031.stdout:8/592: truncate d4/d9e/d38/f3c 1348229 0 2024-09-16T08:57:27.006 INFO:tasks.workunit.client.0.smithi031.stdout:6/807: symlink d0/d10/lfa 0 2024-09-16T08:57:27.008 INFO:tasks.workunit.client.0.smithi031.stdout:5/645: symlink d2/dc/d25/dc8/ld0 0 2024-09-16T08:57:27.010 INFO:tasks.workunit.client.0.smithi031.stdout:9/817: dread d3/d5/da/d34/d3c/d5e/d9c/d95/fd2 [4194304,4194304] 0 2024-09-16T08:57:27.011 INFO:tasks.workunit.client.0.smithi031.stdout:6/808: mknod d0/d10/dc8/cfb 0 2024-09-16T08:57:27.012 INFO:tasks.workunit.client.0.smithi031.stdout:5/646: symlink d2/d4b/ld1 0 2024-09-16T08:57:27.012 INFO:tasks.workunit.client.0.smithi031.stdout:5/647: dread - d2/dc/d36/d76/f9a zero size 2024-09-16T08:57:27.014 INFO:tasks.workunit.client.0.smithi031.stdout:4/851: dwrite d0/d102/d8d/fc1 [0,4194304] 0 2024-09-16T08:57:27.014 INFO:tasks.workunit.client.0.smithi031.stdout:4/852: fsync d0/d2/d16/ddd/fdf 0 2024-09-16T08:57:27.022 INFO:tasks.workunit.client.0.smithi031.stdout:9/818: mknod d3/d5/da/db/d21/d3b/dad/dd3/c11f 0 2024-09-16T08:57:27.027 INFO:tasks.workunit.client.0.smithi031.stdout:5/648: truncate d2/dc/d8c/fc9 2389582 0 2024-09-16T08:57:27.035 INFO:tasks.workunit.client.0.smithi031.stdout:6/809: write d0/dd/d35/d6c/d83/f89 [927891,33119] 0 2024-09-16T08:57:27.035 INFO:tasks.workunit.client.0.smithi031.stdout:6/810: fsync d0/dd/d74/d5e/d7c/d3f/f48 0 2024-09-16T08:57:27.040 INFO:tasks.workunit.client.0.smithi031.stdout:4/853: mknod d0/d2/c111 0 2024-09-16T08:57:27.053 INFO:tasks.workunit.client.0.smithi031.stdout:8/593: rename d4/d7/d21 to d4/d2f/dc3 0 2024-09-16T08:57:27.053 INFO:tasks.workunit.client.0.smithi031.stdout:8/594: truncate d4/d77/f9b 834788 0 2024-09-16T08:57:27.054 INFO:tasks.workunit.client.0.smithi031.stdout:4/854: dwrite d0/d102/dd5/ff7 [0,4194304] 0 2024-09-16T08:57:27.054 INFO:tasks.workunit.client.0.smithi031.stdout:4/855: read d0/d102/f63 [3476790,10675] 0 2024-09-16T08:57:27.054 INFO:tasks.workunit.client.0.smithi031.stdout:4/856: truncate d0/d2/d16/d1e/f3b 5189191 0 2024-09-16T08:57:27.054 INFO:tasks.workunit.client.0.smithi031.stdout:4/857: getdents d0/d8e/d75 0 2024-09-16T08:57:27.055 INFO:tasks.workunit.client.0.smithi031.stdout:9/819: mkdir d3/d5/da/d34/d3c/d5e/d9c/d120 0 2024-09-16T08:57:27.059 INFO:tasks.workunit.client.0.smithi031.stdout:5/649: symlink d2/dc/d25/da2/ld2 0 2024-09-16T08:57:27.061 INFO:tasks.workunit.client.0.smithi031.stdout:6/811: mknod d0/dd/df3/cfc 0 2024-09-16T08:57:27.061 INFO:tasks.workunit.client.0.smithi031.stdout:6/812: fsync d0/f7f 0 2024-09-16T08:57:27.061 INFO:tasks.workunit.client.0.smithi031.stdout:6/813: creat d0/ffd x:0 0 0 2024-09-16T08:57:27.061 INFO:tasks.workunit.client.0.smithi031.stdout:6/814: chown d0/dd/d74/d5e/d7c 21685 1 2024-09-16T08:57:27.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/595: creat d4/d2f/dc3/d37/fc4 x:0 0 0 2024-09-16T08:57:27.070 INFO:tasks.workunit.client.0.smithi031.stdout:4/858: rename d0/d109/d106/lf6 to d0/db5/l112 0 2024-09-16T08:57:27.070 INFO:tasks.workunit.client.0.smithi031.stdout:9/820: dwrite d3/d5/da/f89 [0,4194304] 0 2024-09-16T08:57:27.071 INFO:tasks.workunit.client.0.smithi031.stdout:5/650: creat d2/dc/d91/fd3 x:0 0 0 2024-09-16T08:57:27.071 INFO:tasks.workunit.client.0.smithi031.stdout:5/651: chown d2/dc/f43 7238488 1 2024-09-16T08:57:27.074 INFO:tasks.workunit.client.0.smithi031.stdout:6/815: mknod d0/dd/d74/d5e/d7c/d56/cfe 0 2024-09-16T08:57:27.075 INFO:tasks.workunit.client.0.smithi031.stdout:6/816: fsync d0/dd/d74/d19/f34 0 2024-09-16T08:57:27.076 INFO:tasks.workunit.client.0.smithi031.stdout:3/798: sync 2024-09-16T08:57:27.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/812: sync 2024-09-16T08:57:27.076 INFO:tasks.workunit.client.0.smithi031.stdout:1/687: sync 2024-09-16T08:57:27.076 INFO:tasks.workunit.client.0.smithi031.stdout:1/688: fdatasync d2/d15/d5d/f45 0 2024-09-16T08:57:27.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/813: creat d6/d12/d97/d9e/f106 x:0 0 0 2024-09-16T08:57:27.076 INFO:tasks.workunit.client.0.smithi031.stdout:0/814: write d6/d12/d47/d7c/fdd [327802,36840] 0 2024-09-16T08:57:27.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/859: mkdir d0/d2/d113 0 2024-09-16T08:57:27.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/860: chown d0/d102/l6c 163787883 1 2024-09-16T08:57:27.077 INFO:tasks.workunit.client.0.smithi031.stdout:4/861: readlink d0/d8e/lf5 0 2024-09-16T08:57:27.080 INFO:tasks.workunit.client.0.smithi031.stdout:8/596: dwrite d4/d2d/d97/fb2 [0,4194304] 0 2024-09-16T08:57:27.080 INFO:tasks.workunit.client.0.smithi031.stdout:8/597: write d4/f17 [9194859,108028] 0 2024-09-16T08:57:27.081 INFO:tasks.workunit.client.0.smithi031.stdout:9/821: mknod d3/d5/d73/d81/df9/c121 0 2024-09-16T08:57:27.082 INFO:tasks.workunit.client.0.smithi031.stdout:5/652: mknod d2/dc/d25/da2/cd4 0 2024-09-16T08:57:27.084 INFO:tasks.workunit.client.0.smithi031.stdout:6/817: symlink d0/dd/d74/d5e/d7c/d56/dc5/lff 0 2024-09-16T08:57:27.085 INFO:tasks.workunit.client.0.smithi031.stdout:3/799: link d3/d14/d37/d1a/d1d/d3b/d94/la1 d3/d14/d37/d33/d97/le6 0 2024-09-16T08:57:27.085 INFO:tasks.workunit.client.0.smithi031.stdout:3/800: creat d3/d14/d37/d1a/d1d/d3b/d60/fe7 x:0 0 0 2024-09-16T08:57:27.088 INFO:tasks.workunit.client.0.smithi031.stdout:0/815: mknod d6/d12/d97/db3/c107 0 2024-09-16T08:57:27.091 INFO:tasks.workunit.client.0.smithi031.stdout:4/862: link d0/d102/d84/fc4 d0/d102/dd5/dff/d100/f114 0 2024-09-16T08:57:27.094 INFO:tasks.workunit.client.0.smithi031.stdout:8/598: rmdir d4/d9e/d38 39 2024-09-16T08:57:27.094 INFO:tasks.workunit.client.0.smithi031.stdout:8/599: chown d4/d2d/d97/fb2 1105 1 2024-09-16T08:57:27.094 INFO:tasks.workunit.client.0.smithi031.stdout:1/689: dwrite d2/d15/d1f/de9/d1b/f3b [0,4194304] 0 2024-09-16T08:57:27.095 INFO:tasks.workunit.client.0.smithi031.stdout:9/822: read d3/d5/da/db/fc0 [1047427,59819] 0 2024-09-16T08:57:27.095 INFO:tasks.workunit.client.0.smithi031.stdout:5/653: mknod d2/dc/d3d/d4c/cd5 0 2024-09-16T08:57:27.096 INFO:tasks.workunit.client.0.smithi031.stdout:5/654: readlink d2/l80 0 2024-09-16T08:57:27.098 INFO:tasks.workunit.client.0.smithi031.stdout:6/818: mkdir d0/dd/d74/d67/dbb/d100 0 2024-09-16T08:57:27.100 INFO:tasks.workunit.client.0.smithi031.stdout:3/801: read d3/d14/d37/d1a/d1d/d3b/d3f/d49/faa [1257044,56677] 0 2024-09-16T08:57:27.100 INFO:tasks.workunit.client.0.smithi031.stdout:3/802: truncate d3/d14/d85/fe4 421736 0 2024-09-16T08:57:27.103 INFO:tasks.workunit.client.0.smithi031.stdout:4/863: symlink d0/d102/d8d/l115 0 2024-09-16T08:57:27.103 INFO:tasks.workunit.client.0.smithi031.stdout:5/655: write d2/dc/f43 [384427,33499] 0 2024-09-16T08:57:27.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/600: stat d4/d2f/dc3/d37/d46/d8c/c48 0 2024-09-16T08:57:27.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/601: readlink d4/d7/l85 0 2024-09-16T08:57:27.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/602: read - d4/fa9 zero size 2024-09-16T08:57:27.105 INFO:tasks.workunit.client.0.smithi031.stdout:8/603: creat d4/fc5 x:0 0 0 2024-09-16T08:57:27.108 INFO:tasks.workunit.client.0.smithi031.stdout:1/690: symlink d2/d15/d1f/de9/d67/lf3 0 2024-09-16T08:57:27.108 INFO:tasks.workunit.client.0.smithi031.stdout:1/691: truncate d2/d15/d32/f97 1257969 0 2024-09-16T08:57:27.108 INFO:tasks.workunit.client.0.smithi031.stdout:1/692: fsync d2/d15/d1f/d43/f4c 0 2024-09-16T08:57:27.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/819: creat d0/dd/d74/d5e/d7c/f101 x:0 0 0 2024-09-16T08:57:27.112 INFO:tasks.workunit.client.0.smithi031.stdout:3/803: mknod d3/d14/d37/d1a/ce8 0 2024-09-16T08:57:27.115 INFO:tasks.workunit.client.0.smithi031.stdout:9/823: dwrite d3/d5/da/d34/d3c/f103 [0,4194304] 0 2024-09-16T08:57:27.122 INFO:tasks.workunit.client.0.smithi031.stdout:0/816: rmdir d6/d12/d47 39 2024-09-16T08:57:27.122 INFO:tasks.workunit.client.0.smithi031.stdout:0/817: fsync d6/d12/d97/d9e/fb8 0 2024-09-16T08:57:27.123 INFO:tasks.workunit.client.0.smithi031.stdout:9/824: dread d3/d5/da/db/fc0 [0,4194304] 0 2024-09-16T08:57:27.124 INFO:tasks.workunit.client.0.smithi031.stdout:4/864: mkdir d0/d102/d84/d86/d116 0 2024-09-16T08:57:27.125 INFO:tasks.workunit.client.0.smithi031.stdout:5/656: symlink d2/dc/ld6 0 2024-09-16T08:57:27.128 INFO:tasks.workunit.client.0.smithi031.stdout:8/604: creat d4/d2f/dc3/d24/fc6 x:0 0 0 2024-09-16T08:57:27.130 INFO:tasks.workunit.client.0.smithi031.stdout:1/693: creat d2/d15/d1f/de9/d52/ff4 x:0 0 0 2024-09-16T08:57:27.130 INFO:tasks.workunit.client.0.smithi031.stdout:6/820: getdents d0/dd/d74/d19/de1 0 2024-09-16T08:57:27.134 INFO:tasks.workunit.client.0.smithi031.stdout:3/804: creat d3/d14/d37/d1a/d1d/d3b/d94/fe9 x:0 0 0 2024-09-16T08:57:27.134 INFO:tasks.workunit.client.0.smithi031.stdout:3/805: chown d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f72 201835460 1 2024-09-16T08:57:27.134 INFO:tasks.workunit.client.0.smithi031.stdout:3/806: write d3/d14/d37/d1a/d1d/dac/fbc [472445,29492] 0 2024-09-16T08:57:27.135 INFO:tasks.workunit.client.0.smithi031.stdout:4/865: write d0/d2/d16/d1e/db9/fd2 [1124140,90939] 0 2024-09-16T08:57:27.137 INFO:tasks.workunit.client.0.smithi031.stdout:6/821: write d0/d10/f60 [1590547,19040] 0 2024-09-16T08:57:27.137 INFO:tasks.workunit.client.0.smithi031.stdout:6/822: read d0/dd/d74/f55 [387217,29749] 0 2024-09-16T08:57:27.145 INFO:tasks.workunit.client.0.smithi031.stdout:3/807: write d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/fba [4704262,122746] 0 2024-09-16T08:57:27.151 INFO:tasks.workunit.client.0.smithi031.stdout:0/818: mkdir d6/d12/d13/dca/deb/d108 0 2024-09-16T08:57:27.152 INFO:tasks.workunit.client.0.smithi031.stdout:6/823: dwrite d0/dd/d2b/f45 [0,4194304] 0 2024-09-16T08:57:27.152 INFO:tasks.workunit.client.0.smithi031.stdout:6/824: readlink d0/dd/d74/d19/d9a/l9e 0 2024-09-16T08:57:27.153 INFO:tasks.workunit.client.0.smithi031.stdout:9/825: rename d3/d5/da/db/d21/d3b/lc9 to d3/d5/d73/d81/l122 0 2024-09-16T08:57:27.153 INFO:tasks.workunit.client.0.smithi031.stdout:9/826: fsync d3/d5/da/d34/d3c/d5e/d9c/d95/fae 0 2024-09-16T08:57:27.154 INFO:tasks.workunit.client.0.smithi031.stdout:5/657: unlink d2/dc/d1d/c5b 0 2024-09-16T08:57:27.155 INFO:tasks.workunit.client.0.smithi031.stdout:1/694: symlink d2/d15/d32/d87/lf5 0 2024-09-16T08:57:27.158 INFO:tasks.workunit.client.0.smithi031.stdout:4/866: mknod d0/d2/d16/d2b/dfd/c117 0 2024-09-16T08:57:27.160 INFO:tasks.workunit.client.0.smithi031.stdout:3/808: mknod d3/d14/d37/d1a/d1d/d9a/dde/dd6/cea 0 2024-09-16T08:57:27.161 INFO:tasks.workunit.client.0.smithi031.stdout:6/825: dread d0/ff [0,4194304] 0 2024-09-16T08:57:27.161 INFO:tasks.workunit.client.0.smithi031.stdout:6/826: fdatasync d0/dd/d35/d6c/f9b 0 2024-09-16T08:57:27.161 INFO:tasks.workunit.client.0.smithi031.stdout:6/827: write d0/dd/d74/d67/f7a [1039649,49821] 0 2024-09-16T08:57:27.163 INFO:tasks.workunit.client.0.smithi031.stdout:5/658: write d2/dc/d25/f9f [3103058,75207] 0 2024-09-16T08:57:27.164 INFO:tasks.workunit.client.0.smithi031.stdout:8/605: rename d4/d2f/dc3/l22 to d4/d2f/lc7 0 2024-09-16T08:57:27.165 INFO:tasks.workunit.client.0.smithi031.stdout:9/827: creat d3/d5/d73/df5/f123 x:0 0 0 2024-09-16T08:57:27.167 INFO:tasks.workunit.client.0.smithi031.stdout:1/695: creat d2/de/ff6 x:0 0 0 2024-09-16T08:57:27.167 INFO:tasks.workunit.client.0.smithi031.stdout:1/696: creat d2/de/db5/ff7 x:0 0 0 2024-09-16T08:57:27.168 INFO:tasks.workunit.client.0.smithi031.stdout:3/809: creat d3/d14/d37/d1a/d1d/d3b/d8a/feb x:0 0 0 2024-09-16T08:57:27.170 INFO:tasks.workunit.client.0.smithi031.stdout:6/828: symlink d0/dd/d74/d19/dcd/l102 0 2024-09-16T08:57:27.172 INFO:tasks.workunit.client.0.smithi031.stdout:0/819: rename d6/d12/d47/f7e to d6/d12/d47/d7c/f109 0 2024-09-16T08:57:27.172 INFO:tasks.workunit.client.0.smithi031.stdout:0/820: fsync d6/d12/d13/f46 0 2024-09-16T08:57:27.172 INFO:tasks.workunit.client.0.smithi031.stdout:0/821: write d6/d12/d13/f84 [1675500,13115] 0 2024-09-16T08:57:27.172 INFO:tasks.workunit.client.0.smithi031.stdout:0/822: truncate d6/d12/d13/d36/fa6 947309 0 2024-09-16T08:57:27.172 INFO:tasks.workunit.client.0.smithi031.stdout:0/823: truncate d6/d12/d47/d5b/fee 547626 0 2024-09-16T08:57:27.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/606: mkdir d4/d9e/d38/dc8 0 2024-09-16T08:57:27.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/607: dread - d4/d2d/d6e/fa4 zero size 2024-09-16T08:57:27.173 INFO:tasks.workunit.client.0.smithi031.stdout:9/828: symlink d3/d5/dd6/l124 0 2024-09-16T08:57:27.175 INFO:tasks.workunit.client.0.smithi031.stdout:3/810: dread d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f72 [0,4194304] 0 2024-09-16T08:57:27.176 INFO:tasks.workunit.client.0.smithi031.stdout:1/697: creat d2/d15/d1f/de9/ff8 x:0 0 0 2024-09-16T08:57:27.176 INFO:tasks.workunit.client.0.smithi031.stdout:1/698: read - d2/d15/d1f/de9/d1b/d5c/d6b/da9/fe2 zero size 2024-09-16T08:57:27.176 INFO:tasks.workunit.client.0.smithi031.stdout:1/699: read - d2/de/ff6 zero size 2024-09-16T08:57:27.176 INFO:tasks.workunit.client.0.smithi031.stdout:6/829: truncate d0/dd/d74/d67/fbf 3001076 0 2024-09-16T08:57:27.177 INFO:tasks.workunit.client.0.smithi031.stdout:6/830: chown d0/dd/d74/f18 3208 1 2024-09-16T08:57:27.179 INFO:tasks.workunit.client.0.smithi031.stdout:4/867: dwrite d0/d2/f10 [0,4194304] 0 2024-09-16T08:57:27.179 INFO:tasks.workunit.client.0.smithi031.stdout:5/659: rename d2/dc/d3d/f74 to d2/dc/d36/fd7 0 2024-09-16T08:57:27.179 INFO:tasks.workunit.client.0.smithi031.stdout:9/829: dread d3/d5/da/d34/d3c/d5e/d9c/d95/dcf/fdf [0,4194304] 0 2024-09-16T08:57:27.179 INFO:tasks.workunit.client.0.smithi031.stdout:0/824: mknod d6/d12/d47/d5b/c10a 0 2024-09-16T08:57:27.179 INFO:tasks.workunit.client.0.smithi031.stdout:0/825: creat d6/d61/d91/f10b x:0 0 0 2024-09-16T08:57:27.179 INFO:tasks.workunit.client.0.smithi031.stdout:0/826: write d6/fa [4871598,128756] 0 2024-09-16T08:57:27.180 INFO:tasks.workunit.client.0.smithi031.stdout:8/608: symlink d4/d5b/lc9 0 2024-09-16T08:57:27.180 INFO:tasks.workunit.client.0.smithi031.stdout:8/609: fsync d4/d7/d99/fc1 0 2024-09-16T08:57:27.181 INFO:tasks.workunit.client.0.smithi031.stdout:4/868: dread d0/f94 [0,4194304] 0 2024-09-16T08:57:27.182 INFO:tasks.workunit.client.0.smithi031.stdout:3/811: creat d3/d14/d37/d1a/d1d/d3b/fec x:0 0 0 2024-09-16T08:57:27.182 INFO:tasks.workunit.client.0.smithi031.stdout:1/700: mknod d2/d15/d38/cf9 0 2024-09-16T08:57:27.187 INFO:tasks.workunit.client.0.smithi031.stdout:6/831: symlink d0/dd/d74/d5e/d7c/d3f/d65/l103 0 2024-09-16T08:57:27.187 INFO:tasks.workunit.client.0.smithi031.stdout:6/832: chown d0/dd/d74/d5e/d7c/d56/c71 75337895 1 2024-09-16T08:57:27.187 INFO:tasks.workunit.client.0.smithi031.stdout:6/833: write d0/d10/f3b [870515,24726] 0 2024-09-16T08:57:27.187 INFO:tasks.workunit.client.0.smithi031.stdout:3/812: fsync d3/d14/d37/d1a/d1d/dac/fbc 0 2024-09-16T08:57:27.189 INFO:tasks.workunit.client.0.smithi031.stdout:5/660: mknod d2/dc/d3d/db2/cd8 0 2024-09-16T08:57:27.189 INFO:tasks.workunit.client.0.smithi031.stdout:5/661: chown d2/d4b/f8a 254467196 1 2024-09-16T08:57:27.189 INFO:tasks.workunit.client.0.smithi031.stdout:5/662: chown d2/dc/d1d/d97 89765 1 2024-09-16T08:57:27.190 INFO:tasks.workunit.client.0.smithi031.stdout:9/830: symlink d3/d5/d6a/l125 0 2024-09-16T08:57:27.193 INFO:tasks.workunit.client.0.smithi031.stdout:0/827: creat d6/d12/d47/d5b/d9b/f10c x:0 0 0 2024-09-16T08:57:27.195 INFO:tasks.workunit.client.0.smithi031.stdout:4/869: creat d0/d109/d106/f118 x:0 0 0 2024-09-16T08:57:27.197 INFO:tasks.workunit.client.0.smithi031.stdout:6/834: link d0/dd/d74/d19/c8c d0/dd/d35/d6c/dac/c104 0 2024-09-16T08:57:27.198 INFO:tasks.workunit.client.0.smithi031.stdout:6/835: creat d0/dd/d35/d6c/d83/f105 x:0 0 0 2024-09-16T08:57:27.198 INFO:tasks.workunit.client.0.smithi031.stdout:6/836: creat d0/dd/d74/d5e/d7c/d3f/d65/f106 x:0 0 0 2024-09-16T08:57:27.200 INFO:tasks.workunit.client.0.smithi031.stdout:3/813: mkdir d3/d14/d37/d47/ded 0 2024-09-16T08:57:27.203 INFO:tasks.workunit.client.0.smithi031.stdout:5/663: rename d2/f2b to d2/dc/d91/fd9 0 2024-09-16T08:57:27.209 INFO:tasks.workunit.client.0.smithi031.stdout:4/870: dwrite d0/d3c/d6b/d9c/f110 [0,4194304] 0 2024-09-16T08:57:27.210 INFO:tasks.workunit.client.0.smithi031.stdout:4/871: stat d0/d2/d16/d2b/daf/f108 0 2024-09-16T08:57:27.210 INFO:tasks.workunit.client.0.smithi031.stdout:8/610: rmdir d4/d2d/d97 39 2024-09-16T08:57:27.214 INFO:tasks.workunit.client.0.smithi031.stdout:6/837: creat d0/d10/dd2/de3/f107 x:0 0 0 2024-09-16T08:57:27.217 INFO:tasks.workunit.client.0.smithi031.stdout:0/828: symlink d6/d12/d13/l10d 0 2024-09-16T08:57:27.220 INFO:tasks.workunit.client.0.smithi031.stdout:4/872: creat d0/d2/d16/d2b/d10e/f119 x:0 0 0 2024-09-16T08:57:27.220 INFO:tasks.workunit.client.0.smithi031.stdout:4/873: dread - d0/d102/d84/d86/d9b/dcf/dde/fe6 zero size 2024-09-16T08:57:27.220 INFO:tasks.workunit.client.0.smithi031.stdout:3/814: dread d3/f2a [0,4194304] 0 2024-09-16T08:57:27.220 INFO:tasks.workunit.client.0.smithi031.stdout:3/815: creat d3/d14/d93/fee x:0 0 0 2024-09-16T08:57:27.220 INFO:tasks.workunit.client.0.smithi031.stdout:8/611: mknod d4/d2f/dc3/d37/cca 0 2024-09-16T08:57:27.221 INFO:tasks.workunit.client.0.smithi031.stdout:5/664: dwrite d2/dc/f19 [4194304,4194304] 0 2024-09-16T08:57:27.222 INFO:tasks.workunit.client.0.smithi031.stdout:6/838: creat d0/df1/f108 x:0 0 0 2024-09-16T08:57:27.223 INFO:tasks.workunit.client.0.smithi031.stdout:6/839: stat d0/dd/d74/d5e 0 2024-09-16T08:57:27.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/829: unlink d6/d12/d47/d5b/f67 0 2024-09-16T08:57:27.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/830: fdatasync d6/f4f 0 2024-09-16T08:57:27.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/831: creat d6/d12/d97/db3/f10e x:0 0 0 2024-09-16T08:57:27.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/832: truncate d6/d12/d13/dca/ffd 262250 0 2024-09-16T08:57:27.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/833: readlink d6/d12/d13/d3c/le1 0 2024-09-16T08:57:27.226 INFO:tasks.workunit.client.0.smithi031.stdout:0/834: fsync d6/d12/d97/d9e/fd6 0 2024-09-16T08:57:27.226 INFO:tasks.workunit.client.0.smithi031.stdout:4/874: mknod d0/d2/d16/d2b/c11a 0 2024-09-16T08:57:27.230 INFO:tasks.workunit.client.0.smithi031.stdout:5/665: link d2/dc/f89 d2/dc/d36/d76/fda 0 2024-09-16T08:57:27.230 INFO:tasks.workunit.client.0.smithi031.stdout:8/612: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:27.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/816: dread d3/f5 [0,4194304] 0 2024-09-16T08:57:27.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/817: fdatasync d3/d14/d37/d1a/d1d/d6a/fdf 0 2024-09-16T08:57:27.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/818: fdatasync d3/d14/d37/d1a/d1d/d3b/d94/fe9 0 2024-09-16T08:57:27.234 INFO:tasks.workunit.client.0.smithi031.stdout:3/819: creat d3/d14/d37/d1a/d1d/dac/fef x:0 0 0 2024-09-16T08:57:27.235 INFO:tasks.workunit.client.0.smithi031.stdout:4/875: link d0/d8e/d75/lf3 d0/d2/d16/d2b/d10e/l11b 0 2024-09-16T08:57:27.235 INFO:tasks.workunit.client.0.smithi031.stdout:6/840: dread d0/dd/d74/d19/f61 [0,4194304] 0 2024-09-16T08:57:27.235 INFO:tasks.workunit.client.0.smithi031.stdout:6/841: write d0/dd/d74/d5e/d7c/f101 [154633,127151] 0 2024-09-16T08:57:27.236 INFO:tasks.workunit.client.0.smithi031.stdout:6/842: chown d0/c51 37087915 1 2024-09-16T08:57:27.240 INFO:tasks.workunit.client.0.smithi031.stdout:3/820: mkdir d3/d14/d37/d1a/d1d/df0 0 2024-09-16T08:57:27.241 INFO:tasks.workunit.client.0.smithi031.stdout:2/779: sync 2024-09-16T08:57:27.241 INFO:tasks.workunit.client.0.smithi031.stdout:2/780: getdents d6/d2e/d40/dc3 0 2024-09-16T08:57:27.241 INFO:tasks.workunit.client.0.smithi031.stdout:2/781: chown d6/d2e/d40/d58 363 1 2024-09-16T08:57:27.241 INFO:tasks.workunit.client.0.smithi031.stdout:2/782: readlink d6/da/d37/l6b 0 2024-09-16T08:57:27.241 INFO:tasks.workunit.client.0.smithi031.stdout:4/876: creat d0/db5/f11c x:0 0 0 2024-09-16T08:57:27.241 INFO:tasks.workunit.client.0.smithi031.stdout:4/877: write d0/d102/f63 [4841704,27426] 0 2024-09-16T08:57:27.249 INFO:tasks.workunit.client.0.smithi031.stdout:7/846: sync 2024-09-16T08:57:27.261 INFO:tasks.workunit.client.0.smithi031.stdout:2/783: read d6/da/d4b/d4c/df9/fe1 [343722,89373] 0 2024-09-16T08:57:27.262 INFO:tasks.workunit.client.0.smithi031.stdout:0/835: rename d6/d12/d47/d7c to d6/d12/d47/dc5/dc9/d10f 0 2024-09-16T08:57:27.263 INFO:tasks.workunit.client.0.smithi031.stdout:0/836: write d6/f1e [2777143,38265] 0 2024-09-16T08:57:27.264 INFO:tasks.workunit.client.0.smithi031.stdout:2/784: link d6/d20/da1/dcf/dda/de3/fe8 d6/d20/da1/dcf/dda/f104 0 2024-09-16T08:57:27.267 INFO:tasks.workunit.client.0.smithi031.stdout:6/843: rename d0/dd/d74/d5e/d7c/d3f/cd1 to d0/dd/d35/d6c/dac/dba/c109 0 2024-09-16T08:57:27.268 INFO:tasks.workunit.client.0.smithi031.stdout:2/785: stat d6/d2e/d40/d6c/l86 0 2024-09-16T08:57:27.270 INFO:tasks.workunit.client.0.smithi031.stdout:6/844: mkdir d0/dd/d35/d6c/d81/d10a 0 2024-09-16T08:57:27.271 INFO:tasks.workunit.client.0.smithi031.stdout:6/845: truncate d0/dd/d74/d67/f7a 2116382 0 2024-09-16T08:57:27.271 INFO:tasks.workunit.client.0.smithi031.stdout:2/786: symlink d6/d2e/d40/d58/dae/l105 0 2024-09-16T08:57:27.273 INFO:tasks.workunit.client.0.smithi031.stdout:6/846: creat d0/dd/d74/d5e/d7c/d3f/ddd/f10b x:0 0 0 2024-09-16T08:57:27.275 INFO:tasks.workunit.client.0.smithi031.stdout:2/787: mkdir d6/da/d37/deb/d106 0 2024-09-16T08:57:27.275 INFO:tasks.workunit.client.0.smithi031.stdout:8/613: dwrite d4/d9e/f7e [4194304,4194304] 0 2024-09-16T08:57:27.275 INFO:tasks.workunit.client.0.smithi031.stdout:8/614: read - d4/fb4 zero size 2024-09-16T08:57:27.276 INFO:tasks.workunit.client.0.smithi031.stdout:4/878: read d0/d2/f3d [160338,109636] 0 2024-09-16T08:57:27.276 INFO:tasks.workunit.client.0.smithi031.stdout:4/879: chown d0/d102/d84/d86/d116 468839113 1 2024-09-16T08:57:27.277 INFO:tasks.workunit.client.0.smithi031.stdout:6/847: dread d0/dd/d74/f55 [0,4194304] 0 2024-09-16T08:57:27.277 INFO:tasks.workunit.client.0.smithi031.stdout:6/848: write d0/dd/d74/d67/dbb/fc4 [192437,103968] 0 2024-09-16T08:57:27.278 INFO:tasks.workunit.client.0.smithi031.stdout:8/615: rename d4/d2d/d97 to d4/d2f/dc3/dcb 0 2024-09-16T08:57:27.278 INFO:tasks.workunit.client.0.smithi031.stdout:8/616: chown d4/d7/d66 137790947 1 2024-09-16T08:57:27.281 INFO:tasks.workunit.client.0.smithi031.stdout:6/849: creat d0/dd/d74/d19/db4/f10c x:0 0 0 2024-09-16T08:57:27.282 INFO:tasks.workunit.client.0.smithi031.stdout:8/617: mknod d4/d9e/daa/ccc 0 2024-09-16T08:57:27.283 INFO:tasks.workunit.client.0.smithi031.stdout:4/880: rmdir d0/d2/d16/d2b/daf 39 2024-09-16T08:57:27.283 INFO:tasks.workunit.client.0.smithi031.stdout:4/881: read d0/d3c/f62 [3862947,112356] 0 2024-09-16T08:57:27.284 INFO:tasks.workunit.client.0.smithi031.stdout:5/666: dwrite d2/dc/d25/fc1 [0,4194304] 0 2024-09-16T08:57:27.284 INFO:tasks.workunit.client.0.smithi031.stdout:7/847: dwrite d2/d2d/fd8 [0,4194304] 0 2024-09-16T08:57:27.285 INFO:tasks.workunit.client.0.smithi031.stdout:2/788: dread d6/d2e/d40/d6c/d2c/d53/d60/d91/ff3 [0,4194304] 0 2024-09-16T08:57:27.285 INFO:tasks.workunit.client.0.smithi031.stdout:6/850: rename d0/dd/d74/d5e/d7c/c8d to d0/d10/c10d 0 2024-09-16T08:57:27.285 INFO:tasks.workunit.client.0.smithi031.stdout:6/851: fsync d0/dd/d74/d67/fb0 0 2024-09-16T08:57:27.286 INFO:tasks.workunit.client.0.smithi031.stdout:6/852: chown d0/dd/d74/d5e/d7c/lab 161032 1 2024-09-16T08:57:27.287 INFO:tasks.workunit.client.0.smithi031.stdout:0/837: dwrite d6/d12/f95 [0,4194304] 0 2024-09-16T08:57:27.287 INFO:tasks.workunit.client.0.smithi031.stdout:0/838: write d6/d61/d91/f10b [576499,61644] 0 2024-09-16T08:57:27.287 INFO:tasks.workunit.client.0.smithi031.stdout:3/821: dwrite d3/d14/d37/d1a/d1d/dac/fef [0,4194304] 0 2024-09-16T08:57:27.287 INFO:tasks.workunit.client.0.smithi031.stdout:3/822: read d3/d14/d37/d1a/d1d/d3b/dc4/fca [2108942,126657] 0 2024-09-16T08:57:27.287 INFO:tasks.workunit.client.0.smithi031.stdout:4/882: creat d0/d5b/db8/df8/f11d x:0 0 0 2024-09-16T08:57:27.293 INFO:tasks.workunit.client.0.smithi031.stdout:7/848: unlink d2/d8/d86/da1/da9/cf0 0 2024-09-16T08:57:27.294 INFO:tasks.workunit.client.0.smithi031.stdout:6/853: mknod d0/dd/d74/d19/dcd/c10e 0 2024-09-16T08:57:27.296 INFO:tasks.workunit.client.0.smithi031.stdout:6/854: write d0/dd/d74/f2d [1447734,103875] 0 2024-09-16T08:57:27.296 INFO:tasks.workunit.client.0.smithi031.stdout:6/855: stat d0/d10/l44 0 2024-09-16T08:57:27.300 INFO:tasks.workunit.client.0.smithi031.stdout:3/823: mkdir d3/d14/d37/d1a/d1d/df1 0 2024-09-16T08:57:27.301 INFO:tasks.workunit.client.0.smithi031.stdout:4/883: dread d0/f38 [4194304,4194304] 0 2024-09-16T08:57:27.302 INFO:tasks.workunit.client.0.smithi031.stdout:4/884: write d0/d2/d16/d1e/f3b [4157113,39607] 0 2024-09-16T08:57:27.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/618: rename d4/d2f/dc3/d35 to d4/d5b/d8d/dcd 0 2024-09-16T08:57:27.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/619: write d4/fb5 [1205788,104612] 0 2024-09-16T08:57:27.306 INFO:tasks.workunit.client.0.smithi031.stdout:8/620: write d4/d5b/d8d/dcd/fbe [635985,1088] 0 2024-09-16T08:57:27.306 INFO:tasks.workunit.client.0.smithi031.stdout:5/667: dwrite d2/f59 [0,4194304] 0 2024-09-16T08:57:27.316 INFO:tasks.workunit.client.0.smithi031.stdout:8/621: dread d4/d9e/d38/f73 [0,4194304] 0 2024-09-16T08:57:27.319 INFO:tasks.workunit.client.0.smithi031.stdout:6/856: mkdir d0/d10/dc8/d10f 0 2024-09-16T08:57:27.319 INFO:tasks.workunit.client.0.smithi031.stdout:6/857: write d0/dd/d74/d19/f32 [1158284,90911] 0 2024-09-16T08:57:27.329 INFO:tasks.workunit.client.0.smithi031.stdout:5/668: symlink d2/dc/d1d/dbb/ldb 0 2024-09-16T08:57:27.329 INFO:tasks.workunit.client.0.smithi031.stdout:5/669: chown d2/dc/d36/f94 1106 1 2024-09-16T08:57:27.331 INFO:tasks.workunit.client.0.smithi031.stdout:7/849: rename d2/d8/d86/da1/da9/fda to d2/d8/d11/d81/d10b/f117 0 2024-09-16T08:57:27.333 INFO:tasks.workunit.client.0.smithi031.stdout:8/622: symlink d4/d77/lce 0 2024-09-16T08:57:27.334 INFO:tasks.workunit.client.0.smithi031.stdout:7/850: dread d2/d8/d11/d5d/d7e/faf [0,4194304] 0 2024-09-16T08:57:27.334 INFO:tasks.workunit.client.0.smithi031.stdout:7/851: write d2/d8/d86/f88 [933912,84111] 0 2024-09-16T08:57:27.334 INFO:tasks.workunit.client.0.smithi031.stdout:7/852: fsync d2/d2d/f43 0 2024-09-16T08:57:27.342 INFO:tasks.workunit.client.0.smithi031.stdout:6/858: rename d0/dd/d35/d6c/d83 to d0/dd/d74/d5e/d110 0 2024-09-16T08:57:27.343 INFO:tasks.workunit.client.0.smithi031.stdout:9/831: write d3/d5/da/db/d21/d3b/f11d [833217,40730] 0 2024-09-16T08:57:27.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/853: write d2/d8/f63 [4120239,122860] 0 2024-09-16T08:57:27.347 INFO:tasks.workunit.client.0.smithi031.stdout:5/670: rename d2/dc/d3d/fc2 to d2/dc/d91/fdc 0 2024-09-16T08:57:27.351 INFO:tasks.workunit.client.0.smithi031.stdout:5/671: creat d2/d4b/fdd x:0 0 0 2024-09-16T08:57:27.351 INFO:tasks.workunit.client.0.smithi031.stdout:5/672: write d2/dc/d1d/d97/fb9 [519674,63786] 0 2024-09-16T08:57:27.354 INFO:tasks.workunit.client.0.smithi031.stdout:8/623: dwrite d4/d9e/d38/f87 [0,4194304] 0 2024-09-16T08:57:27.354 INFO:tasks.workunit.client.0.smithi031.stdout:8/624: readlink d4/d77/l80 0 2024-09-16T08:57:27.354 INFO:tasks.workunit.client.0.smithi031.stdout:8/625: truncate d4/d77/f9b 1683529 0 2024-09-16T08:57:27.354 INFO:tasks.workunit.client.0.smithi031.stdout:8/626: write d4/d5b/d8d/dcd/d79/f8a [272959,51616] 0 2024-09-16T08:57:27.356 INFO:tasks.workunit.client.0.smithi031.stdout:6/859: dread d0/dd/d35/d6c/f9b [0,4194304] 0 2024-09-16T08:57:27.356 INFO:tasks.workunit.client.0.smithi031.stdout:6/860: dread - d0/d8e/fe8 zero size 2024-09-16T08:57:27.357 INFO:tasks.workunit.client.0.smithi031.stdout:6/861: chown d0/dd/d74/d67/dbb/d100 29 1 2024-09-16T08:57:27.357 INFO:tasks.workunit.client.0.smithi031.stdout:5/673: dread d2/dc/d36/d76/fda [4194304,4194304] 0 2024-09-16T08:57:27.358 INFO:tasks.workunit.client.0.smithi031.stdout:8/627: creat d4/d2f/dc3/d37/d46/fcf x:0 0 0 2024-09-16T08:57:27.360 INFO:tasks.workunit.client.0.smithi031.stdout:6/862: symlink d0/dd/d74/d5e/d110/l111 0 2024-09-16T08:57:27.362 INFO:tasks.workunit.client.0.smithi031.stdout:9/832: dwrite d3/d5/da/fef [0,4194304] 0 2024-09-16T08:57:27.364 INFO:tasks.workunit.client.0.smithi031.stdout:5/674: unlink d2/dc/d25/d21/f4e 0 2024-09-16T08:57:27.365 INFO:tasks.workunit.client.0.smithi031.stdout:6/863: rename d0/dd/d74/d67/f7a to d0/dd/d74/d5e/d7c/d3f/f112 0 2024-09-16T08:57:27.369 INFO:tasks.workunit.client.0.smithi031.stdout:9/833: mkdir d3/d5/dd6/d126 0 2024-09-16T08:57:27.369 INFO:tasks.workunit.client.0.smithi031.stdout:9/834: fsync d3/d5/d73/f9b 0 2024-09-16T08:57:27.371 INFO:tasks.workunit.client.0.smithi031.stdout:5/675: mknod d2/dc/d3d/db2/cde 0 2024-09-16T08:57:27.373 INFO:tasks.workunit.client.0.smithi031.stdout:9/835: creat d3/d5/da/d34/d3c/d5e/f127 x:0 0 0 2024-09-16T08:57:27.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/864: read d0/dd/d74/f30 [70759,130100] 0 2024-09-16T08:57:27.377 INFO:tasks.workunit.client.0.smithi031.stdout:5/676: link d2/dc/c5e d2/dc/d25/d21/d81/cdf 0 2024-09-16T08:57:27.381 INFO:tasks.workunit.client.0.smithi031.stdout:6/865: link d0/dd/d74/f7e d0/dd/d74/d19/d54/f113 0 2024-09-16T08:57:27.383 INFO:tasks.workunit.client.0.smithi031.stdout:5/677: unlink d2/dc/lb6 0 2024-09-16T08:57:27.383 INFO:tasks.workunit.client.0.smithi031.stdout:5/678: chown d2/dc/d91/cb8 13020182 1 2024-09-16T08:57:27.384 INFO:tasks.workunit.client.0.smithi031.stdout:6/866: mkdir d0/dd/d74/d67/dbb/d100/d114 0 2024-09-16T08:57:27.389 INFO:tasks.workunit.client.0.smithi031.stdout:6/867: mknod d0/d10/dd2/de3/c115 0 2024-09-16T08:57:27.397 INFO:tasks.workunit.client.0.smithi031.stdout:8/628: dwrite d4/d7/f51 [0,4194304] 0 2024-09-16T08:57:27.400 INFO:tasks.workunit.client.0.smithi031.stdout:9/836: dwrite d3/d5/dd6/f10a [0,4194304] 0 2024-09-16T08:57:27.418 INFO:tasks.workunit.client.0.smithi031.stdout:0/839: truncate d6/f1e 1945422 0 2024-09-16T08:57:27.430 INFO:tasks.workunit.client.0.smithi031.stdout:6/868: rename d0/dd/d74/cb3 to d0/d10/dc8/df6/c116 0 2024-09-16T08:57:27.433 INFO:tasks.workunit.client.0.smithi031.stdout:8/629: symlink d4/d9e/d38/dc8/ld0 0 2024-09-16T08:57:27.439 INFO:tasks.workunit.client.0.smithi031.stdout:9/837: mknod d3/d5/da/d34/d3c/d5e/d62/c128 0 2024-09-16T08:57:27.439 INFO:tasks.workunit.client.0.smithi031.stdout:9/838: creat d3/d5/d73/df5/f129 x:0 0 0 2024-09-16T08:57:27.439 INFO:tasks.workunit.client.0.smithi031.stdout:9/839: chown d3/d5/da/d34/f46 60976 1 2024-09-16T08:57:27.439 INFO:tasks.workunit.client.0.smithi031.stdout:9/840: creat d3/d5/da/f12a x:0 0 0 2024-09-16T08:57:27.443 INFO:tasks.workunit.client.0.smithi031.stdout:0/840: unlink d6/d12/d13/d36/l8f 0 2024-09-16T08:57:27.447 INFO:tasks.workunit.client.0.smithi031.stdout:0/841: dread d6/f8 [0,4194304] 0 2024-09-16T08:57:27.447 INFO:tasks.workunit.client.0.smithi031.stdout:4/885: truncate d0/d102/f63 2019193 0 2024-09-16T08:57:27.453 INFO:tasks.workunit.client.0.smithi031.stdout:6/869: dwrite d0/dd/d74/d19/d54/fe5 [0,4194304] 0 2024-09-16T08:57:27.461 INFO:tasks.workunit.client.0.smithi031.stdout:4/886: write d0/d102/d84/d86/d9b/fa4 [2025991,64867] 0 2024-09-16T08:57:27.461 INFO:tasks.workunit.client.0.smithi031.stdout:3/824: write d3/d14/d37/d1a/d1d/d3b/f43 [1197795,94118] 0 2024-09-16T08:57:27.477 INFO:tasks.workunit.client.0.smithi031.stdout:9/841: mkdir d3/d5/da/db/d21/d3b/d12b 0 2024-09-16T08:57:27.477 INFO:tasks.workunit.client.0.smithi031.stdout:0/842: mkdir d6/d12/d97/d9e/d110 0 2024-09-16T08:57:27.480 INFO:tasks.workunit.client.0.smithi031.stdout:6/870: truncate d0/dd/d74/d5e/d7c/d3f/d92/fa4 583569 0 2024-09-16T08:57:27.487 INFO:tasks.workunit.client.0.smithi031.stdout:3/825: unlink d3/d14/d37/d1a/f8f 0 2024-09-16T08:57:27.495 INFO:tasks.workunit.client.0.smithi031.stdout:4/887: rename d0/d2/d16/d2b/f67 to d0/d102/d84/d86/d116/f11e 0 2024-09-16T08:57:27.495 INFO:tasks.workunit.client.0.smithi031.stdout:6/871: dread d0/dd/d74/d19/d54/fe5 [0,4194304] 0 2024-09-16T08:57:27.497 INFO:tasks.workunit.client.0.smithi031.stdout:5/679: rmdir d2 39 2024-09-16T08:57:27.497 INFO:tasks.workunit.client.0.smithi031.stdout:5/680: truncate d2/dc/d25/d21/d81/fa4 207816 0 2024-09-16T08:57:27.498 INFO:tasks.workunit.client.0.smithi031.stdout:6/872: dread d0/d8e/fb2 [0,4194304] 0 2024-09-16T08:57:27.499 INFO:tasks.workunit.client.0.smithi031.stdout:7/854: dwrite d2/d2d/f4b [0,4194304] 0 2024-09-16T08:57:27.502 INFO:tasks.workunit.client.0.smithi031.stdout:3/826: read d3/d14/d37/d1a/d1d/dac/fef [1699903,93200] 0 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Adjusting osd_memory_target on smithi031 to 1217M 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:57:27.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: pgmap v15: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 207 MiB/s rd, 265 MiB/s wr, 383 op/s 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:27.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:27 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:27.514 INFO:tasks.workunit.client.0.smithi031.stdout:3/827: write d3/d14/d37/d1a/d1d/d1e/f81 [2025186,114685] 0 2024-09-16T08:57:27.516 INFO:tasks.workunit.client.0.smithi031.stdout:3/828: dread d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d7d/f9f [0,4194304] 0 2024-09-16T08:57:27.517 INFO:tasks.workunit.client.0.smithi031.stdout:8/630: rmdir d4 39 2024-09-16T08:57:27.517 INFO:tasks.workunit.client.0.smithi031.stdout:8/631: stat d4/d5b/d8d/dcd/d50/da3/lbf 0 2024-09-16T08:57:27.521 INFO:tasks.workunit.client.0.smithi031.stdout:4/888: dwrite d0/d2/fe [0,4194304] 0 2024-09-16T08:57:27.521 INFO:tasks.workunit.client.0.smithi031.stdout:6/873: dwrite d0/dd/f23 [0,4194304] 0 2024-09-16T08:57:27.523 INFO:tasks.workunit.client.0.smithi031.stdout:9/842: mkdir d3/d5/da/d12c 0 2024-09-16T08:57:27.531 INFO:tasks.workunit.client.0.smithi031.stdout:8/632: write d4/d7/d99/fc1 [2676661,102328] 0 2024-09-16T08:57:27.531 INFO:tasks.workunit.client.0.smithi031.stdout:8/633: truncate d4/d5b/d8d/fb0 354620 0 2024-09-16T08:57:27.531 INFO:tasks.workunit.client.0.smithi031.stdout:8/634: chown d4/d2f/dc3/d37/d46/f75 1703725 1 2024-09-16T08:57:27.535 INFO:tasks.workunit.client.0.smithi031.stdout:5/681: creat d2/dc/d3d/d4c/fe0 x:0 0 0 2024-09-16T08:57:27.535 INFO:tasks.workunit.client.0.smithi031.stdout:5/682: truncate d2/dc/d1d/f92 151080 0 2024-09-16T08:57:27.543 INFO:tasks.workunit.client.0.smithi031.stdout:9/843: dwrite d3/d5/da/fef [0,4194304] 0 2024-09-16T08:57:27.543 INFO:tasks.workunit.client.0.smithi031.stdout:9/844: chown d3/d48/f68 1915 1 2024-09-16T08:57:27.544 INFO:tasks.workunit.client.0.smithi031.stdout:3/829: symlink d3/d14/d37/d1a/d1d/d3b/d60/lf2 0 2024-09-16T08:57:27.551 INFO:tasks.workunit.client.0.smithi031.stdout:4/889: symlink d0/d3c/d6b/d9c/l11f 0 2024-09-16T08:57:27.552 INFO:tasks.workunit.client.0.smithi031.stdout:6/874: symlink d0/dd/d74/d5e/l117 0 2024-09-16T08:57:27.555 INFO:tasks.workunit.client.0.smithi031.stdout:0/843: getdents d6/d61/d91 0 2024-09-16T08:57:27.555 INFO:tasks.workunit.client.0.smithi031.stdout:1/701: sync 2024-09-16T08:57:27.555 INFO:tasks.workunit.client.0.smithi031.stdout:1/702: chown d2/d15/d1f/de9/d1b/d5c/d6b/da9 0 1 2024-09-16T08:57:27.556 INFO:tasks.workunit.client.0.smithi031.stdout:7/855: write d2/fb [2773472,60348] 0 2024-09-16T08:57:27.557 INFO:tasks.workunit.client.0.smithi031.stdout:8/635: rename d4/d2f/lac to d4/d2f/dc3/d37/d46/ld1 0 2024-09-16T08:57:27.557 INFO:tasks.workunit.client.0.smithi031.stdout:2/789: sync 2024-09-16T08:57:27.557 INFO:tasks.workunit.client.0.smithi031.stdout:2/790: fdatasync d6/da/d5e/d8d/f9e 0 2024-09-16T08:57:27.558 INFO:tasks.workunit.client.0.smithi031.stdout:8/636: dread d4/d9e/d38/fc2 [0,4194304] 0 2024-09-16T08:57:27.560 INFO:tasks.workunit.client.0.smithi031.stdout:5/683: getdents d2/dc/d36/d76 0 2024-09-16T08:57:27.561 INFO:tasks.workunit.client.0.smithi031.stdout:4/890: symlink d0/d2/d16/d1e/db9/l120 0 2024-09-16T08:57:27.563 INFO:tasks.workunit.client.0.smithi031.stdout:0/844: mknod d6/d12/d13/dde/c111 0 2024-09-16T08:57:27.563 INFO:tasks.workunit.client.0.smithi031.stdout:0/845: truncate d6/d12/d13/f24 1944478 0 2024-09-16T08:57:27.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/845: write d3/d5/da/f14 [3124133,58595] 0 2024-09-16T08:57:27.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/846: dread - d3/d5/d73/df5/f129 zero size 2024-09-16T08:57:27.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/847: truncate d3/d5/da/f4f 653755 0 2024-09-16T08:57:27.566 INFO:tasks.workunit.client.0.smithi031.stdout:9/848: chown d3/d5/da/db/d21/d3b/dad/dea 6 1 2024-09-16T08:57:27.566 INFO:tasks.workunit.client.0.smithi031.stdout:1/703: mknod d2/d15/d1f/d43/cfa 0 2024-09-16T08:57:27.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/637: creat d4/d2f/dc3/d37/d46/fd2 x:0 0 0 2024-09-16T08:57:27.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/638: fdatasync d4/d2f/f41 0 2024-09-16T08:57:27.577 INFO:tasks.workunit.client.0.smithi031.stdout:0/846: dread d6/d12/d13/d36/fa9 [0,4194304] 0 2024-09-16T08:57:27.577 INFO:tasks.workunit.client.0.smithi031.stdout:5/684: mknod d2/dc/d3d/ce1 0 2024-09-16T08:57:27.579 INFO:tasks.workunit.client.0.smithi031.stdout:9/849: getdents d3/d5/da/d34/d3c/d5e/d62/d65 0 2024-09-16T08:57:27.579 INFO:tasks.workunit.client.0.smithi031.stdout:7/856: dread d2/d8/d11/f45 [0,4194304] 0 2024-09-16T08:57:27.581 INFO:tasks.workunit.client.0.smithi031.stdout:1/704: mknod d2/d15/d5d/db4/da3/dcb/cfb 0 2024-09-16T08:57:27.582 INFO:tasks.workunit.client.0.smithi031.stdout:3/830: dwrite d3/d14/d37/d33/f39 [0,4194304] 0 2024-09-16T08:57:27.583 INFO:tasks.workunit.client.0.smithi031.stdout:6/875: dwrite d0/dd/d74/d5e/d7c/d3f/d65/f68 [0,4194304] 0 2024-09-16T08:57:27.585 INFO:tasks.workunit.client.0.smithi031.stdout:6/876: write d0/d8e/fb2 [411779,102843] 0 2024-09-16T08:57:27.585 INFO:tasks.workunit.client.0.smithi031.stdout:6/877: read d0/dd/d74/f2d [1109838,92041] 0 2024-09-16T08:57:27.585 INFO:tasks.workunit.client.0.smithi031.stdout:6/878: write d0/dd/d74/d19/dcd/fd5 [840176,113895] 0 2024-09-16T08:57:27.585 INFO:tasks.workunit.client.0.smithi031.stdout:0/847: mknod d6/d12/d13/dca/deb/d108/c112 0 2024-09-16T08:57:27.585 INFO:tasks.workunit.client.0.smithi031.stdout:0/848: chown d6/d12/d47/dc5/de6/lff 6007716 1 2024-09-16T08:57:27.585 INFO:tasks.workunit.client.0.smithi031.stdout:8/639: dread d4/d5b/f62 [0,4194304] 0 2024-09-16T08:57:27.586 INFO:tasks.workunit.client.0.smithi031.stdout:3/831: dread d3/d14/d37/d1a/d1d/f92 [0,4194304] 0 2024-09-16T08:57:27.586 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.586 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.586 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Adjusting osd_memory_target on smithi031 to 1217M 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:27.587 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: pgmap v15: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 207 MiB/s rd, 265 MiB/s wr, 383 op/s 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.588 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.589 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:27.589 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:27.589 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:27.589 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:27 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:27.590 INFO:tasks.workunit.client.0.smithi031.stdout:4/891: dwrite d0/d3c/f62 [0,4194304] 0 2024-09-16T08:57:27.590 INFO:tasks.workunit.client.0.smithi031.stdout:2/791: rename d6/da/d5e/d8d/dcc to d6/d2e/d107 0 2024-09-16T08:57:27.590 INFO:tasks.workunit.client.0.smithi031.stdout:2/792: fsync d6/da/f92 0 2024-09-16T08:57:27.590 INFO:tasks.workunit.client.0.smithi031.stdout:2/793: chown d6/l59 62879 1 2024-09-16T08:57:27.590 INFO:tasks.workunit.client.0.smithi031.stdout:2/794: creat d6/d20/da1/dcf/f108 x:0 0 0 2024-09-16T08:57:27.590 INFO:tasks.workunit.client.0.smithi031.stdout:3/832: dread d3/d14/d37/d47/fcd [0,4194304] 0 2024-09-16T08:57:27.590 INFO:tasks.workunit.client.0.smithi031.stdout:6/879: link d0/dd/d74/d19/d54/fdc d0/dd/d74/d67/f118 0 2024-09-16T08:57:27.593 INFO:tasks.workunit.client.0.smithi031.stdout:5/685: rename d2/dc/d25/d21/fac to d2/dc/d36/fe2 0 2024-09-16T08:57:27.593 INFO:tasks.workunit.client.0.smithi031.stdout:0/849: dread d6/d12/d47/d5b/d9b/fa8 [0,4194304] 0 2024-09-16T08:57:27.593 INFO:tasks.workunit.client.0.smithi031.stdout:0/850: dread - d6/d12/d13/da5/fbc zero size 2024-09-16T08:57:27.593 INFO:tasks.workunit.client.0.smithi031.stdout:0/851: fdatasync d6/d61/d91/ff9 0 2024-09-16T08:57:27.593 INFO:tasks.workunit.client.0.smithi031.stdout:0/852: readlink d6/d12/d13/l8e 0 2024-09-16T08:57:27.594 INFO:tasks.workunit.client.0.smithi031.stdout:4/892: mkdir d0/d102/d84/d86/d9b/dcf/d121 0 2024-09-16T08:57:27.597 INFO:tasks.workunit.client.0.smithi031.stdout:3/833: mknod d3/d14/d37/cf3 0 2024-09-16T08:57:27.598 INFO:tasks.workunit.client.0.smithi031.stdout:8/640: write d4/d7/f51 [765480,44482] 0 2024-09-16T08:57:27.598 INFO:tasks.workunit.client.0.smithi031.stdout:8/641: stat d4/d2f/dc3/f23 0 2024-09-16T08:57:27.599 INFO:tasks.workunit.client.0.smithi031.stdout:9/850: rename d3/d5/da/d34/d3c/l7f to d3/d5/d73/d81/df9/l12d 0 2024-09-16T08:57:27.600 INFO:tasks.workunit.client.0.smithi031.stdout:5/686: symlink d2/dc/d25/dc8/le3 0 2024-09-16T08:57:27.600 INFO:tasks.workunit.client.0.smithi031.stdout:5/687: dread - d2/dc/d25/d21/fab zero size 2024-09-16T08:57:27.600 INFO:tasks.workunit.client.0.smithi031.stdout:9/851: dread d3/d5/da/d34/d3c/d5e/fb1 [0,4194304] 0 2024-09-16T08:57:27.601 INFO:tasks.workunit.client.0.smithi031.stdout:9/852: readlink d3/d5/da/d34/d4d/l54 0 2024-09-16T08:57:27.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/853: link d6/d12/d13/c88 d6/d12/d97/d9e/d110/c113 0 2024-09-16T08:57:27.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/854: chown d6/d12/d97/ccf 77 1 2024-09-16T08:57:27.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/855: stat d6/d12/d47/d5b/fee 0 2024-09-16T08:57:27.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/856: truncate d6/d12/d47/d5b/f92 187703 0 2024-09-16T08:57:27.602 INFO:tasks.workunit.client.0.smithi031.stdout:0/857: write d6/d61/fe9 [436947,105632] 0 2024-09-16T08:57:27.603 INFO:tasks.workunit.client.0.smithi031.stdout:8/642: write d4/d2f/dc3/dcb/fb2 [1777885,62214] 0 2024-09-16T08:57:27.605 INFO:tasks.workunit.client.0.smithi031.stdout:4/893: symlink d0/d102/d84/d86/d9b/dcf/l122 0 2024-09-16T08:57:27.608 INFO:tasks.workunit.client.0.smithi031.stdout:3/834: mkdir d3/d14/d37/d47/ded/df4 0 2024-09-16T08:57:27.611 INFO:tasks.workunit.client.0.smithi031.stdout:7/857: rename d2/d2d/f6f to d2/d8/d86/da1/da9/f118 0 2024-09-16T08:57:27.613 INFO:tasks.workunit.client.0.smithi031.stdout:5/688: mkdir d2/dc/d25/dbf/de4 0 2024-09-16T08:57:27.614 INFO:tasks.workunit.client.0.smithi031.stdout:9/853: mknod d3/d5/da/d34/d3c/d5e/d62/c12e 0 2024-09-16T08:57:27.615 INFO:tasks.workunit.client.0.smithi031.stdout:4/894: mknod d0/d2/c123 0 2024-09-16T08:57:27.617 INFO:tasks.workunit.client.0.smithi031.stdout:3/835: creat d3/d14/d37/ff5 x:0 0 0 2024-09-16T08:57:27.619 INFO:tasks.workunit.client.0.smithi031.stdout:1/705: rename d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea/ca8 to d2/d15/d1f/de9/d1b/d5c/d6b/cfc 0 2024-09-16T08:57:27.622 INFO:tasks.workunit.client.0.smithi031.stdout:0/858: mknod d6/c114 0 2024-09-16T08:57:27.623 INFO:tasks.workunit.client.0.smithi031.stdout:5/689: rename d2/dc/d36/fc5 to d2/dc/d3d/d4c/fe5 0 2024-09-16T08:57:27.623 INFO:tasks.workunit.client.0.smithi031.stdout:1/706: truncate d2/d15/d1f/de9/d1b/d5c/f8e 3464053 0 2024-09-16T08:57:27.624 INFO:tasks.workunit.client.0.smithi031.stdout:8/643: dwrite d4/d9e/d38/f73 [0,4194304] 0 2024-09-16T08:57:27.627 INFO:tasks.workunit.client.0.smithi031.stdout:5/690: link d2/dc/d36/f52 d2/dc/d25/da2/fe6 0 2024-09-16T08:57:27.627 INFO:tasks.workunit.client.0.smithi031.stdout:5/691: fsync d2/dc/d25/d21/f6d 0 2024-09-16T08:57:27.628 INFO:tasks.workunit.client.0.smithi031.stdout:5/692: truncate d2/dc/d25/d21/d2e/f72 2367414 0 2024-09-16T08:57:27.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/693: symlink d2/dc/d25/d68/le7 0 2024-09-16T08:57:27.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/694: chown d2/dc/d36/d76/ca0 167 1 2024-09-16T08:57:27.629 INFO:tasks.workunit.client.0.smithi031.stdout:5/695: stat d2/dc/d25/d21/d2e/f5c 0 2024-09-16T08:57:27.630 INFO:tasks.workunit.client.0.smithi031.stdout:5/696: chown d2/d4b/f8b 252 1 2024-09-16T08:57:27.630 INFO:tasks.workunit.client.0.smithi031.stdout:5/697: truncate d2/dc/d1d/f92 159896 0 2024-09-16T08:57:27.667 INFO:tasks.workunit.client.0.smithi031.stdout:8/644: fsync d4/d5b/f62 0 2024-09-16T08:57:27.678 INFO:tasks.workunit.client.0.smithi031.stdout:8/645: dread d4/d9e/d38/f87 [0,4194304] 0 2024-09-16T08:57:27.678 INFO:tasks.workunit.client.0.smithi031.stdout:6/880: truncate d0/dd/d74/d19/d54/fe5 1014472 0 2024-09-16T08:57:27.678 INFO:tasks.workunit.client.0.smithi031.stdout:6/881: stat d0/dd/d74/f7e 0 2024-09-16T08:57:27.679 INFO:tasks.workunit.client.0.smithi031.stdout:8/646: mkdir d4/d77/dd3 0 2024-09-16T08:57:27.679 INFO:tasks.workunit.client.0.smithi031.stdout:3/836: dwrite d3/d14/d37/d1a/d1d/dac/fc1 [0,4194304] 0 2024-09-16T08:57:27.680 INFO:tasks.workunit.client.0.smithi031.stdout:3/837: dread d3/d14/d37/d1a/d1d/d9a/fa2 [0,4194304] 0 2024-09-16T08:57:27.681 INFO:tasks.workunit.client.0.smithi031.stdout:6/882: link d0/dd/d74/d5e/fed d0/dd/d74/d5e/d7c/d3f/d92/f119 0 2024-09-16T08:57:27.683 INFO:tasks.workunit.client.0.smithi031.stdout:5/698: dwrite d2/dc/d25/d21/d81/dc3/fc6 [0,4194304] 0 2024-09-16T08:57:27.684 INFO:tasks.workunit.client.0.smithi031.stdout:0/859: dwrite d6/d12/d13/ff3 [0,4194304] 0 2024-09-16T08:57:27.684 INFO:tasks.workunit.client.0.smithi031.stdout:0/860: fsync d6/d12/d97/db3/dba/fdc 0 2024-09-16T08:57:27.684 INFO:tasks.workunit.client.0.smithi031.stdout:9/854: rmdir d3/d5/da 39 2024-09-16T08:57:27.685 INFO:tasks.workunit.client.0.smithi031.stdout:3/838: symlink d3/d14/d37/lf6 0 2024-09-16T08:57:27.686 INFO:tasks.workunit.client.0.smithi031.stdout:6/883: creat d0/dd/d2b/f11a x:0 0 0 2024-09-16T08:57:27.692 INFO:tasks.workunit.client.0.smithi031.stdout:5/699: mkdir d2/dc/d8c/de8 0 2024-09-16T08:57:27.693 INFO:tasks.workunit.client.0.smithi031.stdout:9/855: dread d3/d48/fc8 [0,4194304] 0 2024-09-16T08:57:27.693 INFO:tasks.workunit.client.0.smithi031.stdout:0/861: write d6/d12/d13/d3c/f45 [3859743,99613] 0 2024-09-16T08:57:27.698 INFO:tasks.workunit.client.0.smithi031.stdout:3/839: symlink d3/d14/d37/d1a/d1d/d9a/dde/dd6/lf7 0 2024-09-16T08:57:27.702 INFO:tasks.workunit.client.0.smithi031.stdout:9/856: unlink d3/d5/da/d34/d4d/cec 0 2024-09-16T08:57:27.706 INFO:tasks.workunit.client.0.smithi031.stdout:0/862: mkdir d6/dac/d115 0 2024-09-16T08:57:27.707 INFO:tasks.workunit.client.0.smithi031.stdout:3/840: rename d3/d14/d37/d1a/d1d/d3b/d60/f7e to d3/d14/d37/d1a/d1d/d1e/ff8 0 2024-09-16T08:57:27.709 INFO:tasks.workunit.client.0.smithi031.stdout:5/700: dwrite d2/dc/d25/d21/d2e/f8f [0,4194304] 0 2024-09-16T08:57:27.711 INFO:tasks.workunit.client.0.smithi031.stdout:3/841: dread d3/d14/d37/d1a/f65 [0,4194304] 0 2024-09-16T08:57:27.711 INFO:tasks.workunit.client.0.smithi031.stdout:3/842: write d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f73 [2437171,115050] 0 2024-09-16T08:57:27.711 INFO:tasks.workunit.client.0.smithi031.stdout:3/843: fdatasync d3/d14/d37/d1a/d1d/dac/fbc 0 2024-09-16T08:57:27.711 INFO:tasks.workunit.client.0.smithi031.stdout:3/844: truncate d3/d14/d37/d1a/d24/fbf 503960 0 2024-09-16T08:57:27.712 INFO:tasks.workunit.client.0.smithi031.stdout:9/857: symlink d3/d5/d6a/l12f 0 2024-09-16T08:57:27.713 INFO:tasks.workunit.client.0.smithi031.stdout:8/647: truncate d4/d2f/dc3/dcb/fb2 2829192 0 2024-09-16T08:57:27.719 INFO:tasks.workunit.client.0.smithi031.stdout:3/845: creat d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d7d/ff9 x:0 0 0 2024-09-16T08:57:27.720 INFO:tasks.workunit.client.0.smithi031.stdout:1/707: dwrite d2/d15/d4a/f6a [0,4194304] 0 2024-09-16T08:57:27.721 INFO:tasks.workunit.client.0.smithi031.stdout:9/858: getdents d3/d5/da/db 0 2024-09-16T08:57:27.721 INFO:tasks.workunit.client.0.smithi031.stdout:9/859: chown d3/d5/da/d34/d4d/l54 33544276 1 2024-09-16T08:57:27.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/648: creat d4/d5b/fd4 x:0 0 0 2024-09-16T08:57:27.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/708: creat d2/d15/d5d/db4/de3/ffd x:0 0 0 2024-09-16T08:57:27.725 INFO:tasks.workunit.client.0.smithi031.stdout:1/709: chown d2/d15/d1f/de9/d52/fd3 32 1 2024-09-16T08:57:27.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/649: mknod d4/d2d/cd5 0 2024-09-16T08:57:27.726 INFO:tasks.workunit.client.0.smithi031.stdout:8/650: chown d4/c1e 0 1 2024-09-16T08:57:27.727 INFO:tasks.workunit.client.0.smithi031.stdout:1/710: unlink d2/d15/d5d/db4/da3/dcb/ld7 0 2024-09-16T08:57:27.727 INFO:tasks.workunit.client.0.smithi031.stdout:1/711: dread - d2/d15/d5d/db4/fc3 zero size 2024-09-16T08:57:27.728 INFO:tasks.workunit.client.0.smithi031.stdout:8/651: symlink d4/d2f/dc3/d37/d46/d88/ld6 0 2024-09-16T08:57:27.729 INFO:tasks.workunit.client.0.smithi031.stdout:8/652: mkdir d4/d2d/dd7 0 2024-09-16T08:57:27.729 INFO:tasks.workunit.client.0.smithi031.stdout:8/653: readlink d4/l12 0 2024-09-16T08:57:27.729 INFO:tasks.workunit.client.0.smithi031.stdout:8/654: write d4/d5b/d8d/dcd/d79/f8a [1323368,35106] 0 2024-09-16T08:57:27.743 INFO:tasks.workunit.client.0.smithi031.stdout:1/712: dread d2/d15/d38/fb2 [0,4194304] 0 2024-09-16T08:57:27.743 INFO:tasks.workunit.client.0.smithi031.stdout:0/863: dwrite d6/d61/f62 [0,4194304] 0 2024-09-16T08:57:27.744 INFO:tasks.workunit.client.0.smithi031.stdout:1/713: dread d2/d15/d38/f47 [0,4194304] 0 2024-09-16T08:57:27.744 INFO:tasks.workunit.client.0.smithi031.stdout:1/714: write d2/d15/d9c/fce [427394,5399] 0 2024-09-16T08:57:27.745 INFO:tasks.workunit.client.0.smithi031.stdout:0/864: unlink d6/d12/f22 0 2024-09-16T08:57:27.746 INFO:tasks.workunit.client.0.smithi031.stdout:5/701: dwrite d2/dc/d1d/fa1 [0,4194304] 0 2024-09-16T08:57:27.746 INFO:tasks.workunit.client.0.smithi031.stdout:9/860: dwrite d3/d5/db3/f117 [0,4194304] 0 2024-09-16T08:57:27.746 INFO:tasks.workunit.client.0.smithi031.stdout:9/861: stat d3/d48/f9d 0 2024-09-16T08:57:27.747 INFO:tasks.workunit.client.0.smithi031.stdout:8/655: dwrite d4/fb5 [0,4194304] 0 2024-09-16T08:57:27.747 INFO:tasks.workunit.client.0.smithi031.stdout:8/656: chown d4/d7/d66/l7a 7335 1 2024-09-16T08:57:27.748 INFO:tasks.workunit.client.0.smithi031.stdout:0/865: rename d6/d12/d47/c81 to d6/d12/c116 0 2024-09-16T08:57:27.750 INFO:tasks.workunit.client.0.smithi031.stdout:5/702: symlink d2/dc/d25/le9 0 2024-09-16T08:57:27.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/703: creat d2/dc/d3d/d4c/fea x:0 0 0 2024-09-16T08:57:27.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/704: write d2/dc/d3d/db2/fb3 [909318,51417] 0 2024-09-16T08:57:27.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/705: read - d2/dc/d25/d68/fcc zero size 2024-09-16T08:57:27.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/706: write d2/dc/d91/fdc [912872,111910] 0 2024-09-16T08:57:27.751 INFO:tasks.workunit.client.0.smithi031.stdout:5/707: readlink d2/dc/d25/d68/le7 0 2024-09-16T08:57:27.753 INFO:tasks.workunit.client.0.smithi031.stdout:9/862: mknod d3/d5/da/db/dcb/d11c/c130 0 2024-09-16T08:57:27.753 INFO:tasks.workunit.client.0.smithi031.stdout:9/863: truncate d3/d5/da/d34/d3c/d5e/d62/d65/f94 188313 0 2024-09-16T08:57:27.756 INFO:tasks.workunit.client.0.smithi031.stdout:0/866: mknod d6/d12/d13/dca/c117 0 2024-09-16T08:57:27.764 INFO:tasks.workunit.client.0.smithi031.stdout:5/708: dread d2/dc/d25/d68/f6b [0,4194304] 0 2024-09-16T08:57:27.765 INFO:tasks.workunit.client.0.smithi031.stdout:5/709: symlink d2/dc/d1d/d97/leb 0 2024-09-16T08:57:27.766 INFO:tasks.workunit.client.0.smithi031.stdout:5/710: fdatasync d2/dc/fba 0 2024-09-16T08:57:27.766 INFO:tasks.workunit.client.0.smithi031.stdout:5/711: truncate d2/dc/d36/f41 903755 0 2024-09-16T08:57:27.767 INFO:tasks.workunit.client.0.smithi031.stdout:5/712: mknod d2/dc/d25/d68/cec 0 2024-09-16T08:57:27.767 INFO:tasks.workunit.client.0.smithi031.stdout:0/867: dread d6/f27 [0,4194304] 0 2024-09-16T08:57:27.769 INFO:tasks.workunit.client.0.smithi031.stdout:1/715: dwrite d2/de/f13 [4194304,4194304] 0 2024-09-16T08:57:27.769 INFO:tasks.workunit.client.0.smithi031.stdout:8/657: dwrite d4/d2d/d6e/fab [0,4194304] 0 2024-09-16T08:57:27.770 INFO:tasks.workunit.client.0.smithi031.stdout:9/864: write d3/d5/da/d34/d3c/d5e/d62/d65/f93 [139644,84607] 0 2024-09-16T08:57:27.771 INFO:tasks.workunit.client.0.smithi031.stdout:5/713: unlink d2/dc/d25/d21/d81/fa4 0 2024-09-16T08:57:27.771 INFO:tasks.workunit.client.0.smithi031.stdout:5/714: stat d2/dc/d25/dbf/de4 0 2024-09-16T08:57:27.772 INFO:tasks.workunit.client.0.smithi031.stdout:9/865: dread d3/d85/fe6 [0,4194304] 0 2024-09-16T08:57:27.773 INFO:tasks.workunit.client.0.smithi031.stdout:0/868: creat d6/d12/d97/f118 x:0 0 0 2024-09-16T08:57:27.775 INFO:tasks.workunit.client.0.smithi031.stdout:1/716: symlink d2/d15/d4a/lfe 0 2024-09-16T08:57:27.779 INFO:tasks.workunit.client.0.smithi031.stdout:5/715: link d2/dc/d3d/c5a d2/dc/d25/dbf/de4/ced 0 2024-09-16T08:57:27.780 INFO:tasks.workunit.client.0.smithi031.stdout:9/866: creat d3/d5/da/d34/f131 x:0 0 0 2024-09-16T08:57:27.781 INFO:tasks.workunit.client.0.smithi031.stdout:0/869: symlink d6/d12/d13/d3c/l119 0 2024-09-16T08:57:27.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/716: mkdir d2/dc/d91/dee 0 2024-09-16T08:57:27.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/717: dread - d2/dc/d36/fae zero size 2024-09-16T08:57:27.787 INFO:tasks.workunit.client.0.smithi031.stdout:5/718: write d2/dc/d25/da2/fe6 [1121679,63858] 0 2024-09-16T08:57:27.788 INFO:tasks.workunit.client.0.smithi031.stdout:9/867: symlink d3/d5/d73/d81/l132 0 2024-09-16T08:57:27.789 INFO:tasks.workunit.client.0.smithi031.stdout:9/868: creat d3/d5/d73/f133 x:0 0 0 2024-09-16T08:57:27.790 INFO:tasks.workunit.client.0.smithi031.stdout:0/870: unlink d6/d12/d13/f46 0 2024-09-16T08:57:27.792 INFO:tasks.workunit.client.0.smithi031.stdout:5/719: creat d2/dc/d8c/de8/fef x:0 0 0 2024-09-16T08:57:27.796 INFO:tasks.workunit.client.0.smithi031.stdout:9/869: symlink d3/d5/da/db/l134 0 2024-09-16T08:57:27.799 INFO:tasks.workunit.client.0.smithi031.stdout:0/871: creat d6/d12/d13/dca/deb/f11a x:0 0 0 2024-09-16T08:57:27.801 INFO:tasks.workunit.client.0.smithi031.stdout:9/870: rename d3/d5/da/db/d21/lab to d3/d5/da/d34/d3c/d5e/d9c/dcd/l135 0 2024-09-16T08:57:27.802 INFO:tasks.workunit.client.0.smithi031.stdout:9/871: fdatasync d3/d5/d6a/d74/da8/dc3/f7b 0 2024-09-16T08:57:27.802 INFO:tasks.workunit.client.0.smithi031.stdout:8/658: truncate d4/d5b/d8d/dcd/d79/f8a 753586 0 2024-09-16T08:57:27.804 INFO:tasks.workunit.client.0.smithi031.stdout:0/872: truncate d6/d12/d13/d3c/f45 2487231 0 2024-09-16T08:57:27.806 INFO:tasks.workunit.client.0.smithi031.stdout:0/873: getdents d6/d12/d47/d5b/d9b/da2/dcc 0 2024-09-16T08:57:27.807 INFO:tasks.workunit.client.0.smithi031.stdout:0/874: rmdir d6/d12/d47/dc5 39 2024-09-16T08:57:27.815 INFO:tasks.workunit.client.0.smithi031.stdout:8/659: write d4/f6 [1870879,6032] 0 2024-09-16T08:57:27.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/660: read d4/d2f/dc3/dcb/fb2 [2190822,119638] 0 2024-09-16T08:57:27.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/661: readlink d4/d9e/l4e 0 2024-09-16T08:57:27.816 INFO:tasks.workunit.client.0.smithi031.stdout:8/662: chown d4/d7/d66/l7a 1225303 1 2024-09-16T08:57:27.817 INFO:tasks.workunit.client.0.smithi031.stdout:5/720: dwrite d2/fa [0,4194304] 0 2024-09-16T08:57:27.817 INFO:tasks.workunit.client.0.smithi031.stdout:8/663: getdents d4/d9e/d38 0 2024-09-16T08:57:27.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/721: rename d2/dc/d25/d68/fcc to d2/dc/d25/dc8/ff0 0 2024-09-16T08:57:27.818 INFO:tasks.workunit.client.0.smithi031.stdout:5/722: fdatasync d2/dc/d25/d21/f85 0 2024-09-16T08:57:27.818 INFO:tasks.workunit.client.0.smithi031.stdout:8/664: mknod d4/d2f/dc3/cd8 0 2024-09-16T08:57:27.819 INFO:tasks.workunit.client.0.smithi031.stdout:9/872: dwrite d3/d5/dd6/fdb [0,4194304] 0 2024-09-16T08:57:27.820 INFO:tasks.workunit.client.0.smithi031.stdout:5/723: link d2/dc/d3d/d4c/fea d2/dc/d25/dad/ff1 0 2024-09-16T08:57:27.822 INFO:tasks.workunit.client.0.smithi031.stdout:0/875: read d6/f33 [9120,28394] 0 2024-09-16T08:57:27.825 INFO:tasks.workunit.client.0.smithi031.stdout:8/665: rename d4/fb7 to d4/d2f/dc3/d37/d46/d8c/fd9 0 2024-09-16T08:57:27.825 INFO:tasks.workunit.client.0.smithi031.stdout:8/666: getdents d4/d2d/dd7 0 2024-09-16T08:57:27.825 INFO:tasks.workunit.client.0.smithi031.stdout:8/667: stat d4/d5b/l90 0 2024-09-16T08:57:27.825 INFO:tasks.workunit.client.0.smithi031.stdout:9/873: truncate d3/d5/da/d34/d3c/d5e/d62/d65/f93 3421484 0 2024-09-16T08:57:27.832 INFO:tasks.workunit.client.0.smithi031.stdout:8/668: unlink d4/d7/d66/l89 0 2024-09-16T08:57:27.836 INFO:tasks.workunit.client.0.smithi031.stdout:8/669: creat d4/d9e/d38/fda x:0 0 0 2024-09-16T08:57:27.847 INFO:tasks.workunit.client.0.smithi031.stdout:9/874: dwrite d3/d5/da/f4f [0,4194304] 0 2024-09-16T08:57:27.850 INFO:tasks.workunit.client.0.smithi031.stdout:9/875: unlink d3/d48/f68 0 2024-09-16T08:57:27.850 INFO:tasks.workunit.client.0.smithi031.stdout:9/876: chown d3/d5/d6a/d74/da8 5 1 2024-09-16T08:57:27.857 INFO:tasks.workunit.client.0.smithi031.stdout:5/724: dwrite d2/dc/d36/fe2 [0,4194304] 0 2024-09-16T08:57:27.858 INFO:tasks.workunit.client.0.smithi031.stdout:0/876: dwrite d6/d12/d13/f39 [0,4194304] 0 2024-09-16T08:57:27.859 INFO:tasks.workunit.client.0.smithi031.stdout:0/877: chown d6/d61/d91/lbf 1 1 2024-09-16T08:57:27.860 INFO:tasks.workunit.client.0.smithi031.stdout:8/670: dwrite d4/d2f/dc3/d37/d46/db1/f6a [0,4194304] 0 2024-09-16T08:57:27.861 INFO:tasks.workunit.client.0.smithi031.stdout:9/877: dread d3/d5/da/db/faa [0,4194304] 0 2024-09-16T08:57:27.864 INFO:tasks.workunit.client.0.smithi031.stdout:9/878: rename d3/d85/fe6 to d3/d5/da/d34/f136 0 2024-09-16T08:57:27.864 INFO:tasks.workunit.client.0.smithi031.stdout:9/879: fdatasync d3/d5/da/db/d21/d3b/dad/dea/fac 0 2024-09-16T08:57:27.868 INFO:tasks.workunit.client.0.smithi031.stdout:8/671: rmdir d4/d2d/dd7 0 2024-09-16T08:57:27.868 INFO:tasks.workunit.client.0.smithi031.stdout:8/672: chown d4/d5b/d8d/c94 861615055 1 2024-09-16T08:57:27.868 INFO:tasks.workunit.client.0.smithi031.stdout:8/673: write d4/d2f/f95 [113636,115197] 0 2024-09-16T08:57:27.869 INFO:tasks.workunit.client.0.smithi031.stdout:9/880: dread d3/fc [0,4194304] 0 2024-09-16T08:57:27.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/878: dread d6/d12/d47/d5b/f71 [0,4194304] 0 2024-09-16T08:57:27.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/879: readlink d6/d12/d13/d3c/l119 0 2024-09-16T08:57:27.871 INFO:tasks.workunit.client.0.smithi031.stdout:0/880: chown d6/d12/d13/dca/ffd 578377 1 2024-09-16T08:57:27.872 INFO:tasks.workunit.client.0.smithi031.stdout:0/881: rename d6/d12/c1d to d6/d12/c11b 0 2024-09-16T08:57:27.875 INFO:tasks.workunit.client.0.smithi031.stdout:5/725: dwrite d2/dc/d91/fb4 [0,4194304] 0 2024-09-16T08:57:27.875 INFO:tasks.workunit.client.0.smithi031.stdout:5/726: truncate d2/dc/d25/d21/d2e/fa8 994520 0 2024-09-16T08:57:27.878 INFO:tasks.workunit.client.0.smithi031.stdout:8/674: dread d4/d77/f81 [0,4194304] 0 2024-09-16T08:57:27.879 INFO:tasks.workunit.client.0.smithi031.stdout:5/727: symlink d2/dc/d8c/de8/lf2 0 2024-09-16T08:57:27.885 INFO:tasks.workunit.client.0.smithi031.stdout:9/881: dwrite d3/d5/da/d34/d4d/df1/f10d [0,4194304] 0 2024-09-16T08:57:27.891 INFO:tasks.workunit.client.0.smithi031.stdout:8/675: creat d4/d77/dd3/fdb x:0 0 0 2024-09-16T08:57:27.893 INFO:tasks.workunit.client.0.smithi031.stdout:8/676: fdatasync d4/d2f/f95 0 2024-09-16T08:57:27.893 INFO:tasks.workunit.client.0.smithi031.stdout:0/882: dwrite d6/d12/d13/da5/fbc [0,4194304] 0 2024-09-16T08:57:27.893 INFO:tasks.workunit.client.0.smithi031.stdout:0/883: write d6/f6e [5127264,96544] 0 2024-09-16T08:57:28.019 INFO:tasks.workunit.client.0.smithi031.stdout:5/728: mkdir d2/dc/df3 0 2024-09-16T08:57:28.021 INFO:tasks.workunit.client.0.smithi031.stdout:5/729: write d2/dc/d91/fdc [630750,78132] 0 2024-09-16T08:57:28.028 INFO:tasks.workunit.client.0.smithi031.stdout:8/677: truncate d4/d5b/d8d/dcd/d50/f98 760375 0 2024-09-16T08:57:28.028 INFO:tasks.workunit.client.0.smithi031.stdout:8/678: chown d4/d5b/d7b/l8b 91374 1 2024-09-16T08:57:28.030 INFO:tasks.workunit.client.0.smithi031.stdout:0/884: mkdir d6/d12/d13/dca/d11c 0 2024-09-16T08:57:28.032 INFO:tasks.workunit.client.0.smithi031.stdout:9/882: dread d3/d5/f2d [0,4194304] 0 2024-09-16T08:57:28.032 INFO:tasks.workunit.client.0.smithi031.stdout:5/730: rmdir d2/dc/d1d 39 2024-09-16T08:57:28.034 INFO:tasks.workunit.client.0.smithi031.stdout:0/885: unlink d6/d12/d47/dc5/de6/lff 0 2024-09-16T08:57:28.034 INFO:tasks.workunit.client.0.smithi031.stdout:9/883: dread d3/d5/da/f33 [0,4194304] 0 2024-09-16T08:57:28.034 INFO:tasks.workunit.client.0.smithi031.stdout:9/884: write d3/d5/da/f12a [524380,105300] 0 2024-09-16T08:57:28.036 INFO:tasks.workunit.client.0.smithi031.stdout:5/731: symlink d2/dc/d25/lf4 0 2024-09-16T08:57:28.036 INFO:tasks.workunit.client.0.smithi031.stdout:5/732: fsync d2/dc/f19 0 2024-09-16T08:57:28.037 INFO:tasks.workunit.client.0.smithi031.stdout:0/886: symlink d6/d12/d97/db3/dba/l11d 0 2024-09-16T08:57:28.038 INFO:tasks.workunit.client.0.smithi031.stdout:8/679: dread d4/d7/d66/f70 [4194304,4194304] 0 2024-09-16T08:57:28.042 INFO:tasks.workunit.client.0.smithi031.stdout:9/885: mkdir d3/d5/d6a/d74/da8/dc3/dbc/d137 0 2024-09-16T08:57:28.042 INFO:tasks.workunit.client.0.smithi031.stdout:9/886: chown d3/d5/da/db/d21/d3b/dad/dea/fac 1902383785 1 2024-09-16T08:57:28.042 INFO:tasks.workunit.client.0.smithi031.stdout:9/887: read f2 [1546735,130895] 0 2024-09-16T08:57:28.042 INFO:tasks.workunit.client.0.smithi031.stdout:9/888: fsync d3/d5/da/d34/d4d/f105 0 2024-09-16T08:57:28.043 INFO:tasks.workunit.client.0.smithi031.stdout:5/733: creat d2/dc/d36/ff5 x:0 0 0 2024-09-16T08:57:28.044 INFO:tasks.workunit.client.0.smithi031.stdout:8/680: mkdir d4/d77/ddc 0 2024-09-16T08:57:28.048 INFO:tasks.workunit.client.0.smithi031.stdout:9/889: creat d3/d5/da/db/dcb/f138 x:0 0 0 2024-09-16T08:57:28.051 INFO:tasks.workunit.client.0.smithi031.stdout:5/734: unlink d2/dc/d25/da2/ld2 0 2024-09-16T08:57:28.051 INFO:tasks.workunit.client.0.smithi031.stdout:5/735: fdatasync d2/dc/f93 0 2024-09-16T08:57:28.054 INFO:tasks.workunit.client.0.smithi031.stdout:0/887: rename d6/d12/d97/d9e/d110 to d6/d12/d13/d11e 0 2024-09-16T08:57:28.055 INFO:tasks.workunit.client.0.smithi031.stdout:8/681: mknod d4/d2f/dc3/d37/d46/cdd 0 2024-09-16T08:57:28.058 INFO:tasks.workunit.client.0.smithi031.stdout:9/890: getdents d3/d5/d6a/d74/da8/dc3 0 2024-09-16T08:57:28.060 INFO:tasks.workunit.client.0.smithi031.stdout:0/888: write d6/d12/d47/f55 [1352276,64320] 0 2024-09-16T08:57:28.060 INFO:tasks.workunit.client.0.smithi031.stdout:8/682: dread d4/d2f/dc3/f3b [0,4194304] 0 2024-09-16T08:57:28.071 INFO:tasks.workunit.client.0.smithi031.stdout:8/683: dread d4/d2d/d6e/fab [0,4194304] 0 2024-09-16T08:57:28.085 INFO:tasks.workunit.client.0.smithi031.stdout:5/736: symlink d2/dc/d25/d68/lf6 0 2024-09-16T08:57:28.086 INFO:tasks.workunit.client.0.smithi031.stdout:9/891: mkdir d3/d5/d6a/d74/da8/d139 0 2024-09-16T08:57:28.093 INFO:tasks.workunit.client.0.smithi031.stdout:8/684: creat d4/d2d/fde x:0 0 0 2024-09-16T08:57:28.095 INFO:tasks.workunit.client.0.smithi031.stdout:5/737: rename d2/dc/d25/da2/cd4 to d2/dc/d25/dbf/cf7 0 2024-09-16T08:57:28.098 INFO:tasks.workunit.client.0.smithi031.stdout:9/892: mkdir d3/d5/da/d34/d3c/d5e/d62/d13a 0 2024-09-16T08:57:28.100 INFO:tasks.workunit.client.0.smithi031.stdout:0/889: getdents d6/d12/d13 0 2024-09-16T08:57:28.102 INFO:tasks.workunit.client.0.smithi031.stdout:8/685: mknod d4/d9e/d38/dc8/cdf 0 2024-09-16T08:57:28.102 INFO:tasks.workunit.client.0.smithi031.stdout:8/686: readlink d4/l13 0 2024-09-16T08:57:28.103 INFO:tasks.workunit.client.0.smithi031.stdout:0/890: dread d6/d12/d13/f21 [0,4194304] 0 2024-09-16T08:57:28.103 INFO:tasks.workunit.client.0.smithi031.stdout:0/891: fdatasync d6/d12/d13/d36/fd1 0 2024-09-16T08:57:28.106 INFO:tasks.workunit.client.0.smithi031.stdout:9/893: creat d3/d5/da/db/d21/d3b/dad/dd3/f13b x:0 0 0 2024-09-16T08:57:28.107 INFO:tasks.workunit.client.0.smithi031.stdout:0/892: creat d6/d12/d13/f11f x:0 0 0 2024-09-16T08:57:28.110 INFO:tasks.workunit.client.0.smithi031.stdout:9/894: link d3/d5/da/db/d21/d3b/dad/dea/fac d3/d5/dd6/de0/f13c 0 2024-09-16T08:57:28.112 INFO:tasks.workunit.client.0.smithi031.stdout:9/895: creat d3/d5/dd6/d116/f13d x:0 0 0 2024-09-16T08:57:28.114 INFO:tasks.workunit.client.0.smithi031.stdout:5/738: dread d2/f13 [0,4194304] 0 2024-09-16T08:57:28.114 INFO:tasks.workunit.client.0.smithi031.stdout:5/739: dread - d2/dc/f75 zero size 2024-09-16T08:57:28.115 INFO:tasks.workunit.client.0.smithi031.stdout:0/893: dread d6/f7 [0,4194304] 0 2024-09-16T08:57:28.116 INFO:tasks.workunit.client.0.smithi031.stdout:9/896: rename d3/f2a to d3/d5/da/db/dcb/f13e 0 2024-09-16T08:57:28.117 INFO:tasks.workunit.client.0.smithi031.stdout:5/740: mknod d2/dc/d1d/d97/cf8 0 2024-09-16T08:57:28.118 INFO:tasks.workunit.client.0.smithi031.stdout:0/894: mknod d6/d12/d13/dca/d11c/c120 0 2024-09-16T08:57:28.120 INFO:tasks.workunit.client.0.smithi031.stdout:9/897: unlink d3/d5/da/d34/d3c/d5e/d62/c128 0 2024-09-16T08:57:28.121 INFO:tasks.workunit.client.0.smithi031.stdout:0/895: unlink d6/d12/d47/d5b/l5c 0 2024-09-16T08:57:28.124 INFO:tasks.workunit.client.0.smithi031.stdout:8/687: dwrite d4/fb4 [0,4194304] 0 2024-09-16T08:57:28.126 INFO:tasks.workunit.client.0.smithi031.stdout:0/896: rmdir d6/d61/dce 39 2024-09-16T08:57:28.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/688: dread d4/d2f/dc3/f3b [0,4194304] 0 2024-09-16T08:57:28.127 INFO:tasks.workunit.client.0.smithi031.stdout:8/689: creat d4/d9e/d38/fe0 x:0 0 0 2024-09-16T08:57:28.130 INFO:tasks.workunit.client.0.smithi031.stdout:5/741: dread d2/dc/d91/fb4 [0,4194304] 0 2024-09-16T08:57:28.132 INFO:tasks.workunit.client.0.smithi031.stdout:0/897: unlink d6/d12/d47/ce7 0 2024-09-16T08:57:28.135 INFO:tasks.workunit.client.0.smithi031.stdout:8/690: mkdir d4/d7/d66/de1 0 2024-09-16T08:57:28.135 INFO:tasks.workunit.client.0.smithi031.stdout:5/742: creat d2/dc/d1d/d97/ff9 x:0 0 0 2024-09-16T08:57:28.136 INFO:tasks.workunit.client.0.smithi031.stdout:8/691: mknod d4/d77/dd3/ce2 0 2024-09-16T08:57:28.137 INFO:tasks.workunit.client.0.smithi031.stdout:5/743: rename d2/dc/d3d/d4c/l64 to d2/dc/d25/dbf/de4/lfa 0 2024-09-16T08:57:28.138 INFO:tasks.workunit.client.0.smithi031.stdout:8/692: creat d4/d2f/dc3/d37/fe3 x:0 0 0 2024-09-16T08:57:28.139 INFO:tasks.workunit.client.0.smithi031.stdout:5/744: symlink d2/dc/d91/lfb 0 2024-09-16T08:57:28.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/745: truncate d2/dc/d25/dbf/fc0 243684 0 2024-09-16T08:57:28.140 INFO:tasks.workunit.client.0.smithi031.stdout:5/746: fdatasync d2/dc/d25/d21/fa3 0 2024-09-16T08:57:28.140 INFO:tasks.workunit.client.0.smithi031.stdout:9/898: dwrite d3/d5/f6 [4194304,4194304] 0 2024-09-16T08:57:28.142 INFO:tasks.workunit.client.0.smithi031.stdout:5/747: rename d2/dc/d91/fd9 to d2/dc/d25/d21/d2e/ffc 0 2024-09-16T08:57:28.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/899: mknod d3/d5/d6a/d74/da8/dc3/dbc/d137/c13f 0 2024-09-16T08:57:28.146 INFO:tasks.workunit.client.0.smithi031.stdout:9/900: mkdir d3/d5/d140 0 2024-09-16T08:57:28.146 INFO:tasks.workunit.client.0.smithi031.stdout:9/901: chown d3/d5/da/d34/d3c/cfb 29528 1 2024-09-16T08:57:28.150 INFO:tasks.workunit.client.0.smithi031.stdout:9/902: rmdir d3/d5/dd6/d126 0 2024-09-16T08:57:28.156 INFO:tasks.workunit.client.0.smithi031.stdout:0/898: dwrite d6/d12/d47/f83 [0,4194304] 0 2024-09-16T08:57:28.156 INFO:tasks.workunit.client.0.smithi031.stdout:5/748: read d2/fa [270524,82083] 0 2024-09-16T08:57:28.156 INFO:tasks.workunit.client.0.smithi031.stdout:0/899: read d6/f33 [3636742,121865] 0 2024-09-16T08:57:28.159 INFO:tasks.workunit.client.0.smithi031.stdout:9/903: getdents d3/d85 0 2024-09-16T08:57:28.159 INFO:tasks.workunit.client.0.smithi031.stdout:9/904: creat d3/d5/dd6/de0/f141 x:0 0 0 2024-09-16T08:57:28.161 INFO:tasks.workunit.client.0.smithi031.stdout:5/749: creat d2/dc/ffd x:0 0 0 2024-09-16T08:57:28.163 INFO:tasks.workunit.client.0.smithi031.stdout:8/693: dwrite d4/d7/f51 [0,4194304] 0 2024-09-16T08:57:28.163 INFO:tasks.workunit.client.0.smithi031.stdout:8/694: write d4/f43 [4806357,84915] 0 2024-09-16T08:57:28.164 INFO:tasks.workunit.client.0.smithi031.stdout:0/900: rmdir d6/d12/d97/db3 39 2024-09-16T08:57:28.165 INFO:tasks.workunit.client.0.smithi031.stdout:0/901: chown d6/d12/d47/f55 20573098 1 2024-09-16T08:57:28.166 INFO:tasks.workunit.client.0.smithi031.stdout:5/750: truncate d2/dc/d25/d21/f45 2014854 0 2024-09-16T08:57:28.172 INFO:tasks.workunit.client.0.smithi031.stdout:8/695: link d4/d2f/dc3/d37/c7c d4/d7/d66/ce4 0 2024-09-16T08:57:28.173 INFO:tasks.workunit.client.0.smithi031.stdout:0/902: rmdir d6/d12/d13/dca/deb/d108 39 2024-09-16T08:57:28.174 INFO:tasks.workunit.client.0.smithi031.stdout:5/751: mkdir d2/dc/d1d/dbb/dfe 0 2024-09-16T08:57:28.175 INFO:tasks.workunit.client.0.smithi031.stdout:8/696: mkdir d4/d7/d66/de5 0 2024-09-16T08:57:28.176 INFO:tasks.workunit.client.0.smithi031.stdout:5/752: creat d2/dc/d25/d21/fff x:0 0 0 2024-09-16T08:57:28.178 INFO:tasks.workunit.client.0.smithi031.stdout:9/905: dwrite d3/d5/d6a/d74/da8/fb2 [4194304,4194304] 0 2024-09-16T08:57:28.179 INFO:tasks.workunit.client.0.smithi031.stdout:5/753: mkdir d2/d100 0 2024-09-16T08:57:28.181 INFO:tasks.workunit.client.0.smithi031.stdout:9/906: dread d3/d5/da/db/d21/fa3 [0,4194304] 0 2024-09-16T08:57:28.183 INFO:tasks.workunit.client.0.smithi031.stdout:5/754: rename d2/dc/d1d/d97/lb7 to d2/dc/d1d/d97/l101 0 2024-09-16T08:57:28.183 INFO:tasks.workunit.client.0.smithi031.stdout:5/755: stat d2/dc/f3b 0 2024-09-16T08:57:28.202 INFO:tasks.workunit.client.0.smithi031.stdout:8/697: dwrite d4/d2d/fde [0,4194304] 0 2024-09-16T08:57:28.202 INFO:tasks.workunit.client.0.smithi031.stdout:0/903: dwrite d6/d12/d13/ff3 [0,4194304] 0 2024-09-16T08:57:28.207 INFO:tasks.workunit.client.0.smithi031.stdout:0/904: read d6/d12/d47/d5b/f71 [947651,122376] 0 2024-09-16T08:57:28.209 INFO:tasks.workunit.client.0.smithi031.stdout:0/905: mknod d6/d12/d97/dc6/c121 0 2024-09-16T08:57:28.222 INFO:tasks.workunit.client.0.smithi031.stdout:0/906: dwrite d6/d12/d13/f3e [0,4194304] 0 2024-09-16T08:57:28.224 INFO:tasks.workunit.client.0.smithi031.stdout:0/907: mknod d6/d12/d47/d5b/c122 0 2024-09-16T08:57:28.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/908: write d6/fa [231187,40042] 0 2024-09-16T08:57:28.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/909: fsync d6/d12/d13/d36/fa9 0 2024-09-16T08:57:28.225 INFO:tasks.workunit.client.0.smithi031.stdout:0/910: getdents d6/d12/d13/dca/d11c 0 2024-09-16T08:57:28.227 INFO:tasks.workunit.client.0.smithi031.stdout:0/911: creat d6/d12/d97/db3/dba/f123 x:0 0 0 2024-09-16T08:57:28.228 INFO:tasks.workunit.client.0.smithi031.stdout:0/912: mknod d6/d12/d47/d5b/c124 0 2024-09-16T08:57:28.229 INFO:tasks.workunit.client.0.smithi031.stdout:0/913: creat d6/d12/d13/dca/f125 x:0 0 0 2024-09-16T08:57:28.344 INFO:tasks.workunit.client.0.smithi031.stdout:7/858: sync 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:6/884: sync 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:2/795: sync 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:4/895: sync 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:6/885: chown d0/dd/d74/f55 19093318 1 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:1/717: sync 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:4/896: chown d0/d102/dd5/dff/d100/f114 564 1 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:4/897: readlink d0/d2/l28 0 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:3/846: sync 2024-09-16T08:57:28.345 INFO:tasks.workunit.client.0.smithi031.stdout:3/847: stat d3/d14/d37/d1a/d1d/d3b/dc4/fca 0 2024-09-16T08:57:28.346 INFO:tasks.workunit.client.0.smithi031.stdout:7/859: creat d2/d8/d11/d5d/d7e/f119 x:0 0 0 2024-09-16T08:57:28.347 INFO:tasks.workunit.client.0.smithi031.stdout:6/886: rename d0/dd/d74/d19/d54/cbd to d0/d10/dc8/d10f/c11b 0 2024-09-16T08:57:28.348 INFO:tasks.workunit.client.0.smithi031.stdout:1/718: mkdir d2/d15/dff 0 2024-09-16T08:57:28.350 INFO:tasks.workunit.client.0.smithi031.stdout:4/898: mkdir d0/d5b/db8/dc8/d124 0 2024-09-16T08:57:28.351 INFO:tasks.workunit.client.0.smithi031.stdout:3/848: symlink d3/d14/d37/d1a/d1d/d3b/d8a/lfa 0 2024-09-16T08:57:28.353 INFO:tasks.workunit.client.0.smithi031.stdout:6/887: rename d0/dd/d35/c75 to d0/df1/c11c 0 2024-09-16T08:57:28.353 INFO:tasks.workunit.client.0.smithi031.stdout:6/888: readlink d0/dd/d74/d5e/d7c/d3f/l53 0 2024-09-16T08:57:28.354 INFO:tasks.workunit.client.0.smithi031.stdout:1/719: mknod d2/d15/d1f/de9/d1b/c100 0 2024-09-16T08:57:28.354 INFO:tasks.workunit.client.0.smithi031.stdout:3/849: mknod d3/d14/d37/d1a/d1d/d3b/d3f/d79/d7c/cfb 0 2024-09-16T08:57:28.355 INFO:tasks.workunit.client.0.smithi031.stdout:1/720: creat d2/d15/d1f/de9/d1b/de4/f101 x:0 0 0 2024-09-16T08:57:28.356 INFO:tasks.workunit.client.0.smithi031.stdout:3/850: rename d3/d14/d37/d1a/d1d/dac/fef to d3/ffc 0 2024-09-16T08:57:28.356 INFO:tasks.workunit.client.0.smithi031.stdout:3/851: readlink d3/d14/d37/d33/d97/lc6 0 2024-09-16T08:57:28.357 INFO:tasks.workunit.client.0.smithi031.stdout:3/852: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/lfd 0 2024-09-16T08:57:28.359 INFO:tasks.workunit.client.0.smithi031.stdout:3/853: rmdir d3/d14/d37/d1a/d1d/df0 0 2024-09-16T08:57:28.361 INFO:tasks.workunit.client.0.smithi031.stdout:3/854: dread d3/f2a [0,4194304] 0 2024-09-16T08:57:28.361 INFO:tasks.workunit.client.0.smithi031.stdout:3/855: creat d3/d14/d37/d1a/d1d/d3b/d3f/d49/ffe x:0 0 0 2024-09-16T08:57:28.361 INFO:tasks.workunit.client.0.smithi031.stdout:3/856: write d3/d14/d37/f10 [77556,76769] 0 2024-09-16T08:57:28.364 INFO:tasks.workunit.client.0.smithi031.stdout:2/796: dwrite d6/da/d5e/f70 [4194304,4194304] 0 2024-09-16T08:57:28.364 INFO:tasks.workunit.client.0.smithi031.stdout:2/797: write d6/da/f85 [597323,81728] 0 2024-09-16T08:57:28.377 INFO:tasks.workunit.client.0.smithi031.stdout:6/889: dread d0/dd/d74/d19/f2e [4194304,4194304] 0 2024-09-16T08:57:28.378 INFO:tasks.workunit.client.0.smithi031.stdout:6/890: creat d0/dd/d74/d5e/d7c/d3f/f11d x:0 0 0 2024-09-16T08:57:28.389 INFO:tasks.workunit.client.0.smithi031.stdout:7/860: dwrite d2/d8/d11/d5d/d7e/faf [0,4194304] 0 2024-09-16T08:57:28.390 INFO:tasks.workunit.client.0.smithi031.stdout:7/861: stat d2/d2d/l38 0 2024-09-16T08:57:28.391 INFO:tasks.workunit.client.0.smithi031.stdout:6/891: dread d0/dd/d74/d19/d9a/fc0 [0,4194304] 0 2024-09-16T08:57:28.391 INFO:tasks.workunit.client.0.smithi031.stdout:1/721: dwrite d2/d15/d1f/de9/fa2 [0,4194304] 0 2024-09-16T08:57:28.392 INFO:tasks.workunit.client.0.smithi031.stdout:6/892: symlink d0/d10/dc8/df6/l11e 0 2024-09-16T08:57:28.392 INFO:tasks.workunit.client.0.smithi031.stdout:6/893: truncate d0/dd/d74/d5e/f7b 1877145 0 2024-09-16T08:57:28.393 INFO:tasks.workunit.client.0.smithi031.stdout:6/894: creat d0/dd/d74/d5e/d7c/d3f/f11f x:0 0 0 2024-09-16T08:57:28.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/722: unlink d2/de/db5/cbd 0 2024-09-16T08:57:28.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/723: readlink d2/d15/d5d/la4 0 2024-09-16T08:57:28.394 INFO:tasks.workunit.client.0.smithi031.stdout:3/857: dwrite d3/d14/d37/d33/f4d [0,4194304] 0 2024-09-16T08:57:28.395 INFO:tasks.workunit.client.0.smithi031.stdout:6/895: rmdir d0/dd/d74/d5e/d110 39 2024-09-16T08:57:28.395 INFO:tasks.workunit.client.0.smithi031.stdout:6/896: fsync d0/dd/d74/d19/d54/fd6 0 2024-09-16T08:57:28.396 INFO:tasks.workunit.client.0.smithi031.stdout:4/899: dwrite d0/d2/fe [0,4194304] 0 2024-09-16T08:57:28.397 INFO:tasks.workunit.client.0.smithi031.stdout:2/798: dwrite d6/da/d4b/da0/fde [0,4194304] 0 2024-09-16T08:57:28.397 INFO:tasks.workunit.client.0.smithi031.stdout:6/897: mknod d0/dd/d74/d5e/d7c/d3f/c120 0 2024-09-16T08:57:28.400 INFO:tasks.workunit.client.0.smithi031.stdout:4/900: rename d0/d2/d16/d1e/db9/fd2 to d0/d3c/f125 0 2024-09-16T08:57:28.402 INFO:tasks.workunit.client.0.smithi031.stdout:2/799: rmdir d6/da/d37/deb/d106 0 2024-09-16T08:57:28.406 INFO:tasks.workunit.client.0.smithi031.stdout:6/898: rename d0/dd/d74/d5e/d110/f105 to d0/d10/dd2/de3/f121 0 2024-09-16T08:57:28.407 INFO:tasks.workunit.client.0.smithi031.stdout:2/800: creat d6/d20/d51/f109 x:0 0 0 2024-09-16T08:57:28.409 INFO:tasks.workunit.client.0.smithi031.stdout:4/901: rename d0/db5/l112 to d0/d8e/l126 0 2024-09-16T08:57:28.409 INFO:tasks.workunit.client.0.smithi031.stdout:4/902: chown d0/d2/f7e 2 1 2024-09-16T08:57:28.410 INFO:tasks.workunit.client.0.smithi031.stdout:6/899: symlink d0/dd/d74/d67/dbb/d100/l122 0 2024-09-16T08:57:28.414 INFO:tasks.workunit.client.0.smithi031.stdout:4/903: symlink d0/d3c/d6b/l127 0 2024-09-16T08:57:28.416 INFO:tasks.workunit.client.0.smithi031.stdout:6/900: rename d0/d10/f60 to d0/dd/d74/d5e/d7c/d3f/d65/f123 0 2024-09-16T08:57:28.421 INFO:tasks.workunit.client.0.smithi031.stdout:1/724: dwrite d2/d15/d1f/de9/d52/fca [0,4194304] 0 2024-09-16T08:57:28.421 INFO:tasks.workunit.client.0.smithi031.stdout:3/858: dwrite d3/d14/fb6 [0,4194304] 0 2024-09-16T08:57:28.421 INFO:tasks.workunit.client.0.smithi031.stdout:3/859: stat d3/d14/d37/f10 0 2024-09-16T08:57:28.425 INFO:tasks.workunit.client.0.smithi031.stdout:4/904: rename d0/d2/d16/d2b/l40 to d0/d2/d16/d2b/l128 0 2024-09-16T08:57:28.426 INFO:tasks.workunit.client.0.smithi031.stdout:6/901: symlink d0/dd/d74/d19/dcd/l124 0 2024-09-16T08:57:28.426 INFO:tasks.workunit.client.0.smithi031.stdout:6/902: fdatasync d0/dd/d74/d19/d54/fd6 0 2024-09-16T08:57:28.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/860: dread d3/d14/d37/d1a/d1d/d3b/d3f/f74 [0,4194304] 0 2024-09-16T08:57:28.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/861: creat d3/d14/d37/d1a/d1d/d3b/d60/fff x:0 0 0 2024-09-16T08:57:28.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/862: fsync d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d7d/ff9 0 2024-09-16T08:57:28.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/863: creat d3/d14/d37/f100 x:0 0 0 2024-09-16T08:57:28.435 INFO:tasks.workunit.client.0.smithi031.stdout:4/905: link d0/d2/d16/d1e/db9/ff9 d0/d2/d16/d2b/f129 0 2024-09-16T08:57:28.436 INFO:tasks.workunit.client.0.smithi031.stdout:4/906: read d0/d2/f3d [3912661,22446] 0 2024-09-16T08:57:28.441 INFO:tasks.workunit.client.0.smithi031.stdout:2/801: rmdir d6/d20 39 2024-09-16T08:57:28.442 INFO:tasks.workunit.client.0.smithi031.stdout:1/725: dwrite d2/d15/d32/f88 [0,4194304] 0 2024-09-16T08:57:28.442 INFO:tasks.workunit.client.0.smithi031.stdout:1/726: dread - d2/f96 zero size 2024-09-16T08:57:28.444 INFO:tasks.workunit.client.0.smithi031.stdout:6/903: rmdir d0/dd/d74/d67/dbb 39 2024-09-16T08:57:28.444 INFO:tasks.workunit.client.0.smithi031.stdout:6/904: fsync d0/dd/d74/d5e/d7c/d3f/d65/fef 0 2024-09-16T08:57:28.445 INFO:tasks.workunit.client.0.smithi031.stdout:2/802: symlink d6/da/d4b/da0/l10a 0 2024-09-16T08:57:28.448 INFO:tasks.workunit.client.0.smithi031.stdout:1/727: rename f0 to d2/d15/d38/f102 0 2024-09-16T08:57:28.451 INFO:tasks.workunit.client.0.smithi031.stdout:3/864: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/f74 [0,4194304] 0 2024-09-16T08:57:28.452 INFO:tasks.workunit.client.0.smithi031.stdout:6/905: creat d0/dd/d74/da6/f125 x:0 0 0 2024-09-16T08:57:28.453 INFO:tasks.workunit.client.0.smithi031.stdout:2/803: symlink d6/da/d37/l10b 0 2024-09-16T08:57:28.459 INFO:tasks.workunit.client.0.smithi031.stdout:3/865: unlink d3/d14/d37/f100 0 2024-09-16T08:57:28.460 INFO:tasks.workunit.client.0.smithi031.stdout:6/906: creat d0/dd/d74/d67/dbb/d100/f126 x:0 0 0 2024-09-16T08:57:28.460 INFO:tasks.workunit.client.0.smithi031.stdout:6/907: write d0/dd/d74/da6/f125 [504652,74176] 0 2024-09-16T08:57:28.460 INFO:tasks.workunit.client.0.smithi031.stdout:6/908: readlink d0/dd/d74/d5e/d110/le6 0 2024-09-16T08:57:28.460 INFO:tasks.workunit.client.0.smithi031.stdout:2/804: truncate d6/f16 603258 0 2024-09-16T08:57:28.461 INFO:tasks.workunit.client.0.smithi031.stdout:2/805: write d6/d2e/d40/d6c/d2c/d53/d60/f35 [2293145,37312] 0 2024-09-16T08:57:28.462 INFO:tasks.workunit.client.0.smithi031.stdout:3/866: symlink d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/dce/l101 0 2024-09-16T08:57:28.462 INFO:tasks.workunit.client.0.smithi031.stdout:3/867: write d3/d14/d37/d33/fd9 [419414,105923] 0 2024-09-16T08:57:28.463 INFO:tasks.workunit.client.0.smithi031.stdout:6/909: rmdir d0/dd/d74/d67/dbb/d100/d114 0 2024-09-16T08:57:28.467 INFO:tasks.workunit.client.0.smithi031.stdout:3/868: creat d3/d14/d37/d1a/d1d/d3b/d3f/d79/f102 x:0 0 0 2024-09-16T08:57:28.467 INFO:tasks.workunit.client.0.smithi031.stdout:1/728: dwrite d2/d15/d38/f102 [4194304,4194304] 0 2024-09-16T08:57:28.467 INFO:tasks.workunit.client.0.smithi031.stdout:1/729: creat d2/d15/d1f/de9/d52/f103 x:0 0 0 2024-09-16T08:57:28.467 INFO:tasks.workunit.client.0.smithi031.stdout:6/910: creat d0/d10/f127 x:0 0 0 2024-09-16T08:57:28.468 INFO:tasks.workunit.client.0.smithi031.stdout:3/869: mknod d3/d14/d37/d1a/d1d/d3b/d94/c103 0 2024-09-16T08:57:28.468 INFO:tasks.workunit.client.0.smithi031.stdout:3/870: stat d3/d14/f8c 0 2024-09-16T08:57:28.470 INFO:tasks.workunit.client.0.smithi031.stdout:1/730: creat d2/d15/d1f/de9/f104 x:0 0 0 2024-09-16T08:57:28.471 INFO:tasks.workunit.client.0.smithi031.stdout:1/731: dread d2/f53 [0,4194304] 0 2024-09-16T08:57:28.473 INFO:tasks.workunit.client.0.smithi031.stdout:3/871: symlink d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d7d/l104 0 2024-09-16T08:57:28.473 INFO:tasks.workunit.client.0.smithi031.stdout:3/872: write d3/d14/d37/d33/fd9 [1332938,81478] 0 2024-09-16T08:57:28.473 INFO:tasks.workunit.client.0.smithi031.stdout:3/873: write d3/f4 [4592907,113992] 0 2024-09-16T08:57:28.473 INFO:tasks.workunit.client.0.smithi031.stdout:3/874: fdatasync d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/fb7 0 2024-09-16T08:57:28.473 INFO:tasks.workunit.client.0.smithi031.stdout:3/875: write d3/d14/d37/d47/fcd [1506240,40528] 0 2024-09-16T08:57:28.473 INFO:tasks.workunit.client.0.smithi031.stdout:3/876: creat d3/d14/d37/d1a/d1d/d6a/f105 x:0 0 0 2024-09-16T08:57:28.480 INFO:tasks.workunit.client.0.smithi031.stdout:1/732: rename d2/dae to d2/d15/d1f/de9/d105 0 2024-09-16T08:57:28.480 INFO:tasks.workunit.client.0.smithi031.stdout:1/733: write d2/f96 [686192,79347] 0 2024-09-16T08:57:28.491 INFO:tasks.workunit.client.0.smithi031.stdout:1/734: write d2/d15/d1f/de9/d52/fca [843603,114017] 0 2024-09-16T08:57:28.493 INFO:tasks.workunit.client.0.smithi031.stdout:1/735: symlink d2/d15/d5d/db4/l106 0 2024-09-16T08:57:28.501 INFO:tasks.workunit.client.0.smithi031.stdout:1/736: write d2/de/f79 [452652,93029] 0 2024-09-16T08:57:28.502 INFO:tasks.workunit.client.0.smithi031.stdout:1/737: mknod d2/d15/d38/c107 0 2024-09-16T08:57:28.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:28 smithi097 ceph-mon[35478]: Reconfiguring ceph-exporter.smithi031 (dependencies changed)... 2024-09-16T08:57:28.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:28 smithi097 ceph-mon[35478]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-09-16T08:57:28.515 INFO:tasks.workunit.client.0.smithi031.stdout:1/738: dwrite d2/d15/d1f/de9/d1b/fdc [0,4194304] 0 2024-09-16T08:57:28.521 INFO:tasks.workunit.client.0.smithi031.stdout:1/739: creat d2/d15/dff/f108 x:0 0 0 2024-09-16T08:57:28.531 INFO:tasks.workunit.client.0.smithi031.stdout:1/740: dread d2/d15/d1f/de9/d52/fca [0,4194304] 0 2024-09-16T08:57:28.531 INFO:tasks.workunit.client.0.smithi031.stdout:1/741: dread - d2/d15/d5d/db4/da3/dcb/fee zero size 2024-09-16T08:57:28.536 INFO:tasks.workunit.client.0.smithi031.stdout:1/742: symlink d2/d15/d1f/d43/l109 0 2024-09-16T08:57:28.579 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:28 smithi031 ceph-mon[27844]: Reconfiguring ceph-exporter.smithi031 (dependencies changed)... 2024-09-16T08:57:28.579 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:28 smithi031 ceph-mon[27844]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-09-16T08:57:28.696 INFO:tasks.workunit.client.0.smithi031.stdout:9/907: sync 2024-09-16T08:57:28.697 INFO:tasks.workunit.client.0.smithi031.stdout:9/908: dread - d3/d5/da/d34/f131 zero size 2024-09-16T08:57:28.698 INFO:tasks.workunit.client.0.smithi031.stdout:9/909: dread d3/d5/f2d [0,4194304] 0 2024-09-16T08:57:28.714 INFO:tasks.workunit.client.0.smithi031.stdout:9/910: dwrite d3/d5/da/d34/d3c/d5e/d9c/d95/fae [0,4194304] 0 2024-09-16T08:57:28.715 INFO:tasks.workunit.client.0.smithi031.stdout:9/911: write d3/d5/da/f12a [88856,9484] 0 2024-09-16T08:57:28.717 INFO:tasks.workunit.client.0.smithi031.stdout:9/912: creat d3/d5/da/d34/d3c/d5e/de1/d115/f142 x:0 0 0 2024-09-16T08:57:29.109 INFO:tasks.workunit.client.0.smithi031.stdout:5/756: sync 2024-09-16T08:57:29.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/698: sync 2024-09-16T08:57:29.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/699: dread - d4/d2f/dc3/d37/d46/fcf zero size 2024-09-16T08:57:29.122 INFO:tasks.workunit.client.0.smithi031.stdout:8/700: write d4/d2d/d6e/fab [3598488,7527] 0 2024-09-16T08:57:29.254 INFO:tasks.workunit.client.0.smithi031.stdout:5/757: symlink d2/dc/d25/da2/l102 0 2024-09-16T08:57:29.255 INFO:tasks.workunit.client.0.smithi031.stdout:8/701: mknod d4/d9e/d5e/ce6 0 2024-09-16T08:57:29.255 INFO:tasks.workunit.client.0.smithi031.stdout:8/702: getdents d4/d5b/d7b 0 2024-09-16T08:57:29.258 INFO:tasks.workunit.client.0.smithi031.stdout:5/758: symlink d2/dc/d91/dee/l103 0 2024-09-16T08:57:29.259 INFO:tasks.workunit.client.0.smithi031.stdout:5/759: symlink d2/dc/d25/d21/d2e/l104 0 2024-09-16T08:57:29.259 INFO:tasks.workunit.client.0.smithi031.stdout:5/760: fsync d2/dc/d25/d21/d2e/f5c 0 2024-09-16T08:57:29.260 INFO:tasks.workunit.client.0.smithi031.stdout:5/761: truncate d2/dc/d25/d21/f45 930765 0 2024-09-16T08:57:29.264 INFO:tasks.workunit.client.0.smithi031.stdout:5/762: creat d2/f105 x:0 0 0 2024-09-16T08:57:29.264 INFO:tasks.workunit.client.0.smithi031.stdout:5/763: write d2/dc/d25/d68/f6b [2042291,8684] 0 2024-09-16T08:57:29.264 INFO:tasks.workunit.client.0.smithi031.stdout:8/703: write d4/d7/f51 [845560,35345] 0 2024-09-16T08:57:29.268 INFO:tasks.workunit.client.0.smithi031.stdout:5/764: unlink d2/dc/f93 0 2024-09-16T08:57:29.270 INFO:tasks.workunit.client.0.smithi031.stdout:5/765: creat d2/dc/df3/f106 x:0 0 0 2024-09-16T08:57:29.273 INFO:tasks.workunit.client.0.smithi031.stdout:8/704: rmdir d4/d9e/d38 39 2024-09-16T08:57:29.273 INFO:tasks.workunit.client.0.smithi031.stdout:8/705: write d4/d2d/d6e/fa4 [159826,118568] 0 2024-09-16T08:57:29.276 INFO:tasks.workunit.client.0.smithi031.stdout:8/706: creat d4/d5b/d8d/dcd/d50/fe7 x:0 0 0 2024-09-16T08:57:29.276 INFO:tasks.workunit.client.0.smithi031.stdout:8/707: chown d4/d7/d99/fb6 1120518 1 2024-09-16T08:57:29.282 INFO:tasks.workunit.client.0.smithi031.stdout:5/766: dread d2/dc/d25/d68/f6b [0,4194304] 0 2024-09-16T08:57:29.292 INFO:tasks.workunit.client.0.smithi031.stdout:7/862: sync 2024-09-16T08:57:29.292 INFO:tasks.workunit.client.0.smithi031.stdout:0/914: sync 2024-09-16T08:57:29.292 INFO:tasks.workunit.client.0.smithi031.stdout:0/915: readlink d6/d61/l72 0 2024-09-16T08:57:29.292 INFO:tasks.workunit.client.0.smithi031.stdout:0/916: truncate d6/d12/d13/d3c/f45 2938933 0 2024-09-16T08:57:29.292 INFO:tasks.workunit.client.0.smithi031.stdout:4/907: sync 2024-09-16T08:57:29.294 INFO:tasks.workunit.client.0.smithi031.stdout:7/863: rename d2/d8/d11/d81/cc1 to d2/d8/c11a 0 2024-09-16T08:57:29.296 INFO:tasks.workunit.client.0.smithi031.stdout:4/908: unlink d0/d2/f10 0 2024-09-16T08:57:29.297 INFO:tasks.workunit.client.0.smithi031.stdout:7/864: readlink d2/l3b 0 2024-09-16T08:57:29.300 INFO:tasks.workunit.client.0.smithi031.stdout:4/909: mknod d0/d3c/d6b/d9c/c12a 0 2024-09-16T08:57:29.300 INFO:tasks.workunit.client.0.smithi031.stdout:4/910: dread - d0/d3c/d6b/d9c/fee zero size 2024-09-16T08:57:29.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/865: dread d2/d8/f3f [0,4194304] 0 2024-09-16T08:57:29.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/866: creat d2/d8/d11/d5d/d6e/f11b x:0 0 0 2024-09-16T08:57:29.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/867: fdatasync d2/d8/d86/da1/da9/f108 0 2024-09-16T08:57:29.308 INFO:tasks.workunit.client.0.smithi031.stdout:7/868: rename d2/d8/d11/d66/d9a/d9c/db3 to d2/d2d/d68/d113/d11c 0 2024-09-16T08:57:29.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/917: dwrite d6/d12/d47/d5b/fb2 [0,4194304] 0 2024-09-16T08:57:29.319 INFO:tasks.workunit.client.0.smithi031.stdout:7/869: dread d2/d2d/d6d/fb4 [0,4194304] 0 2024-09-16T08:57:29.319 INFO:tasks.workunit.client.0.smithi031.stdout:0/918: readlink d6/lf1 0 2024-09-16T08:57:29.319 INFO:tasks.workunit.client.0.smithi031.stdout:7/870: dread - d2/d8/d11/d5d/fec zero size 2024-09-16T08:57:29.336 INFO:tasks.workunit.client.0.smithi031.stdout:4/911: dwrite d0/d2/d16/d1e/f3b [0,4194304] 0 2024-09-16T08:57:29.336 INFO:tasks.workunit.client.0.smithi031.stdout:4/912: fdatasync d0/d2/d16/d1e/f34 0 2024-09-16T08:57:29.337 INFO:tasks.workunit.client.0.smithi031.stdout:0/919: dread d6/d12/f77 [0,4194304] 0 2024-09-16T08:57:29.352 INFO:tasks.workunit.client.0.smithi031.stdout:7/871: dwrite d2/d8/fe5 [0,4194304] 0 2024-09-16T08:57:29.353 INFO:tasks.workunit.client.0.smithi031.stdout:7/872: read - d2/d2d/d6d/fe6 zero size 2024-09-16T08:57:29.353 INFO:tasks.workunit.client.0.smithi031.stdout:7/873: write d2/d8/d11/d66/d9a/d9c/ffa [869389,120273] 0 2024-09-16T08:57:29.356 INFO:tasks.workunit.client.0.smithi031.stdout:7/874: creat d2/d8/d11/d5d/d7e/f11d x:0 0 0 2024-09-16T08:57:29.358 INFO:tasks.workunit.client.0.smithi031.stdout:7/875: symlink d2/d8/d86/dd4/df1/l11e 0 2024-09-16T08:57:29.358 INFO:tasks.workunit.client.0.smithi031.stdout:4/913: dwrite d0/d2/f13 [0,4194304] 0 2024-09-16T08:57:29.359 INFO:tasks.workunit.client.0.smithi031.stdout:7/876: truncate d2/d8/d11/d5d/d6e/f95 3673029 0 2024-09-16T08:57:29.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/914: rename d0/d3c/f62 to d0/d109/d106/d55/f12b 0 2024-09-16T08:57:29.362 INFO:tasks.workunit.client.0.smithi031.stdout:4/915: truncate d0/d102/fd3 154795 0 2024-09-16T08:57:29.363 INFO:tasks.workunit.client.0.smithi031.stdout:4/916: truncate d0/f18 2087329 0 2024-09-16T08:57:29.365 INFO:tasks.workunit.client.0.smithi031.stdout:4/917: creat d0/d2/d16/d2b/daf/f12c x:0 0 0 2024-09-16T08:57:29.365 INFO:tasks.workunit.client.0.smithi031.stdout:4/918: write d0/d8e/fc2 [390648,124047] 0 2024-09-16T08:57:29.365 INFO:tasks.workunit.client.0.smithi031.stdout:0/920: dwrite d6/d12/d13/d36/f8a [0,4194304] 0 2024-09-16T08:57:29.365 INFO:tasks.workunit.client.0.smithi031.stdout:0/921: stat d6/d12/d13/d3c/f70 0 2024-09-16T08:57:29.368 INFO:tasks.workunit.client.0.smithi031.stdout:4/919: creat d0/db5/f12d x:0 0 0 2024-09-16T08:57:29.377 INFO:tasks.workunit.client.0.smithi031.stdout:0/922: mknod d6/d12/d13/dde/c126 0 2024-09-16T08:57:29.379 INFO:tasks.workunit.client.0.smithi031.stdout:4/920: link d0/d109/d106/c68 d0/db5/c12e 0 2024-09-16T08:57:29.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/921: creat d0/d102/d84/d86/d116/f12f x:0 0 0 2024-09-16T08:57:29.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/922: truncate d0/d2/d16/f4e 4546017 0 2024-09-16T08:57:29.382 INFO:tasks.workunit.client.0.smithi031.stdout:0/923: dread d6/d12/d13/f6f [0,4194304] 0 2024-09-16T08:57:29.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/923: rename d0/d8e/f77 to d0/d5b/f130 0 2024-09-16T08:57:29.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/924: readlink d0/d102/d84/d86/d9b/dcf/l122 0 2024-09-16T08:57:29.385 INFO:tasks.workunit.client.0.smithi031.stdout:4/925: chown d0/d102/d8d/fc1 201102 1 2024-09-16T08:57:29.388 INFO:tasks.workunit.client.0.smithi031.stdout:4/926: getdents d0/d2/d16/d1e/db9 0 2024-09-16T08:57:29.391 INFO:tasks.workunit.client.0.smithi031.stdout:4/927: mknod d0/d8e/d75/c131 0 2024-09-16T08:57:29.404 INFO:tasks.workunit.client.0.smithi031.stdout:4/928: write d0/d102/d8d/fc1 [3254213,39651] 0 2024-09-16T08:57:29.407 INFO:tasks.workunit.client.0.smithi031.stdout:4/929: rename d0/d2/d16/d1e/dbe/cea to d0/d2/d16/c132 0 2024-09-16T08:57:29.411 INFO:tasks.workunit.client.0.smithi031.stdout:4/930: creat d0/d5b/f133 x:0 0 0 2024-09-16T08:57:29.414 INFO:tasks.workunit.client.0.smithi031.stdout:2/806: sync 2024-09-16T08:57:29.419 INFO:tasks.workunit.client.0.smithi031.stdout:2/807: symlink d6/l10c 0 2024-09-16T08:57:29.421 INFO:tasks.workunit.client.0.smithi031.stdout:2/808: creat d6/d2e/d40/d6c/d2c/d53/f10d x:0 0 0 2024-09-16T08:57:29.421 INFO:tasks.workunit.client.0.smithi031.stdout:2/809: write d6/da/d4b/fc8 [1038251,18618] 0 2024-09-16T08:57:29.438 INFO:tasks.workunit.client.0.smithi031.stdout:2/810: dwrite d6/da/d37/f63 [0,4194304] 0 2024-09-16T08:57:29.439 INFO:tasks.workunit.client.0.smithi031.stdout:2/811: link d6/da/d37/l6b d6/d2e/d40/l10e 0 2024-09-16T08:57:29.440 INFO:tasks.workunit.client.0.smithi031.stdout:2/812: getdents d6/da/d5e/d72 0 2024-09-16T08:57:29.441 INFO:tasks.workunit.client.0.smithi031.stdout:2/813: creat d6/d2e/d40/d6c/f10f x:0 0 0 2024-09-16T08:57:29.442 INFO:tasks.workunit.client.0.smithi031.stdout:2/814: mkdir d6/da/d4b/da0/d110 0 2024-09-16T08:57:29.443 INFO:tasks.workunit.client.0.smithi031.stdout:2/815: creat d6/d20/da1/dcf/dda/f111 x:0 0 0 2024-09-16T08:57:29.443 INFO:tasks.workunit.client.0.smithi031.stdout:2/816: chown d6/d2e/d40/d6c/f5a 875000 1 2024-09-16T08:57:29.443 INFO:tasks.workunit.client.0.smithi031.stdout:2/817: dread - d6/d20/da1/dcf/dda/de3/fe8 zero size 2024-09-16T08:57:29.444 INFO:tasks.workunit.client.0.smithi031.stdout:2/818: rename d6/da/d4b/cb2 to d6/d2e/d40/d6c/c112 0 2024-09-16T08:57:29.445 INFO:tasks.workunit.client.0.smithi031.stdout:2/819: mknod d6/da/d4b/d4c/c113 0 2024-09-16T08:57:29.445 INFO:tasks.workunit.client.0.smithi031.stdout:2/820: chown d6/d20/da1/dcf/dda/f111 4 1 2024-09-16T08:57:29.445 INFO:tasks.workunit.client.0.smithi031.stdout:2/821: chown d6/da/d4b/d4c/l9c 15228 1 2024-09-16T08:57:29.445 INFO:tasks.workunit.client.0.smithi031.stdout:2/822: dread - d6/d20/fac zero size 2024-09-16T08:57:29.446 INFO:tasks.workunit.client.0.smithi031.stdout:2/823: rename d6/da/d4b/d4c/d73/ca9 to d6/d2e/d40/d6c/c114 0 2024-09-16T08:57:29.461 INFO:tasks.workunit.client.0.smithi031.stdout:2/824: dwrite d6/da/d37/f63 [0,4194304] 0 2024-09-16T08:57:29.462 INFO:tasks.workunit.client.0.smithi031.stdout:2/825: truncate d6/d20/f32 431178 0 2024-09-16T08:57:29.465 INFO:tasks.workunit.client.0.smithi031.stdout:2/826: dread d6/f16 [0,4194304] 0 2024-09-16T08:57:29.465 INFO:tasks.workunit.client.0.smithi031.stdout:2/827: chown d6/d2e/d40/fc5 459 1 2024-09-16T08:57:29.466 INFO:tasks.workunit.client.0.smithi031.stdout:2/828: rmdir d6/d2e/d40/d6c/d2c/d53/d60 39 2024-09-16T08:57:29.480 INFO:tasks.workunit.client.0.smithi031.stdout:2/829: dwrite d6/d20/d51/f109 [0,4194304] 0 2024-09-16T08:57:29.480 INFO:tasks.workunit.client.0.smithi031.stdout:2/830: chown d6/d20/da1/dcf/dda/f111 45180330 1 2024-09-16T08:57:29.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/831: unlink d6/l10c 0 2024-09-16T08:57:29.482 INFO:tasks.workunit.client.0.smithi031.stdout:2/832: symlink d6/da/d5e/d72/l115 0 2024-09-16T08:57:29.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:29 smithi097 ceph-mon[35478]: pgmap v16: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 207 MiB/s rd, 265 MiB/s wr, 383 op/s 2024-09-16T08:57:29.514 INFO:tasks.workunit.client.0.smithi031.stdout:4/931: dwrite d0/d2/f1a [0,4194304] 0 2024-09-16T08:57:29.514 INFO:tasks.workunit.client.0.smithi031.stdout:4/932: read - d0/d2/d16/d2b/d10e/f119 zero size 2024-09-16T08:57:29.514 INFO:tasks.workunit.client.0.smithi031.stdout:2/833: dwrite d6/da/d4b/d4c/d68/fdf [0,4194304] 0 2024-09-16T08:57:29.516 INFO:tasks.workunit.client.0.smithi031.stdout:4/933: mknod d0/d5b/db8/dc8/c134 0 2024-09-16T08:57:29.565 INFO:tasks.workunit.client.0.smithi031.stdout:1/743: sync 2024-09-16T08:57:29.565 INFO:tasks.workunit.client.0.smithi031.stdout:6/911: sync 2024-09-16T08:57:29.565 INFO:tasks.workunit.client.0.smithi031.stdout:3/877: sync 2024-09-16T08:57:29.565 INFO:tasks.workunit.client.0.smithi031.stdout:6/912: getdents d0/dd/d74/d19/df9 0 2024-09-16T08:57:29.565 INFO:tasks.workunit.client.0.smithi031.stdout:9/913: sync 2024-09-16T08:57:29.566 INFO:tasks.workunit.client.0.smithi031.stdout:3/878: rmdir d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/dce 39 2024-09-16T08:57:29.567 INFO:tasks.workunit.client.0.smithi031.stdout:1/744: symlink d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea/l10a 0 2024-09-16T08:57:29.569 INFO:tasks.workunit.client.0.smithi031.stdout:3/879: truncate d3/d14/d37/d33/f7a 1353455 0 2024-09-16T08:57:29.571 INFO:tasks.workunit.client.0.smithi031.stdout:9/914: symlink d3/d5/da/d34/d3c/d5e/d9c/dcd/l143 0 2024-09-16T08:57:29.572 INFO:tasks.workunit.client.0.smithi031.stdout:3/880: symlink d3/d14/d37/d1a/d1d/dac/l106 0 2024-09-16T08:57:29.572 INFO:tasks.workunit.client.0.smithi031.stdout:3/881: truncate d3/d14/d37/d1a/d1d/d9a/dde/d7f/f9e 840432 0 2024-09-16T08:57:29.574 INFO:tasks.workunit.client.0.smithi031.stdout:6/913: dread d0/dd/d74/f7e [0,4194304] 0 2024-09-16T08:57:29.574 INFO:tasks.workunit.client.0.smithi031.stdout:9/915: rename d3/d5/da/db/d21/f41 to d3/d5/da/db/d21/d3b/dad/dd3/f144 0 2024-09-16T08:57:29.574 INFO:tasks.workunit.client.0.smithi031.stdout:9/916: fsync d3/d5/da/d34/d3c/d5e/d62/d65/f94 0 2024-09-16T08:57:29.576 INFO:tasks.workunit.client.0.smithi031.stdout:1/745: dread d2/d15/d1f/de9/fa2 [0,4194304] 0 2024-09-16T08:57:29.578 INFO:tasks.workunit.client.0.smithi031.stdout:9/917: mknod d3/d5/dd6/c145 0 2024-09-16T08:57:29.582 INFO:tasks.workunit.client.0.smithi031.stdout:1/746: symlink d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/l10b 0 2024-09-16T08:57:29.582 INFO:tasks.workunit.client.0.smithi031.stdout:1/747: fdatasync d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea/f7e 0 2024-09-16T08:57:29.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/914: dread d0/dd/f3e [0,4194304] 0 2024-09-16T08:57:29.582 INFO:tasks.workunit.client.0.smithi031.stdout:6/915: chown d0/dd/d74/d5e 303574 1 2024-09-16T08:57:29.582 INFO:tasks.workunit.client.0.smithi031.stdout:9/918: dread d3/d5/da/f33 [0,4194304] 0 2024-09-16T08:57:29.584 INFO:tasks.workunit.client.0.smithi031.stdout:1/748: unlink d2/d15/d1f/de9/d52/f63 0 2024-09-16T08:57:29.584 INFO:tasks.workunit.client.0.smithi031.stdout:1/749: chown d2/d15/d1f/de9/d67/lf3 1128 1 2024-09-16T08:57:29.589 INFO:tasks.workunit.client.0.smithi031.stdout:3/882: dwrite d3/d14/d37/d1a/d1d/d3b/d8a/feb [0,4194304] 0 2024-09-16T08:57:29.589 INFO:tasks.workunit.client.0.smithi031.stdout:9/919: mkdir d3/d85/d146 0 2024-09-16T08:57:29.599 INFO:tasks.workunit.client.0.smithi031.stdout:3/883: write d3/d14/d37/d1a/f9b [3086074,26728] 0 2024-09-16T08:57:29.599 INFO:tasks.workunit.client.0.smithi031.stdout:3/884: chown d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/le1 16 1 2024-09-16T08:57:29.604 INFO:tasks.workunit.client.0.smithi031.stdout:3/885: creat d3/d14/d37/d1a/d1d/d3b/d3f/f107 x:0 0 0 2024-09-16T08:57:29.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/916: dwrite d0/dd/d74/d19/d9a/fc0 [4194304,4194304] 0 2024-09-16T08:57:29.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/917: readlink d0/dd/d74/d19/d9a/lc1 0 2024-09-16T08:57:29.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/918: dread - d0/dd/d74/d67/f118 zero size 2024-09-16T08:57:29.604 INFO:tasks.workunit.client.0.smithi031.stdout:6/919: write d0/dd/d74/d67/dbb/d100/f126 [498247,14530] 0 2024-09-16T08:57:29.605 INFO:tasks.workunit.client.0.smithi031.stdout:1/750: dwrite d2/d15/d1f/de9/d1b/d5c/d6b/f7b [0,4194304] 0 2024-09-16T08:57:29.605 INFO:tasks.workunit.client.0.smithi031.stdout:1/751: fsync d2/d15/d1f/d43/fb7 0 2024-09-16T08:57:29.611 INFO:tasks.workunit.client.0.smithi031.stdout:6/920: dread d0/dd/d74/d5e/f7b [0,4194304] 0 2024-09-16T08:57:29.612 INFO:tasks.workunit.client.0.smithi031.stdout:1/752: truncate d2/f17 3187064 0 2024-09-16T08:57:29.616 INFO:tasks.workunit.client.0.smithi031.stdout:6/921: mknod d0/dd/d74/d5e/d7c/d3f/d92/c128 0 2024-09-16T08:57:29.620 INFO:tasks.workunit.client.0.smithi031.stdout:6/922: creat d0/dd/d35/d6c/f129 x:0 0 0 2024-09-16T08:57:29.620 INFO:tasks.workunit.client.0.smithi031.stdout:1/753: dread d2/d15/d32/d87/fa0 [0,4194304] 0 2024-09-16T08:57:29.621 INFO:tasks.workunit.client.0.smithi031.stdout:6/923: mkdir d0/dd/d74/d19/d54/d12a 0 2024-09-16T08:57:29.621 INFO:tasks.workunit.client.0.smithi031.stdout:6/924: dread - d0/d10/dc8/fec zero size 2024-09-16T08:57:29.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/886: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/d79/f102 [0,4194304] 0 2024-09-16T08:57:29.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/887: creat d3/d14/d85/f108 x:0 0 0 2024-09-16T08:57:29.622 INFO:tasks.workunit.client.0.smithi031.stdout:3/888: write d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/fe5 [726556,101636] 0 2024-09-16T08:57:29.622 INFO:tasks.workunit.client.0.smithi031.stdout:6/925: mkdir d0/dd/d35/d6c/d12b 0 2024-09-16T08:57:29.622 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:29 smithi031 ceph-mon[27844]: pgmap v16: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail; 207 MiB/s rd, 265 MiB/s wr, 383 op/s 2024-09-16T08:57:29.635 INFO:tasks.workunit.client.0.smithi031.stdout:1/754: dwrite d2/d15/d1f/de9/d52/ff4 [0,4194304] 0 2024-09-16T08:57:29.635 INFO:tasks.workunit.client.0.smithi031.stdout:1/755: creat d2/d15/d4a/f10c x:0 0 0 2024-09-16T08:57:29.635 INFO:tasks.workunit.client.0.smithi031.stdout:3/889: symlink d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/ddd/l109 0 2024-09-16T08:57:29.646 INFO:tasks.workunit.client.0.smithi031.stdout:1/756: chown d2/d15/d1f/de9/f5 0 1 2024-09-16T08:57:29.646 INFO:tasks.workunit.client.0.smithi031.stdout:1/757: stat d2/d15/d1f/de9/l2e 0 2024-09-16T08:57:29.646 INFO:tasks.workunit.client.0.smithi031.stdout:1/758: fdatasync d2/f3c 0 2024-09-16T08:57:29.647 INFO:tasks.workunit.client.0.smithi031.stdout:3/890: creat d3/d14/d37/d1a/d1d/d9a/dde/dd6/f10a x:0 0 0 2024-09-16T08:57:29.648 INFO:tasks.workunit.client.0.smithi031.stdout:6/926: rmdir d0/df1 39 2024-09-16T08:57:29.649 INFO:tasks.workunit.client.0.smithi031.stdout:6/927: fsync d0/dd/d74/f55 0 2024-09-16T08:57:29.649 INFO:tasks.workunit.client.0.smithi031.stdout:1/759: truncate d2/d15/d1f/de9/d67/f72 652433 0 2024-09-16T08:57:29.649 INFO:tasks.workunit.client.0.smithi031.stdout:1/760: chown d2/de/db5/le7 2153 1 2024-09-16T08:57:29.651 INFO:tasks.workunit.client.0.smithi031.stdout:6/928: mkdir d0/dd/d74/d19/df9/d12c 0 2024-09-16T08:57:29.654 INFO:tasks.workunit.client.0.smithi031.stdout:6/929: symlink d0/dd/d74/d5e/d110/l12d 0 2024-09-16T08:57:29.655 INFO:tasks.workunit.client.0.smithi031.stdout:6/930: stat d0/dd/d2b/l8f 0 2024-09-16T08:57:29.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/891: dread d3/f4 [0,4194304] 0 2024-09-16T08:57:29.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/892: stat d3/ce0 0 2024-09-16T08:57:29.656 INFO:tasks.workunit.client.0.smithi031.stdout:3/893: write d3/d14/d37/fb0 [2315677,123406] 0 2024-09-16T08:57:29.657 INFO:tasks.workunit.client.0.smithi031.stdout:3/894: dread d3/d14/d37/f84 [0,4194304] 0 2024-09-16T08:57:29.657 INFO:tasks.workunit.client.0.smithi031.stdout:3/895: truncate d3/d14/d37/d33/d97/fb5 924702 0 2024-09-16T08:57:29.657 INFO:tasks.workunit.client.0.smithi031.stdout:3/896: write d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f72 [1861989,28116] 0 2024-09-16T08:57:29.657 INFO:tasks.workunit.client.0.smithi031.stdout:3/897: readlink d3/d14/d37/d1a/d1d/l86 0 2024-09-16T08:57:29.657 INFO:tasks.workunit.client.0.smithi031.stdout:3/898: fdatasync d3/d14/d37/f84 0 2024-09-16T08:57:29.658 INFO:tasks.workunit.client.0.smithi031.stdout:6/931: rename d0/dd/d74/d19/d54/lf0 to d0/dd/d74/d5e/d7c/d56/dc5/l12e 0 2024-09-16T08:57:29.663 INFO:tasks.workunit.client.0.smithi031.stdout:3/899: unlink d3/d14/d37/d1a/d1d/d3b/d3f/d79/f102 0 2024-09-16T08:57:29.664 INFO:tasks.workunit.client.0.smithi031.stdout:6/932: creat d0/d10/dd2/f12f x:0 0 0 2024-09-16T08:57:29.665 INFO:tasks.workunit.client.0.smithi031.stdout:3/900: mknod d3/d14/d85/c10b 0 2024-09-16T08:57:29.667 INFO:tasks.workunit.client.0.smithi031.stdout:1/761: dwrite d2/f96 [0,4194304] 0 2024-09-16T08:57:29.678 INFO:tasks.workunit.client.0.smithi031.stdout:3/901: dread d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f72 [0,4194304] 0 2024-09-16T08:57:29.679 INFO:tasks.workunit.client.0.smithi031.stdout:1/762: read d2/d15/d1f/de9/d67/fbc [1858187,106595] 0 2024-09-16T08:57:29.680 INFO:tasks.workunit.client.0.smithi031.stdout:6/933: dwrite d0/ffd [0,4194304] 0 2024-09-16T08:57:29.681 INFO:tasks.workunit.client.0.smithi031.stdout:1/763: getdents d2 0 2024-09-16T08:57:29.684 INFO:tasks.workunit.client.0.smithi031.stdout:3/902: dread d3/d14/d37/d33/fd9 [0,4194304] 0 2024-09-16T08:57:29.687 INFO:tasks.workunit.client.0.smithi031.stdout:3/903: link d3/d14/d37/d1a/d1d/d1e/f81 d3/d14/d37/d1a/d1d/d3b/d3f/dd1/dd3/f10c 0 2024-09-16T08:57:29.690 INFO:tasks.workunit.client.0.smithi031.stdout:1/764: write d2/d15/d4a/f6a [997831,3753] 0 2024-09-16T08:57:29.690 INFO:tasks.workunit.client.0.smithi031.stdout:1/765: readlink d2/d15/d1f/l7c 0 2024-09-16T08:57:29.690 INFO:tasks.workunit.client.0.smithi031.stdout:1/766: chown d2/d15/d9c 1 1 2024-09-16T08:57:29.691 INFO:tasks.workunit.client.0.smithi031.stdout:8/708: sync 2024-09-16T08:57:29.692 INFO:tasks.workunit.client.0.smithi031.stdout:8/709: read d4/d77/f9b [502073,130465] 0 2024-09-16T08:57:29.692 INFO:tasks.workunit.client.0.smithi031.stdout:8/710: fsync d4/d2f/dc3/f23 0 2024-09-16T08:57:29.692 INFO:tasks.workunit.client.0.smithi031.stdout:1/767: symlink d2/d15/d1f/de9/d1b/d5c/d6b/dc1/df0/l10d 0 2024-09-16T08:57:29.692 INFO:tasks.workunit.client.0.smithi031.stdout:1/768: chown d2/d15/ld9 2 1 2024-09-16T08:57:29.692 INFO:tasks.workunit.client.0.smithi031.stdout:1/769: fsync d2/f96 0 2024-09-16T08:57:29.693 INFO:tasks.workunit.client.0.smithi031.stdout:1/770: creat d2/d15/d9c/f10e x:0 0 0 2024-09-16T08:57:29.694 INFO:tasks.workunit.client.0.smithi031.stdout:8/711: dread d4/f27 [4194304,4194304] 0 2024-09-16T08:57:29.696 INFO:tasks.workunit.client.0.smithi031.stdout:1/771: truncate d2/f3e 401622 0 2024-09-16T08:57:29.697 INFO:tasks.workunit.client.0.smithi031.stdout:3/904: dread d3/d14/fb6 [0,4194304] 0 2024-09-16T08:57:29.699 INFO:tasks.workunit.client.0.smithi031.stdout:3/905: creat d3/d14/d37/d47/ded/f10d x:0 0 0 2024-09-16T08:57:29.700 INFO:tasks.workunit.client.0.smithi031.stdout:3/906: rename d3/d14/c22 to d3/d14/d37/d1a/d1d/d3b/d3f/d79/d7c/c10e 0 2024-09-16T08:57:29.702 INFO:tasks.workunit.client.0.smithi031.stdout:6/934: dwrite d0/dd/d74/d5e/d7c/d3f/f11f [0,4194304] 0 2024-09-16T08:57:29.702 INFO:tasks.workunit.client.0.smithi031.stdout:6/935: chown d0/df1 1752055087 1 2024-09-16T08:57:29.702 INFO:tasks.workunit.client.0.smithi031.stdout:6/936: creat d0/dd/d74/d5e/f130 x:0 0 0 2024-09-16T08:57:29.705 INFO:tasks.workunit.client.0.smithi031.stdout:6/937: symlink d0/dd/d74/d5e/d7c/d56/dc5/l131 0 2024-09-16T08:57:29.705 INFO:tasks.workunit.client.0.smithi031.stdout:6/938: chown d0/dd/d74/d5e/d7c/d56/dc5 1620 1 2024-09-16T08:57:29.708 INFO:tasks.workunit.client.0.smithi031.stdout:6/939: getdents d0/dd/d74/d5e/d110 0 2024-09-16T08:57:29.713 INFO:tasks.workunit.client.0.smithi031.stdout:6/940: dread d0/dd/d74/d5e/d7c/d56/f78 [0,4194304] 0 2024-09-16T08:57:29.713 INFO:tasks.workunit.client.0.smithi031.stdout:6/941: fdatasync d0/dd/d74/d5e/d7c/d3f/ddd/f10b 0 2024-09-16T08:57:29.713 INFO:tasks.workunit.client.0.smithi031.stdout:6/942: chown d0/dd/d74/d5e/d7c/d3f/d65/f68 417 1 2024-09-16T08:57:29.714 INFO:tasks.workunit.client.0.smithi031.stdout:6/943: link d0/dd/d35/d6c/d81/cb8 d0/dd/d74/d19/db4/c132 0 2024-09-16T08:57:29.714 INFO:tasks.workunit.client.0.smithi031.stdout:6/944: chown d0/dd/d74/d67/dbb/cea 15 1 2024-09-16T08:57:29.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/945: chown d0/d10/dc8/df6/c116 143529886 1 2024-09-16T08:57:29.715 INFO:tasks.workunit.client.0.smithi031.stdout:6/946: creat d0/dd/d74/d19/dcd/f133 x:0 0 0 2024-09-16T08:57:29.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/712: dwrite d4/d2f/dc3/d37/d46/f75 [0,4194304] 0 2024-09-16T08:57:29.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/713: fdatasync d4/d9e/d38/fda 0 2024-09-16T08:57:29.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/714: write d4/d2d/d6e/fab [3229761,129291] 0 2024-09-16T08:57:29.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/715: chown d4/d7/d66/f70 0 1 2024-09-16T08:57:29.723 INFO:tasks.workunit.client.0.smithi031.stdout:1/772: dwrite d2/d15/d1f/de9/f104 [0,4194304] 0 2024-09-16T08:57:29.724 INFO:tasks.workunit.client.0.smithi031.stdout:8/716: symlink d4/d2f/dc3/d37/d46/le8 0 2024-09-16T08:57:29.725 INFO:tasks.workunit.client.0.smithi031.stdout:3/907: dwrite d3/d14/d37/d1a/d1d/d6a/f105 [0,4194304] 0 2024-09-16T08:57:29.725 INFO:tasks.workunit.client.0.smithi031.stdout:3/908: write d3/d14/d37/d1a/d1d/d3b/d60/fb2 [97512,26189] 0 2024-09-16T08:57:29.725 INFO:tasks.workunit.client.0.smithi031.stdout:6/947: dread d0/dd/d74/d19/d9a/fc0 [0,4194304] 0 2024-09-16T08:57:29.728 INFO:tasks.workunit.client.0.smithi031.stdout:1/773: creat d2/d15/d1f/de9/d105/db9/f10f x:0 0 0 2024-09-16T08:57:29.729 INFO:tasks.workunit.client.0.smithi031.stdout:6/948: write d0/dd/d74/d19/d54/fe5 [882507,25107] 0 2024-09-16T08:57:29.729 INFO:tasks.workunit.client.0.smithi031.stdout:8/717: creat d4/d9e/fe9 x:0 0 0 2024-09-16T08:57:29.729 INFO:tasks.workunit.client.0.smithi031.stdout:8/718: dread - d4/d2f/fbd zero size 2024-09-16T08:57:29.731 INFO:tasks.workunit.client.0.smithi031.stdout:8/719: dread d4/d77/f9b [0,4194304] 0 2024-09-16T08:57:29.731 INFO:tasks.workunit.client.0.smithi031.stdout:8/720: fdatasync d4/f17 0 2024-09-16T08:57:29.733 INFO:tasks.workunit.client.0.smithi031.stdout:6/949: rename d0/d8e/fe8 to d0/dd/d74/d19/d54/d12a/f134 0 2024-09-16T08:57:29.733 INFO:tasks.workunit.client.0.smithi031.stdout:3/909: dread d3/d14/d37/d1a/d1d/d1e/f81 [0,4194304] 0 2024-09-16T08:57:29.734 INFO:tasks.workunit.client.0.smithi031.stdout:8/721: mkdir d4/d9e/dea 0 2024-09-16T08:57:29.734 INFO:tasks.workunit.client.0.smithi031.stdout:8/722: creat d4/d77/dd3/feb x:0 0 0 2024-09-16T08:57:29.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/723: write d4/d2f/dc3/d37/fc4 [225516,1600] 0 2024-09-16T08:57:29.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/724: stat d4/d9e/d38/f3c 0 2024-09-16T08:57:29.736 INFO:tasks.workunit.client.0.smithi031.stdout:1/774: dread d2/d15/d38/f102 [0,4194304] 0 2024-09-16T08:57:29.738 INFO:tasks.workunit.client.0.smithi031.stdout:6/950: truncate d0/dd/d74/d19/f85 12518891 0 2024-09-16T08:57:29.738 INFO:tasks.workunit.client.0.smithi031.stdout:6/951: dread - d0/dd/d74/d5e/d7c/d3f/f11d zero size 2024-09-16T08:57:29.739 INFO:tasks.workunit.client.0.smithi031.stdout:3/910: creat d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f10f x:0 0 0 2024-09-16T08:57:29.739 INFO:tasks.workunit.client.0.smithi031.stdout:3/911: fsync d3/d14/d37/d1a/d24/f40 0 2024-09-16T08:57:29.743 INFO:tasks.workunit.client.0.smithi031.stdout:8/725: truncate d4/d2f/f41 3599724 0 2024-09-16T08:57:29.747 INFO:tasks.workunit.client.0.smithi031.stdout:1/775: unlink d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea/cd0 0 2024-09-16T08:57:29.747 INFO:tasks.workunit.client.0.smithi031.stdout:1/776: read - d2/d15/d1f/de9/ff8 zero size 2024-09-16T08:57:29.749 INFO:tasks.workunit.client.0.smithi031.stdout:6/952: symlink d0/dd/d74/d67/dbb/l135 0 2024-09-16T08:57:29.749 INFO:tasks.workunit.client.0.smithi031.stdout:6/953: readlink d0/dd/d74/d67/le2 0 2024-09-16T08:57:29.749 INFO:tasks.workunit.client.0.smithi031.stdout:6/954: truncate d0/dd/d74/d5e/f130 839200 0 2024-09-16T08:57:29.752 INFO:tasks.workunit.client.0.smithi031.stdout:8/726: dread d4/d2f/dc3/d37/f76 [0,4194304] 0 2024-09-16T08:57:29.757 INFO:tasks.workunit.client.0.smithi031.stdout:6/955: dread d0/f91 [0,4194304] 0 2024-09-16T08:57:29.765 INFO:tasks.workunit.client.0.smithi031.stdout:1/777: mkdir d2/d15/d1f/de9/d1b/d5c/d6b/dc1/d110 0 2024-09-16T08:57:29.765 INFO:tasks.workunit.client.0.smithi031.stdout:8/727: creat d4/d2f/dc3/d24/fec x:0 0 0 2024-09-16T08:57:29.765 INFO:tasks.workunit.client.0.smithi031.stdout:8/728: getdents d4/d5b/d7b 0 2024-09-16T08:57:29.766 INFO:tasks.workunit.client.0.smithi031.stdout:8/729: dread - d4/d77/dd3/fdb zero size 2024-09-16T08:57:29.766 INFO:tasks.workunit.client.0.smithi031.stdout:8/730: chown d4/d9e/d38/f87 1710419115 1 2024-09-16T08:57:29.770 INFO:tasks.workunit.client.0.smithi031.stdout:8/731: creat d4/d9e/d5e/fed x:0 0 0 2024-09-16T08:57:29.774 INFO:tasks.workunit.client.0.smithi031.stdout:6/956: dwrite d0/dd/d74/d5e/d7c/d3f/d92/ff4 [0,4194304] 0 2024-09-16T08:57:29.774 INFO:tasks.workunit.client.0.smithi031.stdout:6/957: fsync d0/dd/d74/d67/dbb/d100/f126 0 2024-09-16T08:57:29.778 INFO:tasks.workunit.client.0.smithi031.stdout:1/778: rename d2/d15/d4a to d2/d15/d1f/d111 0 2024-09-16T08:57:29.778 INFO:tasks.workunit.client.0.smithi031.stdout:1/779: dread - d2/d15/d1f/de9/d52/fd3 zero size 2024-09-16T08:57:29.778 INFO:tasks.workunit.client.0.smithi031.stdout:1/780: chown d2/d15/d38/fb2 1 1 2024-09-16T08:57:29.782 INFO:tasks.workunit.client.0.smithi031.stdout:5/767: sync 2024-09-16T08:57:29.782 INFO:tasks.workunit.client.0.smithi031.stdout:5/768: fsync d2/dc/d25/d21/f85 0 2024-09-16T08:57:29.783 INFO:tasks.workunit.client.0.smithi031.stdout:1/781: rmdir d2/d15/d1f 39 2024-09-16T08:57:29.788 INFO:tasks.workunit.client.0.smithi031.stdout:1/782: symlink d2/d15/d32/l112 0 2024-09-16T08:57:29.788 INFO:tasks.workunit.client.0.smithi031.stdout:1/783: creat d2/d15/d1f/de9/d52/f113 x:0 0 0 2024-09-16T08:57:29.791 INFO:tasks.workunit.client.0.smithi031.stdout:1/784: symlink d2/l114 0 2024-09-16T08:57:29.799 INFO:tasks.workunit.client.0.smithi031.stdout:8/732: dwrite d4/f43 [4194304,4194304] 0 2024-09-16T08:57:29.800 INFO:tasks.workunit.client.0.smithi031.stdout:8/733: dread - d4/d5b/fd4 zero size 2024-09-16T08:57:29.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/769: dwrite d2/d4b/fdd [0,4194304] 0 2024-09-16T08:57:29.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/770: creat d2/dc/d8c/de8/f107 x:0 0 0 2024-09-16T08:57:29.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/771: fsync d2/dc/d25/d21/f85 0 2024-09-16T08:57:29.805 INFO:tasks.workunit.client.0.smithi031.stdout:5/772: read d2/f13 [2012445,59313] 0 2024-09-16T08:57:29.805 INFO:tasks.workunit.client.0.smithi031.stdout:6/958: dwrite d0/dd/d74/d5e/f98 [0,4194304] 0 2024-09-16T08:57:29.811 INFO:tasks.workunit.client.0.smithi031.stdout:5/773: getdents d2/dc/d25/dbf/de4 0 2024-09-16T08:57:29.812 INFO:tasks.workunit.client.0.smithi031.stdout:5/774: dread - d2/dc/d3d/d4c/fbd zero size 2024-09-16T08:57:29.812 INFO:tasks.workunit.client.0.smithi031.stdout:5/775: truncate d2/dc/df3/f106 956165 0 2024-09-16T08:57:29.812 INFO:tasks.workunit.client.0.smithi031.stdout:1/785: dwrite d2/d15/d9c/f10e [0,4194304] 0 2024-09-16T08:57:29.819 INFO:tasks.workunit.client.0.smithi031.stdout:6/959: unlink d0/dd/f3e 0 2024-09-16T08:57:29.819 INFO:tasks.workunit.client.0.smithi031.stdout:6/960: creat d0/dd/d74/d5e/d7c/d3f/ddd/f136 x:0 0 0 2024-09-16T08:57:29.826 INFO:tasks.workunit.client.0.smithi031.stdout:8/734: dwrite d4/d9e/d38/f73 [0,4194304] 0 2024-09-16T08:57:29.826 INFO:tasks.workunit.client.0.smithi031.stdout:6/961: creat d0/dd/d35/d6c/dac/f137 x:0 0 0 2024-09-16T08:57:29.830 INFO:tasks.workunit.client.0.smithi031.stdout:8/735: unlink d4/d2f/dc3/d37/d46/d88/ld6 0 2024-09-16T08:57:29.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/736: mkdir d4/d9e/d38/dc8/dee 0 2024-09-16T08:57:29.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/737: dread - d4/d77/dd3/fdb zero size 2024-09-16T08:57:29.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/738: fsync d4/d9e/fe9 0 2024-09-16T08:57:29.833 INFO:tasks.workunit.client.0.smithi031.stdout:8/739: truncate d4/d2f/f95 359565 0 2024-09-16T08:57:29.834 INFO:tasks.workunit.client.0.smithi031.stdout:8/740: chown d4/d9e/l4e 57707 1 2024-09-16T08:57:29.834 INFO:tasks.workunit.client.0.smithi031.stdout:8/741: stat d4/d7/f28 0 2024-09-16T08:57:29.835 INFO:tasks.workunit.client.0.smithi031.stdout:8/742: symlink d4/lef 0 2024-09-16T08:57:29.835 INFO:tasks.workunit.client.0.smithi031.stdout:8/743: chown d4/l12 172 1 2024-09-16T08:57:29.835 INFO:tasks.workunit.client.0.smithi031.stdout:8/744: getdents d4/db8 0 2024-09-16T08:57:29.836 INFO:tasks.workunit.client.0.smithi031.stdout:8/745: readlink d4/d2f/dc3/l9f 0 2024-09-16T08:57:29.837 INFO:tasks.workunit.client.0.smithi031.stdout:8/746: unlink d4/d7/fad 0 2024-09-16T08:57:29.837 INFO:tasks.workunit.client.0.smithi031.stdout:8/747: write d4/d2f/dc3/d24/fc6 [421101,53518] 0 2024-09-16T08:57:29.837 INFO:tasks.workunit.client.0.smithi031.stdout:8/748: stat d4/d2f/dc3/dcb/fa1 0 2024-09-16T08:57:29.838 INFO:tasks.workunit.client.0.smithi031.stdout:8/749: dread d4/d2f/dc3/d37/fc4 [0,4194304] 0 2024-09-16T08:57:29.838 INFO:tasks.workunit.client.0.smithi031.stdout:8/750: write d4/d2f/dc3/dcb/fa1 [59093,73309] 0 2024-09-16T08:57:29.839 INFO:tasks.workunit.client.0.smithi031.stdout:5/776: dwrite d2/dc/d91/fdc [0,4194304] 0 2024-09-16T08:57:29.839 INFO:tasks.workunit.client.0.smithi031.stdout:1/786: dwrite d2/d15/d1f/de9/f104 [0,4194304] 0 2024-09-16T08:57:29.840 INFO:tasks.workunit.client.0.smithi031.stdout:8/751: creat d4/d2f/dc3/dcb/ff0 x:0 0 0 2024-09-16T08:57:29.845 INFO:tasks.workunit.client.0.smithi031.stdout:5/777: rename d2/dc/d91/cb8 to d2/dc/df3/c108 0 2024-09-16T08:57:29.845 INFO:tasks.workunit.client.0.smithi031.stdout:5/778: write d2/dc/d25/f9f [3223344,125606] 0 2024-09-16T08:57:29.846 INFO:tasks.workunit.client.0.smithi031.stdout:8/752: mkdir d4/d7/d66/df1 0 2024-09-16T08:57:29.855 INFO:tasks.workunit.client.0.smithi031.stdout:8/753: read d4/fb4 [3666075,47725] 0 2024-09-16T08:57:29.856 INFO:tasks.workunit.client.0.smithi031.stdout:1/787: dwrite d2/d15/d1f/de9/d1b/d5c/d6b/da9/fbe [0,4194304] 0 2024-09-16T08:57:29.856 INFO:tasks.workunit.client.0.smithi031.stdout:5/779: write d2/dc/d25/d21/d81/dc3/fc6 [2024859,80635] 0 2024-09-16T08:57:29.857 INFO:tasks.workunit.client.0.smithi031.stdout:8/754: getdents d4/d5b/d8d/dcd 0 2024-09-16T08:57:29.857 INFO:tasks.workunit.client.0.smithi031.stdout:8/755: stat d4/d2f/dc3/d37/d46/db1/c71 0 2024-09-16T08:57:29.857 INFO:tasks.workunit.client.0.smithi031.stdout:8/756: creat d4/d2f/dc3/dcb/ff2 x:0 0 0 2024-09-16T08:57:29.858 INFO:tasks.workunit.client.0.smithi031.stdout:1/788: mkdir d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/d115 0 2024-09-16T08:57:29.858 INFO:tasks.workunit.client.0.smithi031.stdout:1/789: stat d2/d15/d5d/db4/da3/dcb 0 2024-09-16T08:57:29.858 INFO:tasks.workunit.client.0.smithi031.stdout:1/790: chown d2/d15/d1f/de9/l2e 5588 1 2024-09-16T08:57:29.861 INFO:tasks.workunit.client.0.smithi031.stdout:5/780: unlink d2/dc/d25/d21/d2e/f5c 0 2024-09-16T08:57:29.863 INFO:tasks.workunit.client.0.smithi031.stdout:8/757: creat d4/d2d/d6e/ff3 x:0 0 0 2024-09-16T08:57:29.865 INFO:tasks.workunit.client.0.smithi031.stdout:5/781: mkdir d2/dc/d36/d76/d109 0 2024-09-16T08:57:29.865 INFO:tasks.workunit.client.0.smithi031.stdout:8/758: dread d4/d2f/f95 [0,4194304] 0 2024-09-16T08:57:29.866 INFO:tasks.workunit.client.0.smithi031.stdout:8/759: write d4/d2f/dc3/d37/fe3 [30966,90863] 0 2024-09-16T08:57:29.867 INFO:tasks.workunit.client.0.smithi031.stdout:5/782: unlink d2/dc/d25/d68/lf6 0 2024-09-16T08:57:29.867 INFO:tasks.workunit.client.0.smithi031.stdout:5/783: write d2/dc/d36/f41 [1426458,125855] 0 2024-09-16T08:57:29.867 INFO:tasks.workunit.client.0.smithi031.stdout:5/784: fsync d2/dc/f3b 0 2024-09-16T08:57:29.867 INFO:tasks.workunit.client.0.smithi031.stdout:5/785: write d2/dc/d25/f53 [510889,41036] 0 2024-09-16T08:57:29.867 INFO:tasks.workunit.client.0.smithi031.stdout:5/786: write d2/dc/d3d/d4c/f62 [356218,71205] 0 2024-09-16T08:57:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:8/760: creat d4/d77/ddc/ff4 x:0 0 0 2024-09-16T08:57:29.868 INFO:tasks.workunit.client.0.smithi031.stdout:8/761: creat d4/d9e/d5e/ff5 x:0 0 0 2024-09-16T08:57:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:5/787: link d2/f87 d2/dc/d25/f10a 0 2024-09-16T08:57:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:5/788: creat d2/dc/df3/f10b x:0 0 0 2024-09-16T08:57:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:5/789: readlink d2/dc/d25/lf4 0 2024-09-16T08:57:29.870 INFO:tasks.workunit.client.0.smithi031.stdout:8/762: truncate d4/d2f/dc3/d37/d46/db1/f6a 875693 0 2024-09-16T08:57:29.872 INFO:tasks.workunit.client.0.smithi031.stdout:5/790: creat d2/d4b/f10c x:0 0 0 2024-09-16T08:57:29.873 INFO:tasks.workunit.client.0.smithi031.stdout:5/791: dread - d2/dc/d36/fae zero size 2024-09-16T08:57:29.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/791: dwrite d2/d15/d1f/de9/d52/fd3 [0,4194304] 0 2024-09-16T08:57:29.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/792: truncate d2/d15/d1f/de9/d1b/f50 1680216 0 2024-09-16T08:57:29.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/793: truncate d2/d15/d1f/d111/fdd 660719 0 2024-09-16T08:57:29.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/794: truncate d2/d15/d5d/db4/fc3 880148 0 2024-09-16T08:57:29.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/795: read - d2/d15/d1f/de9/d52/fe0 zero size 2024-09-16T08:57:29.890 INFO:tasks.workunit.client.0.smithi031.stdout:8/763: dwrite d4/d7/f28 [0,4194304] 0 2024-09-16T08:57:29.896 INFO:tasks.workunit.client.0.smithi031.stdout:5/792: rmdir d2 39 2024-09-16T08:57:29.900 INFO:tasks.workunit.client.0.smithi031.stdout:1/796: mkdir d2/d15/d1f/de9/d116 0 2024-09-16T08:57:29.927 INFO:tasks.workunit.client.0.smithi031.stdout:8/764: dwrite d4/d5b/d7b/f7d [0,4194304] 0 2024-09-16T08:57:29.928 INFO:tasks.workunit.client.0.smithi031.stdout:5/793: dwrite d2/dc/d25/d21/fff [0,4194304] 0 2024-09-16T08:57:29.929 INFO:tasks.workunit.client.0.smithi031.stdout:1/797: dwrite d2/f96 [0,4194304] 0 2024-09-16T08:57:29.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/765: truncate d4/d5b/d7b/f7d 3272602 0 2024-09-16T08:57:29.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/766: readlink d4/d2f/dc3/l64 0 2024-09-16T08:57:29.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/767: fdatasync d4/d2f/dc3/d24/f86 0 2024-09-16T08:57:29.938 INFO:tasks.workunit.client.0.smithi031.stdout:8/768: chown d4/fc5 2 1 2024-09-16T08:57:29.939 INFO:tasks.workunit.client.0.smithi031.stdout:8/769: truncate d4/d2f/dc3/d24/fec 613028 0 2024-09-16T08:57:29.940 INFO:tasks.workunit.client.0.smithi031.stdout:8/770: symlink d4/d2f/dc3/d24/lf6 0 2024-09-16T08:57:29.941 INFO:tasks.workunit.client.0.smithi031.stdout:8/771: readlink d4/d2f/dc3/d24/lf6 0 2024-09-16T08:57:29.942 INFO:tasks.workunit.client.0.smithi031.stdout:5/794: write d2/dc/d3d/d4c/f7a [278094,79574] 0 2024-09-16T08:57:29.943 INFO:tasks.workunit.client.0.smithi031.stdout:1/798: dread d2/d15/d38/fb2 [0,4194304] 0 2024-09-16T08:57:29.944 INFO:tasks.workunit.client.0.smithi031.stdout:1/799: dread d2/f53 [0,4194304] 0 2024-09-16T08:57:29.945 INFO:tasks.workunit.client.0.smithi031.stdout:8/772: symlink d4/d9e/d38/dc8/lf7 0 2024-09-16T08:57:29.948 INFO:tasks.workunit.client.0.smithi031.stdout:1/800: dread d2/d15/d1f/de9/f54 [0,4194304] 0 2024-09-16T08:57:29.949 INFO:tasks.workunit.client.0.smithi031.stdout:5/795: mknod d2/d100/c10d 0 2024-09-16T08:57:29.949 INFO:tasks.workunit.client.0.smithi031.stdout:5/796: dread - d2/dc/d1d/dbb/fcb zero size 2024-09-16T08:57:29.951 INFO:tasks.workunit.client.0.smithi031.stdout:8/773: rename d4/d5b/d8d/l96 to d4/db8/lf8 0 2024-09-16T08:57:29.955 INFO:tasks.workunit.client.0.smithi031.stdout:8/774: link d4/d2f/dc3/d37/d46/l5c d4/d2f/dc3/d24/lf9 0 2024-09-16T08:57:29.959 INFO:tasks.workunit.client.0.smithi031.stdout:5/797: write d2/dc/d25/d21/d2e/f8f [3030527,100095] 0 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:7/877: sync 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:4/934: sync 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:2/834: sync 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:9/920: sync 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:7/878: chown d2/d8/f63 126807692 1 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/962: sync 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:3/912: sync 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/963: dread - d0/d10/fc7 zero size 2024-09-16T08:57:29.960 INFO:tasks.workunit.client.0.smithi031.stdout:6/964: write d0/dd/d74/d5e/f98 [4535721,118942] 0 2024-09-16T08:57:29.961 INFO:tasks.workunit.client.0.smithi031.stdout:0/924: sync 2024-09-16T08:57:29.961 INFO:tasks.workunit.client.0.smithi031.stdout:0/925: write d6/d12/d47/f55 [3192013,39872] 0 2024-09-16T08:57:29.963 INFO:tasks.workunit.client.0.smithi031.stdout:5/798: creat d2/dc/f10e x:0 0 0 2024-09-16T08:57:29.964 INFO:tasks.workunit.client.0.smithi031.stdout:7/879: truncate d2/d2d/fd8 827030 0 2024-09-16T08:57:29.965 INFO:tasks.workunit.client.0.smithi031.stdout:7/880: chown d2/d8/d11/d66/f10e 573016 1 2024-09-16T08:57:29.966 INFO:tasks.workunit.client.0.smithi031.stdout:6/965: mknod d0/dd/d74/d5e/d7c/d3f/c138 0 2024-09-16T08:57:29.968 INFO:tasks.workunit.client.0.smithi031.stdout:9/921: truncate d3/d5/da/d34/d3c/d5e/d9c/f9f 795875 0 2024-09-16T08:57:29.969 INFO:tasks.workunit.client.0.smithi031.stdout:7/881: truncate d2/d8/d11/d5d/d7e/fb9 517341 0 2024-09-16T08:57:29.973 INFO:tasks.workunit.client.0.smithi031.stdout:6/966: rename d0/dd/d74/f2d to d0/dd/d35/d6c/d81/d10a/f139 0 2024-09-16T08:57:29.973 INFO:tasks.workunit.client.0.smithi031.stdout:7/882: creat d2/d2d/d68/d113/d114/f11f x:0 0 0 2024-09-16T08:57:29.974 INFO:tasks.workunit.client.0.smithi031.stdout:1/801: dwrite d2/d15/d1f/de9/d1b/fdc [0,4194304] 0 2024-09-16T08:57:29.974 INFO:tasks.workunit.client.0.smithi031.stdout:1/802: dread - d2/d15/d1f/d111/f5b zero size 2024-09-16T08:57:29.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/803: creat d2/d15/d1f/de9/d1b/de4/f117 x:0 0 0 2024-09-16T08:57:29.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/804: readlink d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea/l57 0 2024-09-16T08:57:29.975 INFO:tasks.workunit.client.0.smithi031.stdout:3/913: dread d3/d14/d37/f10 [0,4194304] 0 2024-09-16T08:57:29.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/805: creat d2/d15/d9c/f118 x:0 0 0 2024-09-16T08:57:29.975 INFO:tasks.workunit.client.0.smithi031.stdout:7/883: truncate d2/f22 5899313 0 2024-09-16T08:57:29.976 INFO:tasks.workunit.client.0.smithi031.stdout:2/835: dread d6/d2e/d40/d6c/d2c/d53/d60/f35 [0,4194304] 0 2024-09-16T08:57:29.976 INFO:tasks.workunit.client.0.smithi031.stdout:2/836: readlink d6/da/d37/d8b/dad/laf 0 2024-09-16T08:57:29.976 INFO:tasks.workunit.client.0.smithi031.stdout:2/837: write d6/d20/d51/ddc/fe6 [203312,51637] 0 2024-09-16T08:57:29.979 INFO:tasks.workunit.client.0.smithi031.stdout:3/914: creat d3/d14/d37/d1a/d1d/d3b/d3f/dd1/f110 x:0 0 0 2024-09-16T08:57:29.979 INFO:tasks.workunit.client.0.smithi031.stdout:7/884: mkdir d2/d2d/d68/d113/d120 0 2024-09-16T08:57:29.981 INFO:tasks.workunit.client.0.smithi031.stdout:2/838: mknod d6/da/d37/deb/c116 0 2024-09-16T08:57:29.982 INFO:tasks.workunit.client.0.smithi031.stdout:7/885: mkdir d2/d8/d11/d66/d9a/d121 0 2024-09-16T08:57:29.982 INFO:tasks.workunit.client.0.smithi031.stdout:7/886: readlink d2/d8/d11/d81/la7 0 2024-09-16T08:57:29.982 INFO:tasks.workunit.client.0.smithi031.stdout:7/887: chown d2/d8/d11/d66/d9a/fde 1946 1 2024-09-16T08:57:29.984 INFO:tasks.workunit.client.0.smithi031.stdout:2/839: unlink d6/da/f85 0 2024-09-16T08:57:29.985 INFO:tasks.workunit.client.0.smithi031.stdout:2/840: fsync d6/da/d37/f63 0 2024-09-16T08:57:29.986 INFO:tasks.workunit.client.0.smithi031.stdout:3/915: rmdir d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d7d 39 2024-09-16T08:57:29.986 INFO:tasks.workunit.client.0.smithi031.stdout:3/916: creat d3/d14/d37/d47/f111 x:0 0 0 2024-09-16T08:57:29.990 INFO:tasks.workunit.client.0.smithi031.stdout:8/775: dwrite d4/d2d/d6e/f74 [0,4194304] 0 2024-09-16T08:57:29.990 INFO:tasks.workunit.client.0.smithi031.stdout:6/967: dread d0/f91 [0,4194304] 0 2024-09-16T08:57:29.990 INFO:tasks.workunit.client.0.smithi031.stdout:8/776: readlink d4/d9e/d38/dc8/ld0 0 2024-09-16T08:57:29.990 INFO:tasks.workunit.client.0.smithi031.stdout:8/777: write d4/d9e/fe9 [873155,122611] 0 2024-09-16T08:57:29.999 INFO:tasks.workunit.client.0.smithi031.stdout:7/888: dread d2/d8/d11/d66/f10e [4194304,4194304] 0 2024-09-16T08:57:30.010 INFO:tasks.workunit.client.0.smithi031.stdout:8/778: dread d4/d9e/d38/f3c [0,4194304] 0 2024-09-16T08:57:30.011 INFO:tasks.workunit.client.0.smithi031.stdout:8/779: mkdir d4/d7/d66/dfa 0 2024-09-16T08:57:30.016 INFO:tasks.workunit.client.0.smithi031.stdout:5/799: dwrite d2/dc/d36/ff5 [0,4194304] 0 2024-09-16T08:57:30.019 INFO:tasks.workunit.client.0.smithi031.stdout:5/800: dread d2/dc/d25/d21/fa3 [0,4194304] 0 2024-09-16T08:57:30.021 INFO:tasks.workunit.client.0.smithi031.stdout:5/801: creat d2/dc/d91/f10f x:0 0 0 2024-09-16T08:57:30.021 INFO:tasks.workunit.client.0.smithi031.stdout:5/802: fsync d2/dc/d25/d21/f6d 0 2024-09-16T08:57:30.021 INFO:tasks.workunit.client.0.smithi031.stdout:5/803: readlink d2/dc/l17 0 2024-09-16T08:57:30.024 INFO:tasks.workunit.client.0.smithi031.stdout:0/926: dwrite d6/d12/d13/ff3 [0,4194304] 0 2024-09-16T08:57:30.025 INFO:tasks.workunit.client.0.smithi031.stdout:0/927: mknod d6/d12/c127 0 2024-09-16T08:57:30.026 INFO:tasks.workunit.client.0.smithi031.stdout:0/928: mknod d6/d12/d13/dca/d11c/c128 0 2024-09-16T08:57:30.026 INFO:tasks.workunit.client.0.smithi031.stdout:0/929: creat d6/d12/d47/d5b/f129 x:0 0 0 2024-09-16T08:57:30.027 INFO:tasks.workunit.client.0.smithi031.stdout:4/935: dwrite d0/f94 [0,4194304] 0 2024-09-16T08:57:30.027 INFO:tasks.workunit.client.0.smithi031.stdout:4/936: readlink d0/d3c/lb3 0 2024-09-16T08:57:30.028 INFO:tasks.workunit.client.0.smithi031.stdout:9/922: dwrite d3/d5/da/d34/d3c/d5e/d9c/d95/fae [0,4194304] 0 2024-09-16T08:57:30.028 INFO:tasks.workunit.client.0.smithi031.stdout:9/923: dread - d3/d5/da/d34/f131 zero size 2024-09-16T08:57:30.028 INFO:tasks.workunit.client.0.smithi031.stdout:9/924: dread - d3/d5/da/d34/f131 zero size 2024-09-16T08:57:30.028 INFO:tasks.workunit.client.0.smithi031.stdout:4/937: symlink d0/l135 0 2024-09-16T08:57:30.029 INFO:tasks.workunit.client.0.smithi031.stdout:2/841: dwrite d6/da/d4b/d4c/d68/fa8 [0,4194304] 0 2024-09-16T08:57:30.029 INFO:tasks.workunit.client.0.smithi031.stdout:2/842: chown d6/da/c21 1942479 1 2024-09-16T08:57:30.029 INFO:tasks.workunit.client.0.smithi031.stdout:2/843: dread - d6/da/d37/fe5 zero size 2024-09-16T08:57:30.030 INFO:tasks.workunit.client.0.smithi031.stdout:9/925: unlink d3/d5/d6a/d74/dd0/de3/ff7 0 2024-09-16T08:57:30.034 INFO:tasks.workunit.client.0.smithi031.stdout:2/844: rmdir d6/d2e/d40/d6c/d2c/d53/d60/d91 39 2024-09-16T08:57:30.034 INFO:tasks.workunit.client.0.smithi031.stdout:2/845: stat d6/d2e/d40/d6c/d2c/d53/d60/f35 0 2024-09-16T08:57:30.034 INFO:tasks.workunit.client.0.smithi031.stdout:6/968: dwrite d0/dd/d74/d5e/f130 [0,4194304] 0 2024-09-16T08:57:30.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/780: dwrite d4/d2f/dc3/f3b [0,4194304] 0 2024-09-16T08:57:30.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/781: dread - d4/d2f/dc3/dcb/ff2 zero size 2024-09-16T08:57:30.041 INFO:tasks.workunit.client.0.smithi031.stdout:8/782: creat d4/d9e/d5e/ffb x:0 0 0 2024-09-16T08:57:30.041 INFO:tasks.workunit.client.0.smithi031.stdout:0/930: read d6/d12/d13/f3e [879574,17276] 0 2024-09-16T08:57:30.041 INFO:tasks.workunit.client.0.smithi031.stdout:3/917: dwrite d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f10f [0,4194304] 0 2024-09-16T08:57:30.041 INFO:tasks.workunit.client.0.smithi031.stdout:0/931: write d6/d12/d47/dc5/dc9/d10f/f109 [575663,97077] 0 2024-09-16T08:57:30.042 INFO:tasks.workunit.client.0.smithi031.stdout:0/932: dread - d6/d12/d47/dc5/dc9/d10f/fd7 zero size 2024-09-16T08:57:30.048 INFO:tasks.workunit.client.0.smithi031.stdout:3/918: rmdir d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/ddd 39 2024-09-16T08:57:30.048 INFO:tasks.workunit.client.0.smithi031.stdout:3/919: stat d3/d14/d37/d1a/d1d/d3b/d94/le3 0 2024-09-16T08:57:30.048 INFO:tasks.workunit.client.0.smithi031.stdout:3/920: read d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f72 [737635,71808] 0 2024-09-16T08:57:30.050 INFO:tasks.workunit.client.0.smithi031.stdout:8/783: write d4/f17 [7171300,115664] 0 2024-09-16T08:57:30.050 INFO:tasks.workunit.client.0.smithi031.stdout:7/889: getdents d2/d2d/d68/d113/d114 0 2024-09-16T08:57:30.055 INFO:tasks.workunit.client.0.smithi031.stdout:0/933: truncate d6/d12/d47/d5b/f5e 2740686 0 2024-09-16T08:57:30.058 INFO:tasks.workunit.client.0.smithi031.stdout:8/784: rename d4/c1e to d4/d2f/dc3/d37/d46/cfc 0 2024-09-16T08:57:30.058 INFO:tasks.workunit.client.0.smithi031.stdout:8/785: chown d4/d2f/dc3/f3b 706184941 1 2024-09-16T08:57:30.058 INFO:tasks.workunit.client.0.smithi031.stdout:8/786: chown d4/d2f/dc3/dcb/ff0 148031793 1 2024-09-16T08:57:30.058 INFO:tasks.workunit.client.0.smithi031.stdout:8/787: dread - d4/d9e/d38/fe0 zero size 2024-09-16T08:57:30.061 INFO:tasks.workunit.client.0.smithi031.stdout:8/788: creat d4/d7/d66/de1/ffd x:0 0 0 2024-09-16T08:57:30.061 INFO:tasks.workunit.client.0.smithi031.stdout:8/789: mknod d4/d77/dd3/cfe 0 2024-09-16T08:57:30.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/790: creat d4/d2f/dc3/d37/d46/fff x:0 0 0 2024-09-16T08:57:30.064 INFO:tasks.workunit.client.0.smithi031.stdout:7/890: dread d2/d8/f1d [0,4194304] 0 2024-09-16T08:57:30.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/791: write d4/d7/f3d [934363,91261] 0 2024-09-16T08:57:30.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/792: chown d4/d5b/d8d/dcd/c59 3344 1 2024-09-16T08:57:30.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/793: write d4/d2f/dc3/dcb/ff0 [940566,99448] 0 2024-09-16T08:57:30.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/794: chown d4/d2d/d6e/fa4 9 1 2024-09-16T08:57:30.064 INFO:tasks.workunit.client.0.smithi031.stdout:8/795: fdatasync d4/d2d/d6e/f74 0 2024-09-16T08:57:30.066 INFO:tasks.workunit.client.0.smithi031.stdout:7/891: rename d2/d8/d11/d5d/d7e/db6/c105 to d2/d8/d11/d5d/d7e/db6/c122 0 2024-09-16T08:57:30.066 INFO:tasks.workunit.client.0.smithi031.stdout:0/934: dread d6/d12/d47/d4d/fae [0,4194304] 0 2024-09-16T08:57:30.066 INFO:tasks.workunit.client.0.smithi031.stdout:8/796: mknod d4/d7/d99/c100 0 2024-09-16T08:57:30.068 INFO:tasks.workunit.client.0.smithi031.stdout:5/804: dwrite d2/dc/fd [4194304,4194304] 0 2024-09-16T08:57:30.068 INFO:tasks.workunit.client.0.smithi031.stdout:6/969: dwrite d0/dd/d74/d5e/d7c/d3f/f11d [0,4194304] 0 2024-09-16T08:57:30.068 INFO:tasks.workunit.client.0.smithi031.stdout:9/926: dwrite d3/d85/f98 [0,4194304] 0 2024-09-16T08:57:30.068 INFO:tasks.workunit.client.0.smithi031.stdout:4/938: dwrite d0/d2/fdb [0,4194304] 0 2024-09-16T08:57:30.069 INFO:tasks.workunit.client.0.smithi031.stdout:2/846: dwrite d6/d20/da1/dcf/dda/fee [0,4194304] 0 2024-09-16T08:57:30.069 INFO:tasks.workunit.client.0.smithi031.stdout:2/847: fdatasync d6/d20/fac 0 2024-09-16T08:57:30.069 INFO:tasks.workunit.client.0.smithi031.stdout:2/848: fdatasync d6/d2e/d40/d58/dae/db1/fc4 0 2024-09-16T08:57:30.069 INFO:tasks.workunit.client.0.smithi031.stdout:4/939: dread d0/d3c/d6b/d9c/fae [0,4194304] 0 2024-09-16T08:57:30.070 INFO:tasks.workunit.client.0.smithi031.stdout:7/892: mkdir d2/d8/d86/d123 0 2024-09-16T08:57:30.070 INFO:tasks.workunit.client.0.smithi031.stdout:0/935: mknod d6/d12/d47/dc5/c12a 0 2024-09-16T08:57:30.072 INFO:tasks.workunit.client.0.smithi031.stdout:5/805: mknod d2/dc/d1d/d97/c110 0 2024-09-16T08:57:30.073 INFO:tasks.workunit.client.0.smithi031.stdout:5/806: chown d2/dc/l6f 197 1 2024-09-16T08:57:30.073 INFO:tasks.workunit.client.0.smithi031.stdout:8/797: dread d4/d2f/dc3/dcb/ff0 [0,4194304] 0 2024-09-16T08:57:30.073 INFO:tasks.workunit.client.0.smithi031.stdout:8/798: fsync d4/d2f/dc3/f3b 0 2024-09-16T08:57:30.074 INFO:tasks.workunit.client.0.smithi031.stdout:8/799: fdatasync d4/d2f/dc3/d37/d46/fff 0 2024-09-16T08:57:30.076 INFO:tasks.workunit.client.0.smithi031.stdout:6/970: link d0/dd/d74/d19/f2e d0/dd/d74/d67/f13a 0 2024-09-16T08:57:30.076 INFO:tasks.workunit.client.0.smithi031.stdout:6/971: readlink d0/dd/d74/d5e/d7c/l6a 0 2024-09-16T08:57:30.084 INFO:tasks.workunit.client.0.smithi031.stdout:9/927: getdents d3/d5/da/d34/d3c/d5e/d9c/d95 0 2024-09-16T08:57:30.092 INFO:tasks.workunit.client.0.smithi031.stdout:6/972: dwrite d0/dd/d74/d19/d54/f9c [0,4194304] 0 2024-09-16T08:57:30.092 INFO:tasks.workunit.client.0.smithi031.stdout:6/973: truncate d0/dd/d74/da6/fcf 552470 0 2024-09-16T08:57:30.092 INFO:tasks.workunit.client.0.smithi031.stdout:6/974: chown d0/dd/d74/d5e/d110/le6 55085 1 2024-09-16T08:57:30.092 INFO:tasks.workunit.client.0.smithi031.stdout:4/940: creat d0/d102/d84/d86/d9b/dcf/dde/f136 x:0 0 0 2024-09-16T08:57:30.094 INFO:tasks.workunit.client.0.smithi031.stdout:7/893: rename d2/d8/d11/c4c to d2/d8/d11/d66/c124 0 2024-09-16T08:57:30.095 INFO:tasks.workunit.client.0.smithi031.stdout:0/936: mknod d6/d12/d97/dc6/c12b 0 2024-09-16T08:57:30.095 INFO:tasks.workunit.client.0.smithi031.stdout:0/937: fdatasync d6/d12/d13/d3c/fdb 0 2024-09-16T08:57:30.099 INFO:tasks.workunit.client.0.smithi031.stdout:5/807: truncate d2/dc/f1f 2294482 0 2024-09-16T08:57:30.101 INFO:tasks.workunit.client.0.smithi031.stdout:8/800: truncate d4/d2d/d6e/f74 3887562 0 2024-09-16T08:57:30.101 INFO:tasks.workunit.client.0.smithi031.stdout:8/801: creat d4/d2f/dc3/d24/f101 x:0 0 0 2024-09-16T08:57:30.103 INFO:tasks.workunit.client.0.smithi031.stdout:9/928: mkdir d3/d5/da/d34/d3c/d5e/d9c/d95/d147 0 2024-09-16T08:57:30.109 INFO:tasks.workunit.client.0.smithi031.stdout:3/921: dwrite d3/d14/d37/d47/f5b [0,4194304] 0 2024-09-16T08:57:30.109 INFO:tasks.workunit.client.0.smithi031.stdout:3/922: creat d3/d14/d37/d1a/d1d/d3b/d3f/dd1/f112 x:0 0 0 2024-09-16T08:57:30.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/975: creat d0/dd/d74/d67/f13b x:0 0 0 2024-09-16T08:57:30.110 INFO:tasks.workunit.client.0.smithi031.stdout:6/976: creat d0/dd/d74/d19/db4/f13c x:0 0 0 2024-09-16T08:57:30.110 INFO:tasks.workunit.client.0.smithi031.stdout:4/941: mkdir d0/d102/dd5/d137 0 2024-09-16T08:57:30.111 INFO:tasks.workunit.client.0.smithi031.stdout:7/894: creat d2/d8/d11/dd5/f125 x:0 0 0 2024-09-16T08:57:30.114 INFO:tasks.workunit.client.0.smithi031.stdout:5/808: truncate d2/dc/d25/d21/d2e/fa5 3560859 0 2024-09-16T08:57:30.120 INFO:tasks.workunit.client.0.smithi031.stdout:8/802: rename d4/d7/f3d to d4/d5b/d8d/f102 0 2024-09-16T08:57:30.120 INFO:tasks.workunit.client.0.smithi031.stdout:6/977: rename d0/dd/d74 to d0/dd/d74/d5e/d7c/d3f/d65/d13d 22 2024-09-16T08:57:30.120 INFO:tasks.workunit.client.0.smithi031.stdout:8/803: readlink d4/d2f/dc3/d24/lf9 0 2024-09-16T08:57:30.120 INFO:tasks.workunit.client.0.smithi031.stdout:8/804: write d4/d7/d99/fc1 [1354063,95369] 0 2024-09-16T08:57:30.121 INFO:tasks.workunit.client.0.smithi031.stdout:6/978: write d0/d10/dd2/de3/f121 [167038,115019] 0 2024-09-16T08:57:30.122 INFO:tasks.workunit.client.0.smithi031.stdout:9/929: unlink d3/d5/d6a/d74/dd0/cf4 0 2024-09-16T08:57:30.122 INFO:tasks.workunit.client.0.smithi031.stdout:9/930: write f2 [3866918,41185] 0 2024-09-16T08:57:30.125 INFO:tasks.workunit.client.0.smithi031.stdout:3/923: mkdir d3/d14/d85/d113 0 2024-09-16T08:57:30.125 INFO:tasks.workunit.client.0.smithi031.stdout:3/924: fdatasync d3/d14/d37/d47/f5b 0 2024-09-16T08:57:30.125 INFO:tasks.workunit.client.0.smithi031.stdout:3/925: read - d3/d14/d37/d1a/d1d/d9a/dde/dd6/f10a zero size 2024-09-16T08:57:30.127 INFO:tasks.workunit.client.0.smithi031.stdout:4/942: getdents d0/d3c 0 2024-09-16T08:57:30.128 INFO:tasks.workunit.client.0.smithi031.stdout:7/895: mknod d2/d8/d11/d5d/d7e/c126 0 2024-09-16T08:57:30.128 INFO:tasks.workunit.client.0.smithi031.stdout:7/896: read d2/d2d/d6d/fb4 [2625321,92967] 0 2024-09-16T08:57:30.132 INFO:tasks.workunit.client.0.smithi031.stdout:0/938: unlink d6/d12/d47/d5b/c10a 0 2024-09-16T08:57:30.135 INFO:tasks.workunit.client.0.smithi031.stdout:2/849: truncate d6/f13 193283 0 2024-09-16T08:57:30.137 INFO:tasks.workunit.client.0.smithi031.stdout:5/809: getdents d2/dc/d1d 0 2024-09-16T08:57:30.139 INFO:tasks.workunit.client.0.smithi031.stdout:9/931: rmdir d3/d5/da/d34/d3c/d5e/d9c/dcd 39 2024-09-16T08:57:30.139 INFO:tasks.workunit.client.0.smithi031.stdout:9/932: fsync d3/d5/da/db/d21/d3b/dad/dd3/f13b 0 2024-09-16T08:57:30.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/926: rename d3/d14/d37/d47/c8d to d3/d14/d37/d1a/d1d/d3b/d3f/d79/c114 0 2024-09-16T08:57:30.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/927: fdatasync d3/d14/d37/d33/fd9 0 2024-09-16T08:57:30.141 INFO:tasks.workunit.client.0.smithi031.stdout:3/928: chown d3/d14/fb6 910571 1 2024-09-16T08:57:30.143 INFO:tasks.workunit.client.0.smithi031.stdout:2/850: mkdir d6/da/d37/deb/d117 0 2024-09-16T08:57:30.143 INFO:tasks.workunit.client.0.smithi031.stdout:2/851: readlink d6/d2e/d40/d58/dae/l105 0 2024-09-16T08:57:30.143 INFO:tasks.workunit.client.0.smithi031.stdout:5/810: mkdir d2/dc/d91/dee/d111 0 2024-09-16T08:57:30.144 INFO:tasks.workunit.client.0.smithi031.stdout:5/811: stat d2/dc/d3d/db2/cde 0 2024-09-16T08:57:30.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/933: truncate d3/d5/da/f6b 4807720 0 2024-09-16T08:57:30.146 INFO:tasks.workunit.client.0.smithi031.stdout:4/943: rename d0/d5b/f133 to d0/d102/d8d/f138 0 2024-09-16T08:57:30.146 INFO:tasks.workunit.client.0.smithi031.stdout:7/897: dwrite d2/d8/f63 [4194304,4194304] 0 2024-09-16T08:57:30.146 INFO:tasks.workunit.client.0.smithi031.stdout:7/898: stat d2/d8/d11/d81/ffe 0 2024-09-16T08:57:30.147 INFO:tasks.workunit.client.0.smithi031.stdout:6/979: dwrite d0/dd/d74/d19/d54/f9c [0,4194304] 0 2024-09-16T08:57:30.148 INFO:tasks.workunit.client.0.smithi031.stdout:2/852: unlink d6/d2e/l7e 0 2024-09-16T08:57:30.148 INFO:tasks.workunit.client.0.smithi031.stdout:2/853: read - d6/da/d37/d42/fbb zero size 2024-09-16T08:57:30.149 INFO:tasks.workunit.client.0.smithi031.stdout:5/812: creat d2/dc/d25/d68/f112 x:0 0 0 2024-09-16T08:57:30.149 INFO:tasks.workunit.client.0.smithi031.stdout:5/813: write d2/dc/d1d/fa1 [4308085,89248] 0 2024-09-16T08:57:30.150 INFO:tasks.workunit.client.0.smithi031.stdout:4/944: mkdir d0/d2/d16/d2b/d139 0 2024-09-16T08:57:30.151 INFO:tasks.workunit.client.0.smithi031.stdout:0/939: rename d6/d12/d13/d3c/f70 to d6/d12/d13/d11e/f12c 0 2024-09-16T08:57:30.152 INFO:tasks.workunit.client.0.smithi031.stdout:7/899: creat d2/d8/d11/d66/d9a/f127 x:0 0 0 2024-09-16T08:57:30.154 INFO:tasks.workunit.client.0.smithi031.stdout:2/854: symlink d6/da/l118 0 2024-09-16T08:57:30.155 INFO:tasks.workunit.client.0.smithi031.stdout:4/945: creat d0/d2/d16/d1e/db9/f13a x:0 0 0 2024-09-16T08:57:30.155 INFO:tasks.workunit.client.0.smithi031.stdout:3/929: dread d3/d14/fb6 [0,4194304] 0 2024-09-16T08:57:30.156 INFO:tasks.workunit.client.0.smithi031.stdout:0/940: creat d6/d12/d47/dc5/de6/f12d x:0 0 0 2024-09-16T08:57:30.157 INFO:tasks.workunit.client.0.smithi031.stdout:6/980: write d0/dd/d74/d5e/d7c/d3f/d92/ff4 [694182,45710] 0 2024-09-16T08:57:30.157 INFO:tasks.workunit.client.0.smithi031.stdout:7/900: symlink d2/d8/d86/da1/l128 0 2024-09-16T08:57:30.159 INFO:tasks.workunit.client.0.smithi031.stdout:2/855: rename d6/d2e/d40/d58/lc0 to d6/da/d37/deb/d117/l119 0 2024-09-16T08:57:30.159 INFO:tasks.workunit.client.0.smithi031.stdout:2/856: chown d6/d2e/d40/d6c/l86 62 1 2024-09-16T08:57:30.159 INFO:tasks.workunit.client.0.smithi031.stdout:2/857: truncate d6/da/d4b/d4c/df9/fd8 1030649 0 2024-09-16T08:57:30.162 INFO:tasks.workunit.client.0.smithi031.stdout:4/946: unlink d0/fa 0 2024-09-16T08:57:30.162 INFO:tasks.workunit.client.0.smithi031.stdout:4/947: chown d0/d102/d84/d86/d9b/fa4 14439892 1 2024-09-16T08:57:30.163 INFO:tasks.workunit.client.0.smithi031.stdout:3/930: mknod d3/d14/d37/d47/c115 0 2024-09-16T08:57:30.163 INFO:tasks.workunit.client.0.smithi031.stdout:3/931: stat d3/d14/d85 0 2024-09-16T08:57:30.165 INFO:tasks.workunit.client.0.smithi031.stdout:0/941: unlink d6/d12/d13/f24 0 2024-09-16T08:57:30.166 INFO:tasks.workunit.client.0.smithi031.stdout:6/981: symlink d0/dd/d74/d19/df9/d12c/l13e 0 2024-09-16T08:57:30.167 INFO:tasks.workunit.client.0.smithi031.stdout:7/901: mkdir d2/d8/d86/dd4/df1/d129 0 2024-09-16T08:57:30.170 INFO:tasks.workunit.client.0.smithi031.stdout:2/858: truncate d6/da/f27 3509613 0 2024-09-16T08:57:30.172 INFO:tasks.workunit.client.0.smithi031.stdout:3/932: creat d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/dce/f116 x:0 0 0 2024-09-16T08:57:30.174 INFO:tasks.workunit.client.0.smithi031.stdout:6/982: mkdir d0/d10/dd2/de3/d13f 0 2024-09-16T08:57:30.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/934: dwrite d3/d5/dd6/de0/ff6 [0,4194304] 0 2024-09-16T08:57:30.176 INFO:tasks.workunit.client.0.smithi031.stdout:5/814: dwrite d2/dc/d25/d21/fab [0,4194304] 0 2024-09-16T08:57:30.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/935: write d3/d5/da/d34/f136 [1083459,1603] 0 2024-09-16T08:57:30.176 INFO:tasks.workunit.client.0.smithi031.stdout:9/936: chown d3/d5/d73/fa0 2310 1 2024-09-16T08:57:30.176 INFO:tasks.workunit.client.0.smithi031.stdout:7/902: link d2/d8/d11/d66/d9a/d9c/fba d2/d8/d11/d5d/d6e/f12a 0 2024-09-16T08:57:30.179 INFO:tasks.workunit.client.0.smithi031.stdout:9/937: fsync f2 0 2024-09-16T08:57:30.179 INFO:tasks.workunit.client.0.smithi031.stdout:3/933: rename d3/d14/d37/d1a/d24/c36 to d3/d14/d37/d1a/d1d/d3b/d3f/d79/d7c/c117 0 2024-09-16T08:57:30.188 INFO:tasks.workunit.client.0.smithi031.stdout:7/903: rmdir d2/d8/d86/da1 39 2024-09-16T08:57:30.188 INFO:tasks.workunit.client.0.smithi031.stdout:4/948: dwrite d0/d2/d16/d1e/f9e [0,4194304] 0 2024-09-16T08:57:30.188 INFO:tasks.workunit.client.0.smithi031.stdout:4/949: write d0/db5/fb7 [2390036,52545] 0 2024-09-16T08:57:30.188 INFO:tasks.workunit.client.0.smithi031.stdout:4/950: creat d0/d102/d84/d86/d9b/dcf/dde/f13b x:0 0 0 2024-09-16T08:57:30.188 INFO:tasks.workunit.client.0.smithi031.stdout:4/951: chown d0/d2/d16/d1e/cb2 131990895 1 2024-09-16T08:57:30.188 INFO:tasks.workunit.client.0.smithi031.stdout:4/952: chown d0/d2/d16/d1e/c2f 496279 1 2024-09-16T08:57:30.191 INFO:tasks.workunit.client.0.smithi031.stdout:9/938: mkdir d3/d5/d6a/d74/da8/d100/d148 0 2024-09-16T08:57:30.191 INFO:tasks.workunit.client.0.smithi031.stdout:9/939: dread - d3/d5/d73/df5/f123 zero size 2024-09-16T08:57:30.194 INFO:tasks.workunit.client.0.smithi031.stdout:5/815: mkdir d2/dc/d25/d21/d113 0 2024-09-16T08:57:30.199 INFO:tasks.workunit.client.0.smithi031.stdout:6/983: dwrite d0/dd/d74/d19/d54/d12a/f134 [0,4194304] 0 2024-09-16T08:57:30.200 INFO:tasks.workunit.client.0.smithi031.stdout:6/984: chown d0/c46 808761 1 2024-09-16T08:57:30.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/953: symlink d0/d5b/l13c 0 2024-09-16T08:57:30.200 INFO:tasks.workunit.client.0.smithi031.stdout:4/954: readlink d0/d2/d16/d2b/daf/ld1 0 2024-09-16T08:57:30.201 INFO:tasks.workunit.client.0.smithi031.stdout:3/934: mkdir d3/d14/d37/d1a/d1d/d3b/d118 0 2024-09-16T08:57:30.201 INFO:tasks.workunit.client.0.smithi031.stdout:2/859: dwrite d6/d2e/d40/d6c/d2c/d53/f10d [0,4194304] 0 2024-09-16T08:57:30.202 INFO:tasks.workunit.client.0.smithi031.stdout:2/860: fsync d6/d20/f3f 0 2024-09-16T08:57:30.204 INFO:tasks.workunit.client.0.smithi031.stdout:7/904: dwrite d2/d8/d69/d82/fd0 [0,4194304] 0 2024-09-16T08:57:30.206 INFO:tasks.workunit.client.0.smithi031.stdout:9/940: write d3/d5/da/d34/f46 [2728776,111936] 0 2024-09-16T08:57:30.211 INFO:tasks.workunit.client.0.smithi031.stdout:6/985: unlink d0/dd/d74/d5e/d110/f89 0 2024-09-16T08:57:30.211 INFO:tasks.workunit.client.0.smithi031.stdout:6/986: readlink d0/dd/d2b/l95 0 2024-09-16T08:57:30.211 INFO:tasks.workunit.client.0.smithi031.stdout:5/816: dread d2/dc/d25/d21/f26 [0,4194304] 0 2024-09-16T08:57:30.211 INFO:tasks.workunit.client.0.smithi031.stdout:5/817: write d2/dc/d36/d76/f9a [99819,49631] 0 2024-09-16T08:57:30.212 INFO:tasks.workunit.client.0.smithi031.stdout:5/818: creat d2/dc/d91/f114 x:0 0 0 2024-09-16T08:57:30.212 INFO:tasks.workunit.client.0.smithi031.stdout:4/955: mkdir d0/d102/dd5/d137/d13d 0 2024-09-16T08:57:30.212 INFO:tasks.workunit.client.0.smithi031.stdout:2/861: dread d6/d2e/d40/d6c/d2c/d53/d60/f39 [0,4194304] 0 2024-09-16T08:57:30.212 INFO:tasks.workunit.client.0.smithi031.stdout:2/862: dread - d6/da/d37/fe5 zero size 2024-09-16T08:57:30.212 INFO:tasks.workunit.client.0.smithi031.stdout:2/863: fsync d6/d20/d51/f62 0 2024-09-16T08:57:30.213 INFO:tasks.workunit.client.0.smithi031.stdout:2/864: creat d6/da/d4b/d8f/f11a x:0 0 0 2024-09-16T08:57:30.214 INFO:tasks.workunit.client.0.smithi031.stdout:2/865: write d6/d20/d51/ddc/fe6 [71603,30268] 0 2024-09-16T08:57:30.217 INFO:tasks.workunit.client.0.smithi031.stdout:2/866: dread d6/f9 [0,4194304] 0 2024-09-16T08:57:30.223 INFO:tasks.workunit.client.0.smithi031.stdout:7/905: mknod d2/d2d/d68/d113/d114/c12b 0 2024-09-16T08:57:30.223 INFO:tasks.workunit.client.0.smithi031.stdout:7/906: truncate d2/d8/d11/d5d/d6e/f111 875655 0 2024-09-16T08:57:30.228 INFO:tasks.workunit.client.0.smithi031.stdout:3/935: dwrite f1 [4194304,4194304] 0 2024-09-16T08:57:30.229 INFO:tasks.workunit.client.0.smithi031.stdout:3/936: write d3/d14/d37/d33/f6d [5152711,5793] 0 2024-09-16T08:57:30.229 INFO:tasks.workunit.client.0.smithi031.stdout:3/937: write d3/ffc [3942518,92806] 0 2024-09-16T08:57:30.229 INFO:tasks.workunit.client.0.smithi031.stdout:3/938: creat d3/d14/d37/d1a/d24/f119 x:0 0 0 2024-09-16T08:57:30.229 INFO:tasks.workunit.client.0.smithi031.stdout:6/987: mknod d0/dd/d74/d19/df9/c140 0 2024-09-16T08:57:30.229 INFO:tasks.workunit.client.0.smithi031.stdout:6/988: write d0/dd/d74/d19/fdf [5070745,125912] 0 2024-09-16T08:57:30.229 INFO:tasks.workunit.client.0.smithi031.stdout:6/989: chown d0/dd/d74/d5e/d110/cf2 51 1 2024-09-16T08:57:30.232 INFO:tasks.workunit.client.0.smithi031.stdout:5/819: mkdir d2/dc/d25/dad/d115 0 2024-09-16T08:57:30.234 INFO:tasks.workunit.client.0.smithi031.stdout:4/956: truncate d0/d2/d16/d1e/f3b 513221 0 2024-09-16T08:57:30.239 INFO:tasks.workunit.client.0.smithi031.stdout:6/990: read d0/dd/d74/f7e [1767210,60313] 0 2024-09-16T08:57:30.246 INFO:tasks.workunit.client.0.smithi031.stdout:7/907: link d2/d8/d11/d5d/d6e/f11b d2/d8/d86/da1/f12c 0 2024-09-16T08:57:30.246 INFO:tasks.workunit.client.0.smithi031.stdout:7/908: fsync d2/d2d/d6d/ffb 0 2024-09-16T08:57:30.246 INFO:tasks.workunit.client.0.smithi031.stdout:7/909: creat d2/d8/d11/d66/d9a/f12d x:0 0 0 2024-09-16T08:57:30.246 INFO:tasks.workunit.client.0.smithi031.stdout:7/910: chown d2/d8/d69/d82/c85 24 1 2024-09-16T08:57:30.247 INFO:tasks.workunit.client.0.smithi031.stdout:9/941: mknod d3/d5/da/d34/d3c/d5e/d9c/d95/c149 0 2024-09-16T08:57:30.255 INFO:tasks.workunit.client.0.smithi031.stdout:6/991: dwrite d0/dd/d74/da6/fc3 [0,4194304] 0 2024-09-16T08:57:30.262 INFO:tasks.workunit.client.0.smithi031.stdout:4/957: symlink d0/d2/d16/ddd/l13e 0 2024-09-16T08:57:30.269 INFO:tasks.workunit.client.0.smithi031.stdout:2/867: truncate d6/da/f1e 658530 0 2024-09-16T08:57:30.271 INFO:tasks.workunit.client.0.smithi031.stdout:7/911: creat d2/d8/d86/da1/f12e x:0 0 0 2024-09-16T08:57:30.272 INFO:tasks.workunit.client.0.smithi031.stdout:7/912: stat d2/d8/d86/d123 0 2024-09-16T08:57:30.273 INFO:tasks.workunit.client.0.smithi031.stdout:9/942: truncate d3/d48/f9d 3910407 0 2024-09-16T08:57:30.280 INFO:tasks.workunit.client.0.smithi031.stdout:7/913: write d2/d2d/ff4 [3122124,110096] 0 2024-09-16T08:57:30.284 INFO:tasks.workunit.client.0.smithi031.stdout:6/992: mknod d0/dd/d74/d19/df9/c141 0 2024-09-16T08:57:30.285 INFO:tasks.workunit.client.0.smithi031.stdout:6/993: fdatasync d0/dd/d74/d19/d9a/fc0 0 2024-09-16T08:57:30.285 INFO:tasks.workunit.client.0.smithi031.stdout:6/994: creat d0/dd/d74/d5e/d7c/d3f/d92/f142 x:0 0 0 2024-09-16T08:57:30.290 INFO:tasks.workunit.client.0.smithi031.stdout:2/868: mkdir d6/d2e/d40/d6c/d2c/d53/d60/d11b 0 2024-09-16T08:57:30.291 INFO:tasks.workunit.client.0.smithi031.stdout:4/958: dread d0/d2/d16/d1e/f52 [0,4194304] 0 2024-09-16T08:57:30.295 INFO:tasks.workunit.client.0.smithi031.stdout:9/943: creat d3/d5/da/db/d21/f14a x:0 0 0 2024-09-16T08:57:30.295 INFO:tasks.workunit.client.0.smithi031.stdout:9/944: write d3/d5/da/d34/d3c/f103 [4720655,65233] 0 2024-09-16T08:57:30.295 INFO:tasks.workunit.client.0.smithi031.stdout:9/945: chown d3/d5/da/db/d21/f3d 18 1 2024-09-16T08:57:30.295 INFO:tasks.workunit.client.0.smithi031.stdout:9/946: fdatasync d3/d5/da/f2c 0 2024-09-16T08:57:30.295 INFO:tasks.workunit.client.0.smithi031.stdout:9/947: readlink d3/d5/da/d34/d3c/laf 0 2024-09-16T08:57:30.299 INFO:tasks.workunit.client.0.smithi031.stdout:4/959: dread d0/d2/d16/f6e [0,4194304] 0 2024-09-16T08:57:30.299 INFO:tasks.workunit.client.0.smithi031.stdout:4/960: truncate d0/d2/d16/d2b/fd7 1045453 0 2024-09-16T08:57:30.301 INFO:tasks.workunit.client.0.smithi031.stdout:4/961: dread d0/d2/d16/ddd/de4/f103 [0,4194304] 0 2024-09-16T08:57:30.305 INFO:tasks.workunit.client.0.smithi031.stdout:7/914: mknod d2/d2d/d68/d113/d120/c12f 0 2024-09-16T08:57:30.307 INFO:tasks.workunit.client.0.smithi031.stdout:3/939: rename d3/d14/d85 to d3/d14/d37/d1a/d1d/d3b/d118/d11a 0 2024-09-16T08:57:30.307 INFO:tasks.workunit.client.0.smithi031.stdout:3/940: write d3/d14/d37/d1a/d24/fbf [768146,113777] 0 2024-09-16T08:57:30.307 INFO:tasks.workunit.client.0.smithi031.stdout:7/915: dread d2/d8/d11/d81/f9f [0,4194304] 0 2024-09-16T08:57:30.308 INFO:tasks.workunit.client.0.smithi031.stdout:3/941: read d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/fe5 [607820,64447] 0 2024-09-16T08:57:30.308 INFO:tasks.workunit.client.0.smithi031.stdout:3/942: creat d3/d14/d93/f11b x:0 0 0 2024-09-16T08:57:30.310 INFO:tasks.workunit.client.0.smithi031.stdout:6/995: unlink d0/dd/d74/d5e/fb1 0 2024-09-16T08:57:30.312 INFO:tasks.workunit.client.0.smithi031.stdout:2/869: mknod d6/da/c11c 0 2024-09-16T08:57:30.313 INFO:tasks.workunit.client.0.smithi031.stdout:9/948: rmdir d3/d5/d6a/d74/da8/dc3 39 2024-09-16T08:57:30.316 INFO:tasks.workunit.client.0.smithi031.stdout:4/962: rename d0/d102/l6c to d0/d3c/d6b/d9c/l13f 0 2024-09-16T08:57:30.316 INFO:tasks.workunit.client.0.smithi031.stdout:7/916: creat d2/d8/d11/d66/d9a/d121/f130 x:0 0 0 2024-09-16T08:57:30.318 INFO:tasks.workunit.client.0.smithi031.stdout:3/943: mknod d3/d14/d37/d1a/d1d/d3b/d3f/d49/c11c 0 2024-09-16T08:57:30.319 INFO:tasks.workunit.client.0.smithi031.stdout:6/996: getdents d0/dd/d74/d19 0 2024-09-16T08:57:30.320 INFO:tasks.workunit.client.0.smithi031.stdout:2/870: write d6/d2e/d40/d6c/d2c/d53/d60/f93 [524785,17655] 0 2024-09-16T08:57:30.320 INFO:tasks.workunit.client.0.smithi031.stdout:2/871: fsync d6/d2e/d40/d6c/f10f 0 2024-09-16T08:57:30.322 INFO:tasks.workunit.client.0.smithi031.stdout:7/917: mknod d2/d8/d11/d5d/d6e/c131 0 2024-09-16T08:57:30.322 INFO:tasks.workunit.client.0.smithi031.stdout:7/918: creat d2/d8/d11/d5d/d7e/db6/f132 x:0 0 0 2024-09-16T08:57:30.323 INFO:tasks.workunit.client.0.smithi031.stdout:3/944: rmdir d3/d14/d37/d33 39 2024-09-16T08:57:30.323 INFO:tasks.workunit.client.0.smithi031.stdout:3/945: chown d3/le 1 1 2024-09-16T08:57:30.323 INFO:tasks.workunit.client.0.smithi031.stdout:3/946: stat d3/d14/d37/d1a/d1d/d6a/lc9 0 2024-09-16T08:57:30.323 INFO:tasks.workunit.client.0.smithi031.stdout:6/997: creat d0/dd/d35/d6c/d12b/f143 x:0 0 0 2024-09-16T08:57:30.324 INFO:tasks.workunit.client.0.smithi031.stdout:9/949: rename d3/d5/da/d34/d4d/df1 to d3/d5/da/d12c/d14b 0 2024-09-16T08:57:30.326 INFO:tasks.workunit.client.0.smithi031.stdout:2/872: creat d6/d2e/d40/d58/f11d x:0 0 0 2024-09-16T08:57:30.326 INFO:tasks.workunit.client.0.smithi031.stdout:2/873: chown d6/d2e/ffa 4693 1 2024-09-16T08:57:30.329 INFO:tasks.workunit.client.0.smithi031.stdout:4/963: unlink d0/d2/d16/d1e/l5c 0 2024-09-16T08:57:30.332 INFO:tasks.workunit.client.0.smithi031.stdout:7/919: link d2/c47 d2/d8/d11/d81/c133 0 2024-09-16T08:57:30.335 INFO:tasks.workunit.client.0.smithi031.stdout:9/950: read d3/d5/da/d12c/d14b/f10d [1563174,87063] 0 2024-09-16T08:57:30.335 INFO:tasks.workunit.client.0.smithi031.stdout:9/951: creat d3/d5/dd6/f14c x:0 0 0 2024-09-16T08:57:30.335 INFO:tasks.workunit.client.0.smithi031.stdout:9/952: creat d3/d5/dd6/f14d x:0 0 0 2024-09-16T08:57:30.335 INFO:tasks.workunit.client.0.smithi031.stdout:3/947: rename d3/c27 to d3/d14/d37/d1a/d1d/d1e/db8/c11d 0 2024-09-16T08:57:30.337 INFO:tasks.workunit.client.0.smithi031.stdout:4/964: mknod d0/d2/d16/d2b/dfe/c140 0 2024-09-16T08:57:30.339 INFO:tasks.workunit.client.0.smithi031.stdout:6/998: dwrite d0/dd/d74/d5e/d7c/d3f/d92/f119 [0,4194304] 0 2024-09-16T08:57:30.342 INFO:tasks.workunit.client.0.smithi031.stdout:2/874: dread d6/da/d37/d8b/f8c [0,4194304] 0 2024-09-16T08:57:30.342 INFO:tasks.workunit.client.0.smithi031.stdout:2/875: fsync d6/d2e/d40/d58/dae/db1/fc4 0 2024-09-16T08:57:30.342 INFO:tasks.workunit.client.0.smithi031.stdout:2/876: stat f4 0 2024-09-16T08:57:30.343 INFO:tasks.workunit.client.0.smithi031.stdout:3/948: symlink d3/d14/d37/d33/d97/l11e 0 2024-09-16T08:57:30.344 INFO:tasks.workunit.client.0.smithi031.stdout:4/965: truncate d0/d2/d16/d1e/f3b 1423673 0 2024-09-16T08:57:30.344 INFO:tasks.workunit.client.0.smithi031.stdout:4/966: write d0/d5b/db8/df8/f11d [377078,94976] 0 2024-09-16T08:57:30.344 INFO:tasks.workunit.client.0.smithi031.stdout:4/967: creat d0/d5b/db8/df8/f141 x:0 0 0 2024-09-16T08:57:30.344 INFO:tasks.workunit.client.0.smithi031.stdout:4/968: chown d0/d3c/d6b/f5e 0 1 2024-09-16T08:57:30.346 INFO:tasks.workunit.client.0.smithi031.stdout:6/999: rmdir d0/d10/dc8/df6 39 2024-09-16T08:57:30.347 INFO:tasks.workunit.client.0.smithi031.stdout:2/877: mkdir d6/d20/da1/d11e 0 2024-09-16T08:57:30.351 INFO:tasks.workunit.client.0.smithi031.stdout:4/969: unlink d0/d2/f41 0 2024-09-16T08:57:30.351 INFO:tasks.workunit.client.0.smithi031.stdout:4/970: dread - d0/d109/d106/fe3 zero size 2024-09-16T08:57:30.352 INFO:tasks.workunit.client.0.smithi031.stdout:2/878: mknod d6/d20/da1/d11e/c11f 0 2024-09-16T08:57:30.352 INFO:tasks.workunit.client.0.smithi031.stdout:7/920: dwrite d2/d8/d11/d5d/d6e/f11b [0,4194304] 0 2024-09-16T08:57:30.353 INFO:tasks.workunit.client.0.smithi031.stdout:3/949: getdents d3/d14/d37/d1a/d1d/d3b/d60 0 2024-09-16T08:57:30.360 INFO:tasks.workunit.client.0.smithi031.stdout:3/950: mknod d3/d14/d37/d1a/d1d/d1e/c11f 0 2024-09-16T08:57:30.367 INFO:tasks.workunit.client.0.smithi031.stdout:3/951: rename d3/d14/d37/f84 to d3/d14/d37/d1a/d1d/d3b/d3f/f120 0 2024-09-16T08:57:30.367 INFO:tasks.workunit.client.0.smithi031.stdout:3/952: readlink d3/d14/d37/d1a/d1d/d6a/l6e 0 2024-09-16T08:57:30.367 INFO:tasks.workunit.client.0.smithi031.stdout:4/971: getdents d0/d2/d16/d1e/db9 0 2024-09-16T08:57:30.379 INFO:tasks.workunit.client.0.smithi031.stdout:4/972: dread d0/d109/d106/d55/f12b [0,4194304] 0 2024-09-16T08:57:30.381 INFO:tasks.workunit.client.0.smithi031.stdout:4/973: creat d0/d2/d16/d1e/db9/f142 x:0 0 0 2024-09-16T08:57:30.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/974: read - d0/d2/d16/d2b/daf/f108 zero size 2024-09-16T08:57:30.382 INFO:tasks.workunit.client.0.smithi031.stdout:4/975: truncate d0/f18 2918968 0 2024-09-16T08:57:30.383 INFO:tasks.workunit.client.0.smithi031.stdout:3/953: dwrite d3/d14/d37/d47/f66 [4194304,4194304] 0 2024-09-16T08:57:30.386 INFO:tasks.workunit.client.0.smithi031.stdout:3/954: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/l121 0 2024-09-16T08:57:30.388 INFO:tasks.workunit.client.0.smithi031.stdout:3/955: dread d3/d14/d37/d1a/f5c [0,4194304] 0 2024-09-16T08:57:30.390 INFO:tasks.workunit.client.0.smithi031.stdout:3/956: mknod d3/d14/d37/d1a/d1d/d3b/d3f/dd1/c122 0 2024-09-16T08:57:30.401 INFO:tasks.workunit.client.0.smithi031.stdout:9/953: write d3/d5/f2d [760550,98713] 0 2024-09-16T08:57:30.419 INFO:tasks.workunit.client.0.smithi031.stdout:4/976: dwrite d0/d2/d16/ddd/fdf [0,4194304] 0 2024-09-16T08:57:30.419 INFO:tasks.workunit.client.0.smithi031.stdout:3/957: dwrite d3/d14/d37/d47/f5b [0,4194304] 0 2024-09-16T08:57:30.422 INFO:tasks.workunit.client.0.smithi031.stdout:4/977: mknod d0/d71/c143 0 2024-09-16T08:57:30.422 INFO:tasks.workunit.client.0.smithi031.stdout:4/978: write d0/d102/d84/fa2 [1439421,89502] 0 2024-09-16T08:57:30.422 INFO:tasks.workunit.client.0.smithi031.stdout:3/958: mknod d3/d14/d37/d1a/d1d/d3b/d118/d11a/d113/c123 0 2024-09-16T08:57:30.423 INFO:tasks.workunit.client.0.smithi031.stdout:4/979: creat d0/db5/f144 x:0 0 0 2024-09-16T08:57:30.423 INFO:tasks.workunit.client.0.smithi031.stdout:4/980: creat d0/d2/d16/d2b/daf/f145 x:0 0 0 2024-09-16T08:57:30.424 INFO:tasks.workunit.client.0.smithi031.stdout:4/981: write d0/d2/d16/d2b/d10e/f119 [795483,99207] 0 2024-09-16T08:57:30.424 INFO:tasks.workunit.client.0.smithi031.stdout:3/959: unlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f90 0 2024-09-16T08:57:30.425 INFO:tasks.workunit.client.0.smithi031.stdout:9/954: dwrite d3/d5/da/db/fc0 [0,4194304] 0 2024-09-16T08:57:30.427 INFO:tasks.workunit.client.0.smithi031.stdout:7/921: dwrite d2/d8/d11/d5d/d7e/fb9 [0,4194304] 0 2024-09-16T08:57:30.430 INFO:tasks.workunit.client.0.smithi031.stdout:3/960: creat d3/d14/d37/d1a/d1d/d3b/d3f/d49/d87/f124 x:0 0 0 2024-09-16T08:57:30.430 INFO:tasks.workunit.client.0.smithi031.stdout:3/961: chown d3/d14/d37/d1a/d1d/d1e/c11f 298595 1 2024-09-16T08:57:30.430 INFO:tasks.workunit.client.0.smithi031.stdout:3/962: chown d3/d14/d37/d1a/d1d/d1e/c42 1591995998 1 2024-09-16T08:57:30.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/963: dread d3/d14/d37/d1a/d1d/f92 [0,4194304] 0 2024-09-16T08:57:30.431 INFO:tasks.workunit.client.0.smithi031.stdout:3/964: chown d3/d14/d37/d1a/d1d/d3b/f4f 1738 1 2024-09-16T08:57:30.431 INFO:tasks.workunit.client.0.smithi031.stdout:9/955: mknod d3/d5/da/d34/d3c/d5e/d62/c14e 0 2024-09-16T08:57:30.440 INFO:tasks.workunit.client.0.smithi031.stdout:7/922: dread d2/d8/d11/d66/d9a/fca [0,4194304] 0 2024-09-16T08:57:30.447 INFO:tasks.workunit.client.0.smithi031.stdout:3/965: read f1 [4264075,20711] 0 2024-09-16T08:57:30.450 INFO:tasks.workunit.client.0.smithi031.stdout:4/982: unlink d0/d2/d16/d1e/f3f 0 2024-09-16T08:57:30.452 INFO:tasks.workunit.client.0.smithi031.stdout:4/983: truncate d0/d102/d8d/fc1 3069407 0 2024-09-16T08:57:30.452 INFO:tasks.workunit.client.0.smithi031.stdout:4/984: write d0/d102/d84/d86/d9b/fa4 [2556631,26380] 0 2024-09-16T08:57:30.454 INFO:tasks.workunit.client.0.smithi031.stdout:4/985: mkdir d0/d71/d146 0 2024-09-16T08:57:30.455 INFO:tasks.workunit.client.0.smithi031.stdout:4/986: creat d0/d109/d106/f147 x:0 0 0 2024-09-16T08:57:30.455 INFO:tasks.workunit.client.0.smithi031.stdout:4/987: unlink d0/d102/d84/d86/d116/f12f 0 2024-09-16T08:57:30.455 INFO:tasks.workunit.client.0.smithi031.stdout:4/988: write d0/d5b/db8/dc8/f10b [2295883,84637] 0 2024-09-16T08:57:30.464 INFO:tasks.workunit.client.0.smithi031.stdout:9/956: dwrite d3/d5/da/d34/f131 [0,4194304] 0 2024-09-16T08:57:30.488 INFO:tasks.workunit.client.0.smithi031.stdout:7/923: dwrite d2/d8/d11/d81/f9f [0,4194304] 0 2024-09-16T08:57:30.488 INFO:tasks.workunit.client.0.smithi031.stdout:3/966: dwrite d3/d14/d37/d47/ded/f10d [0,4194304] 0 2024-09-16T08:57:30.488 INFO:tasks.workunit.client.0.smithi031.stdout:4/989: dwrite d0/d2/f7e [0,4194304] 0 2024-09-16T08:57:30.546 INFO:tasks.workunit.client.0.smithi031.stdout:9/957: creat d3/d5/d6a/d74/da8/dc3/dbc/f14f x:0 0 0 2024-09-16T08:57:30.551 INFO:tasks.workunit.client.0.smithi031.stdout:7/924: creat d2/d2d/d68/d113/d11c/f134 x:0 0 0 2024-09-16T08:57:30.566 INFO:tasks.workunit.client.0.smithi031.stdout:1/806: sync 2024-09-16T08:57:30.567 INFO:tasks.workunit.client.0.smithi031.stdout:8/805: sync 2024-09-16T08:57:30.567 INFO:tasks.workunit.client.0.smithi031.stdout:8/806: fdatasync d4/d2f/dc3/f3b 0 2024-09-16T08:57:30.567 INFO:tasks.workunit.client.0.smithi031.stdout:8/807: readlink d4/d5b/d7b/l8b 0 2024-09-16T08:57:30.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/925: dread d2/d2d/f43 [0,4194304] 0 2024-09-16T08:57:30.569 INFO:tasks.workunit.client.0.smithi031.stdout:9/958: unlink d3/d5/d73/fa0 0 2024-09-16T08:57:30.572 INFO:tasks.workunit.client.0.smithi031.stdout:4/990: rmdir d0/d2/d16/d1e/db9 39 2024-09-16T08:57:30.572 INFO:tasks.workunit.client.0.smithi031.stdout:4/991: fdatasync d0/f18 0 2024-09-16T08:57:30.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/808: dread d4/d9e/d38/f3c [0,4194304] 0 2024-09-16T08:57:30.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/809: fdatasync d4/d2f/dc3/d24/f31 0 2024-09-16T08:57:30.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/810: chown d4/d2f/f95 26019 1 2024-09-16T08:57:30.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/811: write d4/d2f/dc3/d37/d46/fd2 [933635,64290] 0 2024-09-16T08:57:30.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/812: write d4/d5b/d8d/dcd/d50/fe7 [244756,77881] 0 2024-09-16T08:57:30.574 INFO:tasks.workunit.client.0.smithi031.stdout:8/813: dread - d4/d9e/d5e/fed zero size 2024-09-16T08:57:30.575 INFO:tasks.workunit.client.0.smithi031.stdout:8/814: write d4/d2f/fbd [95643,112312] 0 2024-09-16T08:57:30.575 INFO:tasks.workunit.client.0.smithi031.stdout:8/815: stat d4/d2f/dc3/d37/d46/db1/f6a 0 2024-09-16T08:57:30.575 INFO:tasks.workunit.client.0.smithi031.stdout:1/807: truncate d2/d15/d1f/de9/f31 778170 0 2024-09-16T08:57:30.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/926: getdents d2/d8/d86/dd4 0 2024-09-16T08:57:30.576 INFO:tasks.workunit.client.0.smithi031.stdout:7/927: truncate d2/d8/d86/da1/da9/fd3 771575 0 2024-09-16T08:57:30.576 INFO:tasks.workunit.client.0.smithi031.stdout:4/992: dread d0/d5b/db8/df8/f11d [0,4194304] 0 2024-09-16T08:57:30.581 INFO:tasks.workunit.client.0.smithi031.stdout:9/959: rename d3/d5/d6a/c8a to d3/d5/d6a/d74/da8/dc3/dbc/d137/c150 0 2024-09-16T08:57:30.581 INFO:tasks.workunit.client.0.smithi031.stdout:9/960: dread - d3/d5/dd6/f107 zero size 2024-09-16T08:57:30.584 INFO:tasks.workunit.client.0.smithi031.stdout:8/816: dread d4/f29 [0,4194304] 0 2024-09-16T08:57:30.584 INFO:tasks.workunit.client.0.smithi031.stdout:8/817: chown d4/d2f/dc3/d37/d46/db1/c71 42915312 1 2024-09-16T08:57:30.585 INFO:tasks.workunit.client.0.smithi031.stdout:7/928: mknod d2/d8/d11/d5d/c135 0 2024-09-16T08:57:30.585 INFO:tasks.workunit.client.0.smithi031.stdout:4/993: dread d0/d102/f5d [0,4194304] 0 2024-09-16T08:57:30.586 INFO:tasks.workunit.client.0.smithi031.stdout:1/808: getdents d2/d15/dff 0 2024-09-16T08:57:30.590 INFO:tasks.workunit.client.0.smithi031.stdout:8/818: dread d4/d5b/d8d/dcd/d79/f8a [0,4194304] 0 2024-09-16T08:57:30.590 INFO:tasks.workunit.client.0.smithi031.stdout:8/819: readlink d4/d5b/d8d/dcd/d79/la7 0 2024-09-16T08:57:30.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/929: mkdir d2/d8/d11/d81/d10b/d136 0 2024-09-16T08:57:30.591 INFO:tasks.workunit.client.0.smithi031.stdout:7/930: truncate d2/d2d/d6d/ff5 1851626 0 2024-09-16T08:57:30.598 INFO:tasks.workunit.client.0.smithi031.stdout:4/994: dread d0/d2/d16/d1e/f3b [0,4194304] 0 2024-09-16T08:57:30.599 INFO:tasks.workunit.client.0.smithi031.stdout:4/995: getdents d0/d5b/db8 0 2024-09-16T08:57:30.602 INFO:tasks.workunit.client.0.smithi031.stdout:1/809: rename d2/d15/d1f/de9/f31 to d2/d15/d1f/de9/d1b/d5c/d6b/dc1/df0/f119 0 2024-09-16T08:57:30.606 INFO:tasks.workunit.client.0.smithi031.stdout:8/820: rename d4/c9 to d4/d7/d66/c103 0 2024-09-16T08:57:30.608 INFO:tasks.workunit.client.0.smithi031.stdout:8/821: write d4/d5b/d8d/dcd/fbe [583174,80876] 0 2024-09-16T08:57:30.609 INFO:tasks.workunit.client.0.smithi031.stdout:8/822: symlink d4/d7/d66/dfa/l104 0 2024-09-16T08:57:30.612 INFO:tasks.workunit.client.0.smithi031.stdout:8/823: mkdir d4/d2f/dc3/d37/d46/d105 0 2024-09-16T08:57:30.612 INFO:tasks.workunit.client.0.smithi031.stdout:8/824: read - d4/d9e/d38/fda zero size 2024-09-16T08:57:30.620 INFO:tasks.workunit.client.0.smithi031.stdout:4/996: dwrite d0/d2/d16/f4f [0,4194304] 0 2024-09-16T08:57:30.635 INFO:tasks.workunit.client.0.smithi031.stdout:8/825: dwrite d4/d77/ddc/ff4 [0,4194304] 0 2024-09-16T08:57:30.637 INFO:tasks.workunit.client.0.smithi031.stdout:4/997: dwrite d0/db5/feb [0,4194304] 0 2024-09-16T08:57:30.637 INFO:tasks.workunit.client.0.smithi031.stdout:4/998: read d0/f2d [2503542,42699] 0 2024-09-16T08:57:30.641 INFO:tasks.workunit.client.0.smithi031.stdout:8/826: rename d4/db8/lf8 to d4/d5b/d8d/l106 0 2024-09-16T08:57:30.641 INFO:tasks.workunit.client.0.smithi031.stdout:8/827: fsync d4/d9e/d38/f73 0 2024-09-16T08:57:30.666 INFO:tasks.workunit.client.0.smithi031.stdout:4/999: dwrite d0/d71/fcd [0,4194304] 0 2024-09-16T08:57:30.668 INFO:tasks.workunit.client.0.smithi031.stdout:8/828: dwrite d4/d9e/d38/f73 [0,4194304] 0 2024-09-16T08:57:30.671 INFO:tasks.workunit.client.0.smithi031.stdout:8/829: link d4/d9e/d38/c61 d4/d2f/c107 0 2024-09-16T08:57:30.671 INFO:tasks.workunit.client.0.smithi031.stdout:8/830: fsync d4/f43 0 2024-09-16T08:57:30.673 INFO:tasks.workunit.client.0.smithi031.stdout:8/831: mkdir d4/d7/d99/d108 0 2024-09-16T08:57:30.674 INFO:tasks.workunit.client.0.smithi031.stdout:8/832: read d4/d7/f15 [2830582,43935] 0 2024-09-16T08:57:30.675 INFO:tasks.workunit.client.0.smithi031.stdout:8/833: mknod d4/d7/d66/de1/c109 0 2024-09-16T08:57:30.677 INFO:tasks.workunit.client.0.smithi031.stdout:8/834: rename d4/d2f/dc3/d37/d46/fff to d4/d2d/f10a 0 2024-09-16T08:57:30.678 INFO:tasks.workunit.client.0.smithi031.stdout:8/835: write d4/d9e/d38/f3c [717604,7197] 0 2024-09-16T08:57:30.709 INFO:tasks.workunit.client.0.smithi031.stdout:0/942: sync 2024-09-16T08:57:30.709 INFO:tasks.workunit.client.0.smithi031.stdout:0/943: dread d6/d12/d13/da5/fc0 [4194304,4194304] 0 2024-09-16T08:57:30.710 INFO:tasks.workunit.client.0.smithi031.stdout:0/944: chown d6/d12/d47/d5b/d9b/fec 22407069 1 2024-09-16T08:57:30.713 INFO:tasks.workunit.client.0.smithi031.stdout:0/945: dread d6/d12/d13/d36/fd1 [0,4194304] 0 2024-09-16T08:57:30.731 INFO:tasks.workunit.client.0.smithi031.stdout:0/946: dwrite d6/d12/d47/f83 [0,4194304] 0 2024-09-16T08:57:30.748 INFO:tasks.workunit.client.0.smithi031.stdout:0/947: dwrite d6/d12/d47/dc5/dc9/d10f/fc1 [0,4194304] 0 2024-09-16T08:57:30.751 INFO:tasks.workunit.client.0.smithi031.stdout:0/948: unlink d6/d12/d97/ccb 0 2024-09-16T08:57:30.764 INFO:tasks.workunit.client.0.smithi031.stdout:0/949: dwrite d6/d12/d97/d9e/fb8 [0,4194304] 0 2024-09-16T08:57:30.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/950: link d6/d12/d13/c88 d6/d12/d13/dde/c12e 0 2024-09-16T08:57:30.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/951: readlink d6/d12/d97/db3/dba/l11d 0 2024-09-16T08:57:30.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/952: fdatasync d6/d12/d47/d5b/d9b/da2/fab 0 2024-09-16T08:57:30.769 INFO:tasks.workunit.client.0.smithi031.stdout:0/953: creat d6/dac/f12f x:0 0 0 2024-09-16T08:57:30.771 INFO:tasks.workunit.client.0.smithi031.stdout:0/954: mkdir d6/d12/d47/dc5/de6/d130 0 2024-09-16T08:57:30.771 INFO:tasks.workunit.client.0.smithi031.stdout:0/955: stat d6/f74 0 2024-09-16T08:57:30.774 INFO:tasks.workunit.client.0.smithi031.stdout:0/956: unlink d6/d12/d13/dde/c126 0 2024-09-16T08:57:30.774 INFO:tasks.workunit.client.0.smithi031.stdout:0/957: stat d6/d12/d13/c14 0 2024-09-16T08:57:30.774 INFO:tasks.workunit.client.0.smithi031.stdout:0/958: creat d6/d12/d13/dca/deb/f131 x:0 0 0 2024-09-16T08:57:30.777 INFO:tasks.workunit.client.0.smithi031.stdout:0/959: mkdir d6/d12/d13/dca/deb/d132 0 2024-09-16T08:57:30.785 INFO:tasks.workunit.client.0.smithi031.stdout:0/960: write d6/d12/d13/f39 [3536453,110235] 0 2024-09-16T08:57:30.790 INFO:tasks.workunit.client.0.smithi031.stdout:0/961: write d6/d12/d47/d4d/fae [3043139,87071] 0 2024-09-16T08:57:30.813 INFO:tasks.workunit.client.0.smithi031.stdout:0/962: dwrite d6/d12/d47/dc5/dc9/d10f/f109 [0,4194304] 0 2024-09-16T08:57:30.813 INFO:tasks.workunit.client.0.smithi031.stdout:0/963: chown d6/d12/d13/c80 5695422 1 2024-09-16T08:57:30.813 INFO:tasks.workunit.client.0.smithi031.stdout:0/964: stat d6/f1a 0 2024-09-16T08:57:30.816 INFO:tasks.workunit.client.0.smithi031.stdout:0/965: getdents d6/dd3 0 2024-09-16T08:57:30.817 INFO:tasks.workunit.client.0.smithi031.stdout:0/966: readlink d6/d12/d47/d5b/l87 0 2024-09-16T08:57:30.817 INFO:tasks.workunit.client.0.smithi031.stdout:0/967: readlink d6/d61/lf4 0 2024-09-16T08:57:30.817 INFO:tasks.workunit.client.0.smithi031.stdout:0/968: fdatasync d6/f34 0 2024-09-16T08:57:30.817 INFO:tasks.workunit.client.0.smithi031.stdout:0/969: dread - d6/d12/d97/f118 zero size 2024-09-16T08:57:30.817 INFO:tasks.workunit.client.0.smithi031.stdout:0/970: write d6/f1e [2867989,14774] 0 2024-09-16T08:57:30.891 INFO:tasks.workunit.client.0.smithi031.stdout:0/971: truncate d6/d12/d13/d36/f96 136672 0 2024-09-16T08:57:30.894 INFO:tasks.workunit.client.0.smithi031.stdout:0/972: creat d6/d61/f133 x:0 0 0 2024-09-16T08:57:30.894 INFO:tasks.workunit.client.0.smithi031.stdout:0/973: chown d6/d12/d13/cea 8322714 1 2024-09-16T08:57:30.928 INFO:tasks.workunit.client.0.smithi031.stdout:0/974: truncate d6/d12/d13/f21 220777 0 2024-09-16T08:57:30.930 INFO:tasks.workunit.client.0.smithi031.stdout:0/975: write d6/d12/d47/d5b/fee [218826,35373] 0 2024-09-16T08:57:30.933 INFO:tasks.workunit.client.0.smithi031.stdout:0/976: rmdir d6/d12/d13/d3c 39 2024-09-16T08:57:30.936 INFO:tasks.workunit.client.0.smithi031.stdout:0/977: rename d6/d12/d47/d5b/d9b/f10c to d6/d12/d97/d9e/f134 0 2024-09-16T08:57:30.937 INFO:tasks.workunit.client.0.smithi031.stdout:0/978: readlink d6/d12/d97/db3/dba/l11d 0 2024-09-16T08:57:30.938 INFO:tasks.workunit.client.0.smithi031.stdout:0/979: mkdir d6/d12/d13/dca/deb/d135 0 2024-09-16T08:57:30.944 INFO:tasks.workunit.client.0.smithi031.stdout:0/980: dread d6/d12/d13/d36/fd8 [0,4194304] 0 2024-09-16T08:57:30.945 INFO:tasks.workunit.client.0.smithi031.stdout:0/981: symlink d6/dac/l136 0 2024-09-16T08:57:31.005 INFO:tasks.workunit.client.0.smithi031.stdout:5/820: sync 2024-09-16T08:57:31.006 INFO:tasks.workunit.client.0.smithi031.stdout:5/821: write d2/dc/d25/f53 [444867,93197] 0 2024-09-16T08:57:31.007 INFO:tasks.workunit.client.0.smithi031.stdout:5/822: mkdir d2/dc/d8c/d116 0 2024-09-16T08:57:31.007 INFO:tasks.workunit.client.0.smithi031.stdout:5/823: truncate d2/d4b/f8a 4385942 0 2024-09-16T08:57:31.009 INFO:tasks.workunit.client.0.smithi031.stdout:5/824: mkdir d2/dc/d25/dbf/de4/d117 0 2024-09-16T08:57:31.009 INFO:tasks.workunit.client.0.smithi031.stdout:5/825: getdents d2/dc/d25/dad/d115 0 2024-09-16T08:57:31.113 INFO:tasks.workunit.client.0.smithi031.stdout:3/967: sync 2024-09-16T08:57:31.113 INFO:tasks.workunit.client.0.smithi031.stdout:2/879: sync 2024-09-16T08:57:31.113 INFO:tasks.workunit.client.0.smithi031.stdout:7/931: sync 2024-09-16T08:57:31.113 INFO:tasks.workunit.client.0.smithi031.stdout:8/836: sync 2024-09-16T08:57:31.113 INFO:tasks.workunit.client.0.smithi031.stdout:9/961: sync 2024-09-16T08:57:31.113 INFO:tasks.workunit.client.0.smithi031.stdout:1/810: sync 2024-09-16T08:57:31.114 INFO:tasks.workunit.client.0.smithi031.stdout:7/932: creat d2/d8/d11/d66/d9a/d121/f137 x:0 0 0 2024-09-16T08:57:31.114 INFO:tasks.workunit.client.0.smithi031.stdout:1/811: creat d2/d15/d1f/d43/f11a x:0 0 0 2024-09-16T08:57:31.115 INFO:tasks.workunit.client.0.smithi031.stdout:8/837: mkdir d4/d2d/d6e/d10b 0 2024-09-16T08:57:31.117 INFO:tasks.workunit.client.0.smithi031.stdout:8/838: dread d4/d2f/dc3/d37/fc4 [0,4194304] 0 2024-09-16T08:57:31.117 INFO:tasks.workunit.client.0.smithi031.stdout:8/839: creat d4/d2d/d6e/f10c x:0 0 0 2024-09-16T08:57:31.119 INFO:tasks.workunit.client.0.smithi031.stdout:7/933: rmdir d2/d8/d86/dd4/df1 39 2024-09-16T08:57:31.119 INFO:tasks.workunit.client.0.smithi031.stdout:7/934: creat d2/d8/d69/d82/f138 x:0 0 0 2024-09-16T08:57:31.119 INFO:tasks.workunit.client.0.smithi031.stdout:7/935: chown d2/d8/d11/f40 2460 1 2024-09-16T08:57:31.119 INFO:tasks.workunit.client.0.smithi031.stdout:7/936: fsync d2/d8/f57 0 2024-09-16T08:57:31.120 INFO:tasks.workunit.client.0.smithi031.stdout:9/962: unlink d3/d5/da/d34/d4d/f66 0 2024-09-16T08:57:31.122 INFO:tasks.workunit.client.0.smithi031.stdout:3/968: read d3/d14/d37/f10 [62451,95429] 0 2024-09-16T08:57:31.122 INFO:tasks.workunit.client.0.smithi031.stdout:3/969: truncate d3/d14/d37/d1a/d1d/d3b/fec 571526 0 2024-09-16T08:57:31.122 INFO:tasks.workunit.client.0.smithi031.stdout:3/970: stat d3/d14/d37/d1a/d1d/d9a 0 2024-09-16T08:57:31.123 INFO:tasks.workunit.client.0.smithi031.stdout:2/880: write d6/d2e/d40/d6c/d2c/f64 [479789,8216] 0 2024-09-16T08:57:31.123 INFO:tasks.workunit.client.0.smithi031.stdout:2/881: readlink d6/da/d37/l6b 0 2024-09-16T08:57:31.123 INFO:tasks.workunit.client.0.smithi031.stdout:7/937: unlink d2/d8/d11/d5d/d6e/f71 0 2024-09-16T08:57:31.123 INFO:tasks.workunit.client.0.smithi031.stdout:7/938: readlink d2/d8/d11/d5d/d7e/db6/lbe 0 2024-09-16T08:57:31.123 INFO:tasks.workunit.client.0.smithi031.stdout:7/939: readlink d2/d2d/d6d/lb1 0 2024-09-16T08:57:31.126 INFO:tasks.workunit.client.0.smithi031.stdout:3/971: rename d3/d14/d37/d33/d97/le6 to d3/d14/d37/d1a/d24/l125 0 2024-09-16T08:57:31.128 INFO:tasks.workunit.client.0.smithi031.stdout:2/882: mkdir d6/d20/da1/dcf/dda/de3/d120 0 2024-09-16T08:57:31.128 INFO:tasks.workunit.client.0.smithi031.stdout:1/812: write d2/d15/d38/fb2 [2363200,81134] 0 2024-09-16T08:57:31.131 INFO:tasks.workunit.client.0.smithi031.stdout:7/940: mkdir d2/d8/d86/da1/d139 0 2024-09-16T08:57:31.132 INFO:tasks.workunit.client.0.smithi031.stdout:9/963: rmdir d3/d5/da/db/dcb 39 2024-09-16T08:57:31.133 INFO:tasks.workunit.client.0.smithi031.stdout:9/964: readlink d3/d5/da/db/d21/lb6 0 2024-09-16T08:57:31.136 INFO:tasks.workunit.client.0.smithi031.stdout:7/941: dread d2/d8/d11/d5d/f73 [0,4194304] 0 2024-09-16T08:57:31.136 INFO:tasks.workunit.client.0.smithi031.stdout:2/883: creat d6/f121 x:0 0 0 2024-09-16T08:57:31.140 INFO:tasks.workunit.client.0.smithi031.stdout:8/840: dwrite d4/d9e/f7e [0,4194304] 0 2024-09-16T08:57:31.140 INFO:tasks.workunit.client.0.smithi031.stdout:3/972: dread d3/d14/d37/d1a/d1d/d1e/ff8 [0,4194304] 0 2024-09-16T08:57:31.140 INFO:tasks.workunit.client.0.smithi031.stdout:3/973: dread - d3/d14/d37/d1a/d1d/d3b/d94/fe9 zero size 2024-09-16T08:57:31.141 INFO:tasks.workunit.client.0.smithi031.stdout:1/813: creat d2/d15/d1f/f11b x:0 0 0 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/965: rename d3/d5/dd6/de0/ff0 to d3/d5/da/d34/d3c/d5e/d62/d13a/f151 0 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/966: chown d3/d5/da/db/d21/d3b/dad/dea/fac 824 1 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/967: write d3/d5/da/d34/f46 [1405111,96654] 0 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/968: creat d3/d5/da/db/d21/f152 x:0 0 0 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/969: write d3/d5/da/db/dcb/f138 [244002,130962] 0 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/970: fdatasync d3/d48/f9d 0 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/971: chown d3/d5/d6a/d74/da8/d100/f112 1586 1 2024-09-16T08:57:31.144 INFO:tasks.workunit.client.0.smithi031.stdout:9/972: creat d3/d5/da/db/d21/f153 x:0 0 0 2024-09-16T08:57:31.146 INFO:tasks.workunit.client.0.smithi031.stdout:1/814: link d2/d15/d1f/d43/f11a d2/d15/d38/f11c 0 2024-09-16T08:57:31.153 INFO:tasks.workunit.client.0.smithi031.stdout:9/973: mknod d3/d5/da/d34/d3c/d5e/de1/d115/c154 0 2024-09-16T08:57:31.153 INFO:tasks.workunit.client.0.smithi031.stdout:9/974: write d3/d5/dd6/f14d [424019,13593] 0 2024-09-16T08:57:31.153 INFO:tasks.workunit.client.0.smithi031.stdout:9/975: fdatasync d3/d5/d6a/d74/da8/d100/f112 0 2024-09-16T08:57:31.154 INFO:tasks.workunit.client.0.smithi031.stdout:8/841: getdents d4/d77/ddc 0 2024-09-16T08:57:31.156 INFO:tasks.workunit.client.0.smithi031.stdout:1/815: mknod d2/d15/d5d/db4/de3/c11d 0 2024-09-16T08:57:31.157 INFO:tasks.workunit.client.0.smithi031.stdout:7/942: rename d2/d2d/d68/d113 to d2/d8/d11/d5d/d7e/db6/d13a 0 2024-09-16T08:57:31.157 INFO:tasks.workunit.client.0.smithi031.stdout:7/943: chown d2/d8/d69/d82/f10f 36831 1 2024-09-16T08:57:31.158 INFO:tasks.workunit.client.0.smithi031.stdout:1/816: unlink d2/d15/d1f/d111/l86 0 2024-09-16T08:57:31.158 INFO:tasks.workunit.client.0.smithi031.stdout:1/817: write d2/d15/d38/fc6 [642320,83206] 0 2024-09-16T08:57:31.161 INFO:tasks.workunit.client.0.smithi031.stdout:8/842: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:31.165 INFO:tasks.workunit.client.0.smithi031.stdout:2/884: rename d6/d2e/d40/d6c/d2c/d53/lce to d6/l122 0 2024-09-16T08:57:31.166 INFO:tasks.workunit.client.0.smithi031.stdout:7/944: creat d2/d8/d69/f13b x:0 0 0 2024-09-16T08:57:31.166 INFO:tasks.workunit.client.0.smithi031.stdout:7/945: creat d2/d2d/d68/f13c x:0 0 0 2024-09-16T08:57:31.166 INFO:tasks.workunit.client.0.smithi031.stdout:7/946: chown d2/d8/d11/d5d/d7e/db6 35 1 2024-09-16T08:57:31.168 INFO:tasks.workunit.client.0.smithi031.stdout:1/818: creat d2/d15/d1f/de9/d105/f11e x:0 0 0 2024-09-16T08:57:31.168 INFO:tasks.workunit.client.0.smithi031.stdout:1/819: fdatasync d2/d15/d5d/db4/de3/ffd 0 2024-09-16T08:57:31.168 INFO:tasks.workunit.client.0.smithi031.stdout:1/820: fdatasync d2/d15/d1f/de9/d67/f72 0 2024-09-16T08:57:31.168 INFO:tasks.workunit.client.0.smithi031.stdout:1/821: chown d2/d15/d5d/db4/da3/led 116887560 1 2024-09-16T08:57:31.171 INFO:tasks.workunit.client.0.smithi031.stdout:3/974: rename d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d7d/cbb to d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/d82/d96/c126 0 2024-09-16T08:57:31.174 INFO:tasks.workunit.client.0.smithi031.stdout:7/947: creat d2/d8/d11/d81/d110/f13d x:0 0 0 2024-09-16T08:57:31.174 INFO:tasks.workunit.client.0.smithi031.stdout:7/948: stat d2/d8/d11/d5d/d7e/fe0 0 2024-09-16T08:57:31.176 INFO:tasks.workunit.client.0.smithi031.stdout:8/843: dwrite d4/d2f/dc3/d37/d46/f75 [0,4194304] 0 2024-09-16T08:57:31.177 INFO:tasks.workunit.client.0.smithi031.stdout:1/822: creat d2/d15/d5d/def/f11f x:0 0 0 2024-09-16T08:57:31.180 INFO:tasks.workunit.client.0.smithi031.stdout:3/975: symlink d3/d14/d37/d1a/d1d/d3b/d3f/d49/l127 0 2024-09-16T08:57:31.180 INFO:tasks.workunit.client.0.smithi031.stdout:3/976: fdatasync d3/d14/d37/d47/f66 0 2024-09-16T08:57:31.180 INFO:tasks.workunit.client.0.smithi031.stdout:3/977: write d3/d14/d37/d1a/d1d/d3b/dc4/fca [2990152,42451] 0 2024-09-16T08:57:31.181 INFO:tasks.workunit.client.0.smithi031.stdout:7/949: symlink d2/d8/d69/d82/l13e 0 2024-09-16T08:57:31.181 INFO:tasks.workunit.client.0.smithi031.stdout:7/950: fdatasync d2/d2d/d6d/fb4 0 2024-09-16T08:57:31.183 INFO:tasks.workunit.client.0.smithi031.stdout:8/844: mkdir d4/d5b/d8d/dcd/d50/da3/d10d 0 2024-09-16T08:57:31.183 INFO:tasks.workunit.client.0.smithi031.stdout:1/823: symlink d2/l120 0 2024-09-16T08:57:31.184 INFO:tasks.workunit.client.0.smithi031.stdout:1/824: stat d2/d15/d1f/f62 0 2024-09-16T08:57:31.186 INFO:tasks.workunit.client.0.smithi031.stdout:2/885: rename d6/d2e/d40/dc3/lc7 to d6/d2e/d40/d6c/d2c/l123 0 2024-09-16T08:57:31.186 INFO:tasks.workunit.client.0.smithi031.stdout:3/978: creat d3/d14/d37/d1a/d1d/d3b/d60/f128 x:0 0 0 2024-09-16T08:57:31.188 INFO:tasks.workunit.client.0.smithi031.stdout:8/845: mknod d4/d2f/dc3/d37/c10e 0 2024-09-16T08:57:31.192 INFO:tasks.workunit.client.0.smithi031.stdout:2/886: mkdir d6/d20/d124 0 2024-09-16T08:57:31.194 INFO:tasks.workunit.client.0.smithi031.stdout:8/846: creat d4/d9e/dea/f10f x:0 0 0 2024-09-16T08:57:31.196 INFO:tasks.workunit.client.0.smithi031.stdout:3/979: getdents d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0 0 2024-09-16T08:57:31.197 INFO:tasks.workunit.client.0.smithi031.stdout:7/951: dread d2/f42 [0,4194304] 0 2024-09-16T08:57:31.197 INFO:tasks.workunit.client.0.smithi031.stdout:7/952: truncate d2/d2d/d68/da2/fe2 1292563 0 2024-09-16T08:57:31.197 INFO:tasks.workunit.client.0.smithi031.stdout:7/953: read d2/d8/d11/d5d/d6e/f111 [396142,86029] 0 2024-09-16T08:57:31.198 INFO:tasks.workunit.client.0.smithi031.stdout:8/847: getdents d4/d9e/daa 0 2024-09-16T08:57:31.198 INFO:tasks.workunit.client.0.smithi031.stdout:8/848: readlink d4/d77/l80 0 2024-09-16T08:57:31.198 INFO:tasks.workunit.client.0.smithi031.stdout:8/849: readlink d4/d5b/d8d/dcd/l65 0 2024-09-16T08:57:31.200 INFO:tasks.workunit.client.0.smithi031.stdout:8/850: creat d4/d7/d66/df1/f110 x:0 0 0 2024-09-16T08:57:31.200 INFO:tasks.workunit.client.0.smithi031.stdout:8/851: fsync d4/d7/f51 0 2024-09-16T08:57:31.201 INFO:tasks.workunit.client.0.smithi031.stdout:7/954: getdents d2/d8/d11/d5d 0 2024-09-16T08:57:31.203 INFO:tasks.workunit.client.0.smithi031.stdout:8/852: unlink d4/f17 0 2024-09-16T08:57:31.203 INFO:tasks.workunit.client.0.smithi031.stdout:1/825: dwrite d2/d15/d1f/de9/d52/fd3 [4194304,4194304] 0 2024-09-16T08:57:31.206 INFO:tasks.workunit.client.0.smithi031.stdout:8/853: unlink d4/d7/d66/f70 0 2024-09-16T08:57:31.209 INFO:tasks.workunit.client.0.smithi031.stdout:3/980: read d3/d14/d37/d33/f4d [240352,71589] 0 2024-09-16T08:57:31.209 INFO:tasks.workunit.client.0.smithi031.stdout:2/887: dwrite d6/ff5 [0,4194304] 0 2024-09-16T08:57:31.209 INFO:tasks.workunit.client.0.smithi031.stdout:2/888: read - d6/d2e/d40/d58/f11d zero size 2024-09-16T08:57:31.209 INFO:tasks.workunit.client.0.smithi031.stdout:2/889: truncate d6/d20/da1/dcf/dda/f111 817964 0 2024-09-16T08:57:31.209 INFO:tasks.workunit.client.0.smithi031.stdout:2/890: write d6/d2e/f9a [1418645,102621] 0 2024-09-16T08:57:31.210 INFO:tasks.workunit.client.0.smithi031.stdout:1/826: symlink d2/d15/d5d/def/l121 0 2024-09-16T08:57:31.210 INFO:tasks.workunit.client.0.smithi031.stdout:1/827: chown d2/f17 433003 1 2024-09-16T08:57:31.211 INFO:tasks.workunit.client.0.smithi031.stdout:3/981: dread d3/d14/d37/d1a/d1d/dac/fbc [0,4194304] 0 2024-09-16T08:57:31.211 INFO:tasks.workunit.client.0.smithi031.stdout:3/982: write d3/d14/d37/d1a/d1d/d9a/fa2 [126507,38363] 0 2024-09-16T08:57:31.211 INFO:tasks.workunit.client.0.smithi031.stdout:3/983: write d3/d14/d37/d1a/d1d/d3b/d3f/f74 [2047335,57414] 0 2024-09-16T08:57:31.211 INFO:tasks.workunit.client.0.smithi031.stdout:7/955: link d2/d2d/d6d/ffb d2/f13f 0 2024-09-16T08:57:31.212 INFO:tasks.workunit.client.0.smithi031.stdout:8/854: symlink d4/d5b/d8d/dcd/d50/da3/l111 0 2024-09-16T08:57:31.212 INFO:tasks.workunit.client.0.smithi031.stdout:8/855: fdatasync d4/d77/dd3/feb 0 2024-09-16T08:57:31.214 INFO:tasks.workunit.client.0.smithi031.stdout:1/828: mknod d2/d15/d5d/db4/de3/c122 0 2024-09-16T08:57:31.214 INFO:tasks.workunit.client.0.smithi031.stdout:2/891: dread d6/da/d4b/d4c/d68/fa8 [0,4194304] 0 2024-09-16T08:57:31.214 INFO:tasks.workunit.client.0.smithi031.stdout:2/892: dread - d6/d20/d51/f62 zero size 2024-09-16T08:57:31.217 INFO:tasks.workunit.client.0.smithi031.stdout:3/984: mkdir d3/d14/d37/d1a/d1d/df1/d129 0 2024-09-16T08:57:31.217 INFO:tasks.workunit.client.0.smithi031.stdout:2/893: read d6/da/f1e [113052,109832] 0 2024-09-16T08:57:31.219 INFO:tasks.workunit.client.0.smithi031.stdout:8/856: link d4/l5a d4/d2f/dc3/dcb/l112 0 2024-09-16T08:57:31.219 INFO:tasks.workunit.client.0.smithi031.stdout:8/857: readlink d4/d9e/d38/dc8/ld0 0 2024-09-16T08:57:31.219 INFO:tasks.workunit.client.0.smithi031.stdout:8/858: truncate d4/fb4 4700385 0 2024-09-16T08:57:31.220 INFO:tasks.workunit.client.0.smithi031.stdout:8/859: creat d4/d7/d66/de5/f113 x:0 0 0 2024-09-16T08:57:31.221 INFO:tasks.workunit.client.0.smithi031.stdout:8/860: write d4/d2f/fbc [855078,36204] 0 2024-09-16T08:57:31.224 INFO:tasks.workunit.client.0.smithi031.stdout:1/829: dread d2/d15/d32/f88 [0,4194304] 0 2024-09-16T08:57:31.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/956: dwrite d2/d2d/fdc [0,4194304] 0 2024-09-16T08:57:31.226 INFO:tasks.workunit.client.0.smithi031.stdout:7/957: readlink d2/d8/d11/d5d/d6e/l7b 0 2024-09-16T08:57:31.226 INFO:tasks.workunit.client.0.smithi031.stdout:1/830: rename d2/de/f2f to d2/d15/d1f/de9/d1b/d5c/d6b/da9/f123 0 2024-09-16T08:57:31.231 INFO:tasks.workunit.client.0.smithi031.stdout:3/985: dread d3/d14/d37/d1a/d24/f77 [0,4194304] 0 2024-09-16T08:57:31.231 INFO:tasks.workunit.client.0.smithi031.stdout:7/958: mkdir d2/d8/d86/d123/d140 0 2024-09-16T08:57:31.236 INFO:tasks.workunit.client.0.smithi031.stdout:3/986: truncate d3/d14/d37/d1a/d1d/d9a/dde/dd6/dd0/f10f 3791921 0 2024-09-16T08:57:31.238 INFO:tasks.workunit.client.0.smithi031.stdout:2/894: read d6/da/d5e/d72/fcb [703033,97300] 0 2024-09-16T08:57:31.239 INFO:tasks.workunit.client.0.smithi031.stdout:3/987: unlink d3/d14/d37/d1a/d1d/d3b/d8a/lfa 0 2024-09-16T08:57:31.239 INFO:tasks.workunit.client.0.smithi031.stdout:3/988: readlink d3/d14/d37/d33/d97/lc6 0 2024-09-16T08:57:31.239 INFO:tasks.workunit.client.0.smithi031.stdout:3/989: write d3/d14/d37/d1a/d1d/d3b/d3f/f74 [549009,84336] 0 2024-09-16T08:57:31.242 INFO:tasks.workunit.client.0.smithi031.stdout:2/895: creat d6/d20/da1/d11e/f125 x:0 0 0 2024-09-16T08:57:31.246 INFO:tasks.workunit.client.0.smithi031.stdout:3/990: mkdir d3/d14/d37/d1a/d1d/d3b/dc4/d12a 0 2024-09-16T08:57:31.250 INFO:tasks.workunit.client.0.smithi031.stdout:8/861: dwrite d4/d5b/d8d/dcd/f5d [4194304,4194304] 0 2024-09-16T08:57:31.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/896: creat d6/d2e/d40/d58/dae/f126 x:0 0 0 2024-09-16T08:57:31.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/897: read f1 [10123,87686] 0 2024-09-16T08:57:31.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/898: fsync d6/da/d4b/d4c/df9/fff 0 2024-09-16T08:57:31.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/899: chown d6/d2e/d40/fa5 12627 1 2024-09-16T08:57:31.253 INFO:tasks.workunit.client.0.smithi031.stdout:2/900: creat d6/da/d4b/d4c/df9/f127 x:0 0 0 2024-09-16T08:57:31.254 INFO:tasks.workunit.client.0.smithi031.stdout:3/991: symlink d3/d14/d37/d47/ded/l12b 0 2024-09-16T08:57:31.254 INFO:tasks.workunit.client.0.smithi031.stdout:3/992: chown d3/d14/d37/d1a/d1d/d3b/d60/f9c 186 1 2024-09-16T08:57:31.263 INFO:tasks.workunit.client.0.smithi031.stdout:3/993: dread d3/d14/d37/d47/ded/f10d [0,4194304] 0 2024-09-16T08:57:31.263 INFO:tasks.workunit.client.0.smithi031.stdout:8/862: unlink d4/d5b/d8d/dcd/c59 0 2024-09-16T08:57:31.270 INFO:tasks.workunit.client.0.smithi031.stdout:1/831: write d2/f3e [487622,15110] 0 2024-09-16T08:57:31.270 INFO:tasks.workunit.client.0.smithi031.stdout:1/832: fdatasync d2/d15/d1f/d43/f4c 0 2024-09-16T08:57:31.273 INFO:tasks.workunit.client.0.smithi031.stdout:2/901: symlink d6/da/d37/d8b/dab/l128 0 2024-09-16T08:57:31.273 INFO:tasks.workunit.client.0.smithi031.stdout:2/902: write d6/da/f27 [679605,12055] 0 2024-09-16T08:57:31.275 INFO:tasks.workunit.client.0.smithi031.stdout:3/994: rename d3/d14/d37/d1a/d1d/d3b/d3f/d49/ffe to d3/d14/d93/f12c 0 2024-09-16T08:57:31.278 INFO:tasks.workunit.client.0.smithi031.stdout:1/833: mknod d2/d15/d1f/de9/d67/c124 0 2024-09-16T08:57:31.293 INFO:tasks.workunit.client.0.smithi031.stdout:2/903: dwrite d6/da/d5e/d72/fcb [0,4194304] 0 2024-09-16T08:57:31.294 INFO:tasks.workunit.client.0.smithi031.stdout:5/826: sync 2024-09-16T08:57:31.294 INFO:tasks.workunit.client.0.smithi031.stdout:5/827: fsync d2/dc/df3/f10b 0 2024-09-16T08:57:31.294 INFO:tasks.workunit.client.0.smithi031.stdout:0/982: sync 2024-09-16T08:57:31.294 INFO:tasks.workunit.client.0.smithi031.stdout:0/983: readlink d6/d12/d47/dc5/l100 0 2024-09-16T08:57:31.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/863: getdents d4/d2f/dc3/d37 0 2024-09-16T08:57:31.296 INFO:tasks.workunit.client.0.smithi031.stdout:8/864: stat d4/d7/d99/la5 0 2024-09-16T08:57:31.299 INFO:tasks.workunit.client.0.smithi031.stdout:2/904: chown d6/d2e/d40/d6c/d2c/c3a 333737 1 2024-09-16T08:57:31.301 INFO:tasks.workunit.client.0.smithi031.stdout:0/984: truncate d6/f3b 1572067 0 2024-09-16T08:57:31.304 INFO:tasks.workunit.client.0.smithi031.stdout:2/905: dread d6/d2e/d107/ffd [0,4194304] 0 2024-09-16T08:57:31.305 INFO:tasks.workunit.client.0.smithi031.stdout:8/865: rename d4/d2f/dc3/d37/d46/l9d to d4/d5b/d8d/dcd/d79/l114 0 2024-09-16T08:57:31.306 INFO:tasks.workunit.client.0.smithi031.stdout:0/985: symlink d6/d12/d47/d5b/d9b/da2/l137 0 2024-09-16T08:57:31.306 INFO:tasks.workunit.client.0.smithi031.stdout:0/986: dread - d6/d12/d47/d5b/d9b/da2/fab zero size 2024-09-16T08:57:31.308 INFO:tasks.workunit.client.0.smithi031.stdout:2/906: truncate d6/d2e/d40/d6c/d2c/d53/d60/d91/ff3 2607586 0 2024-09-16T08:57:31.310 INFO:tasks.workunit.client.0.smithi031.stdout:0/987: mkdir d6/d12/d13/d11e/d138 0 2024-09-16T08:57:31.312 INFO:tasks.workunit.client.0.smithi031.stdout:2/907: mknod d6/da/d4b/d8f/c129 0 2024-09-16T08:57:31.312 INFO:tasks.workunit.client.0.smithi031.stdout:1/834: dwrite d2/d15/d1f/de9/d1b/d5c/ff2 [0,4194304] 0 2024-09-16T08:57:31.313 INFO:tasks.workunit.client.0.smithi031.stdout:3/995: dwrite d3/d14/d37/d1a/d1d/d3b/d3f/dd1/f112 [0,4194304] 0 2024-09-16T08:57:31.313 INFO:tasks.workunit.client.0.smithi031.stdout:3/996: stat d3/d14/d93/f12c 0 2024-09-16T08:57:31.314 INFO:tasks.workunit.client.0.smithi031.stdout:5/828: dwrite d2/dc/d1d/dbb/fcb [0,4194304] 0 2024-09-16T08:57:31.315 INFO:tasks.workunit.client.0.smithi031.stdout:2/908: mknod d6/da/d4b/da0/c12a 0 2024-09-16T08:57:31.315 INFO:tasks.workunit.client.0.smithi031.stdout:2/909: stat d6/da/d4b/d4c/d68/c103 0 2024-09-16T08:57:31.323 INFO:tasks.workunit.client.0.smithi031.stdout:3/997: write d3/d14/d37/d1a/d1d/d6a/f105 [932511,104932] 0 2024-09-16T08:57:31.326 INFO:tasks.workunit.client.0.smithi031.stdout:8/866: dwrite d4/d2f/dc3/d37/d46/d8c/f58 [0,4194304] 0 2024-09-16T08:57:31.329 INFO:tasks.workunit.client.0.smithi031.stdout:1/835: dwrite d2/d15/d1f/de9/d105/db9/fe6 [0,4194304] 0 2024-09-16T08:57:31.330 INFO:tasks.workunit.client.0.smithi031.stdout:5/829: unlink d2/dc/d36/f94 0 2024-09-16T08:57:31.332 INFO:tasks.workunit.client.0.smithi031.stdout:2/910: symlink d6/d20/d124/l12b 0 2024-09-16T08:57:31.336 INFO:tasks.workunit.client.0.smithi031.stdout:3/998: symlink d3/d14/d37/d1a/d1d/d1e/l12d 0 2024-09-16T08:57:31.344 INFO:tasks.workunit.client.0.smithi031.stdout:1/836: dwrite d2/d15/d1f/de9/d52/fcc [0,4194304] 0 2024-09-16T08:57:31.344 INFO:tasks.workunit.client.0.smithi031.stdout:1/837: fsync d2/d15/d1f/de9/d1b/f2b 0 2024-09-16T08:57:31.345 INFO:tasks.workunit.client.0.smithi031.stdout:5/830: mkdir d2/dc/d36/d118 0 2024-09-16T08:57:31.349 INFO:tasks.workunit.client.0.smithi031.stdout:2/911: creat d6/da/f12c x:0 0 0 2024-09-16T08:57:31.350 INFO:tasks.workunit.client.0.smithi031.stdout:3/999: creat d3/d14/d37/d1a/d1d/d3b/dc4/d12a/f12e x:0 0 0 2024-09-16T08:57:31.354 INFO:tasks.workunit.client.0.smithi031.stdout:1/838: mkdir d2/d125 0 2024-09-16T08:57:31.356 INFO:tasks.workunit.client.0.smithi031.stdout:1/839: creat d2/d15/d5d/db4/f126 x:0 0 0 2024-09-16T08:57:31.357 INFO:tasks.workunit.client.0.smithi031.stdout:1/840: mknod d2/d15/d1f/de9/d105/c127 0 2024-09-16T08:57:31.358 INFO:tasks.workunit.client.0.smithi031.stdout:1/841: mkdir d2/d15/d32/d128 0 2024-09-16T08:57:31.360 INFO:tasks.workunit.client.0.smithi031.stdout:1/842: rmdir d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0 39 2024-09-16T08:57:31.365 INFO:tasks.workunit.client.0.smithi031.stdout:5/831: dwrite d2/dc/d25/da2/fe6 [0,4194304] 0 2024-09-16T08:57:31.366 INFO:tasks.workunit.client.0.smithi031.stdout:5/832: rmdir d2/dc/d8c/de8 39 2024-09-16T08:57:31.372 INFO:tasks.workunit.client.0.smithi031.stdout:2/912: dwrite d6/da/d4b/d4c/df9/fd8 [0,4194304] 0 2024-09-16T08:57:31.377 INFO:tasks.workunit.client.0.smithi031.stdout:2/913: rename d6/fe9 to d6/da/d4b/d4c/f12d 0 2024-09-16T08:57:31.380 INFO:tasks.workunit.client.0.smithi031.stdout:2/914: mkdir d6/d20/d51/d12e 0 2024-09-16T08:57:31.380 INFO:tasks.workunit.client.0.smithi031.stdout:2/915: write d6/d2e/d40/d6c/d2c/d53/d60/f35 [2744531,103410] 0 2024-09-16T08:57:31.380 INFO:tasks.workunit.client.0.smithi031.stdout:2/916: chown d6/d2e/d40/d6c/d2c 139 1 2024-09-16T08:57:31.385 INFO:tasks.workunit.client.0.smithi031.stdout:1/843: dwrite d2/d15/d1f/de9/d1b/d5c/f66 [0,4194304] 0 2024-09-16T08:57:31.386 INFO:tasks.workunit.client.0.smithi031.stdout:1/844: stat d2/d15/d1f/de9/d1b/f65 0 2024-09-16T08:57:31.386 INFO:tasks.workunit.client.0.smithi031.stdout:5/833: dwrite d2/dc/d25/f9f [0,4194304] 0 2024-09-16T08:57:31.387 INFO:tasks.workunit.client.0.smithi031.stdout:1/845: symlink d2/d15/d1f/de9/d1b/d5c/d6b/dc1/df0/l129 0 2024-09-16T08:57:31.389 INFO:tasks.workunit.client.0.smithi031.stdout:5/834: mknod d2/dc/c119 0 2024-09-16T08:57:31.389 INFO:tasks.workunit.client.0.smithi031.stdout:5/835: dread - d2/dc/d36/fca zero size 2024-09-16T08:57:31.389 INFO:tasks.workunit.client.0.smithi031.stdout:1/846: rmdir d2/d15/dff 39 2024-09-16T08:57:31.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/847: creat d2/d15/d32/f12a x:0 0 0 2024-09-16T08:57:31.393 INFO:tasks.workunit.client.0.smithi031.stdout:1/848: chown d2/d15/d1f/de9/l11 13282644 1 2024-09-16T08:57:31.397 INFO:tasks.workunit.client.0.smithi031.stdout:2/917: dwrite d6/ff5 [0,4194304] 0 2024-09-16T08:57:31.401 INFO:tasks.workunit.client.0.smithi031.stdout:9/976: sync 2024-09-16T08:57:31.402 INFO:tasks.workunit.client.0.smithi031.stdout:5/836: dread d2/dc/d25/d21/d2e/fa5 [0,4194304] 0 2024-09-16T08:57:31.402 INFO:tasks.workunit.client.0.smithi031.stdout:5/837: dread - d2/dc/d3d/fb0 zero size 2024-09-16T08:57:31.417 INFO:tasks.workunit.client.0.smithi031.stdout:1/849: dwrite d2/d15/d1f/de9/d1b/d5c/d6b/da9/fe2 [0,4194304] 0 2024-09-16T08:57:31.432 INFO:tasks.workunit.client.0.smithi031.stdout:2/918: dwrite d6/d2e/d40/d58/dae/db1/fc4 [0,4194304] 0 2024-09-16T08:57:31.435 INFO:tasks.workunit.client.0.smithi031.stdout:9/977: dwrite d3/d5/da/db/d21/f2f [0,4194304] 0 2024-09-16T08:57:31.435 INFO:tasks.workunit.client.0.smithi031.stdout:9/978: chown d3/d5/d6a/d74/da8/dc3/dbc/dd7/d10e/c8f 8676018 1 2024-09-16T08:57:31.436 INFO:tasks.workunit.client.0.smithi031.stdout:5/838: dwrite d2/dc/f3a [0,4194304] 0 2024-09-16T08:57:31.436 INFO:tasks.workunit.client.0.smithi031.stdout:5/839: fdatasync d2/dc/df3/f10b 0 2024-09-16T08:57:31.438 INFO:tasks.workunit.client.0.smithi031.stdout:9/979: creat d3/f155 x:0 0 0 2024-09-16T08:57:31.438 INFO:tasks.workunit.client.0.smithi031.stdout:9/980: fdatasync d3/d5/da/d34/f5c 0 2024-09-16T08:57:31.438 INFO:tasks.workunit.client.0.smithi031.stdout:9/981: dread - d3/d5/da/db/d21/d3b/dad/dd3/f101 zero size 2024-09-16T08:57:31.438 INFO:tasks.workunit.client.0.smithi031.stdout:9/982: truncate d3/d5/dd6/f14d 1135560 0 2024-09-16T08:57:31.450 INFO:tasks.workunit.client.0.smithi031.stdout:9/983: dread d3/d5/da/d34/f35 [0,4194304] 0 2024-09-16T08:57:31.451 INFO:tasks.workunit.client.0.smithi031.stdout:9/984: symlink d3/d5/dd6/de0/l156 0 2024-09-16T08:57:31.452 INFO:tasks.workunit.client.0.smithi031.stdout:9/985: truncate d3/d5/d6a/d74/da8/d100/f112 4948111 0 2024-09-16T08:57:31.452 INFO:tasks.workunit.client.0.smithi031.stdout:9/986: readlink d3/d5/da/d34/d3c/d5e/d9c/dcd/l135 0 2024-09-16T08:57:31.453 INFO:tasks.workunit.client.0.smithi031.stdout:9/987: mkdir d3/d5/da/db/d21/d3b/d12b/d157 0 2024-09-16T08:57:31.455 INFO:tasks.workunit.client.0.smithi031.stdout:2/919: dwrite d6/d20/da1/dcf/dda/f104 [0,4194304] 0 2024-09-16T08:57:31.458 INFO:tasks.workunit.client.0.smithi031.stdout:1/850: dwrite d2/d15/dff/f108 [0,4194304] 0 2024-09-16T08:57:31.463 INFO:tasks.workunit.client.0.smithi031.stdout:9/988: mknod d3/d5/da/c158 0 2024-09-16T08:57:31.468 INFO:tasks.workunit.client.0.smithi031.stdout:2/920: mkdir d6/d20/da1/dcf/d12f 0 2024-09-16T08:57:31.472 INFO:tasks.workunit.client.0.smithi031.stdout:5/840: dwrite d2/dc/f2d [0,4194304] 0 2024-09-16T08:57:31.476 INFO:tasks.workunit.client.0.smithi031.stdout:1/851: rename d2/d15/d1f/de9/d1b/d5c/d6b/dc1/d110 to d2/d15/d5d/d12b 0 2024-09-16T08:57:31.476 INFO:tasks.workunit.client.0.smithi031.stdout:2/921: symlink d6/d20/da1/dcf/dda/l130 0 2024-09-16T08:57:31.481 INFO:tasks.workunit.client.0.smithi031.stdout:1/852: symlink d2/d15/d5d/d12b/l12c 0 2024-09-16T08:57:31.481 INFO:tasks.workunit.client.0.smithi031.stdout:1/853: dread - d2/d15/d1f/f11b zero size 2024-09-16T08:57:31.484 INFO:tasks.workunit.client.0.smithi031.stdout:9/989: dwrite d3/d5/da/db/d21/f153 [0,4194304] 0 2024-09-16T08:57:31.484 INFO:tasks.workunit.client.0.smithi031.stdout:9/990: chown d3/d5/da/db/d113 3684 1 2024-09-16T08:57:31.486 INFO:tasks.workunit.client.0.smithi031.stdout:1/854: truncate d2/d15/d1f/de9/d1b/d5c/d6b/da9/f123 2521400 0 2024-09-16T08:57:31.492 INFO:tasks.workunit.client.0.smithi031.stdout:2/922: dread d6/da/d5e/d8d/f9e [0,4194304] 0 2024-09-16T08:57:31.500 INFO:tasks.workunit.client.0.smithi031.stdout:9/991: mkdir d3/d5/da/d34/d159 0 2024-09-16T08:57:31.508 INFO:tasks.workunit.client.0.smithi031.stdout:1/855: rmdir d2/d15/d9c 39 2024-09-16T08:57:31.509 INFO:tasks.workunit.client.0.smithi031.stdout:1/856: chown d2/d15/d1f/d111/fb6 48 1 2024-09-16T08:57:31.509 INFO:tasks.workunit.client.0.smithi031.stdout:9/992: rmdir d3/d5/db3 39 2024-09-16T08:57:31.509 INFO:tasks.workunit.client.0.smithi031.stdout:9/993: chown d3/d48/fc8 0 1 2024-09-16T08:57:31.512 INFO:tasks.workunit.client.0.smithi031.stdout:1/857: mkdir d2/d15/d1f/de9/d1b/d5c/d6b/dc1/df0/d12d 0 2024-09-16T08:57:31.514 INFO:tasks.workunit.client.0.smithi031.stdout:9/994: link d3/d5/d6a/d74/da8/dc3/dbc/ce5 d3/d5/da/d12c/c15a 0 2024-09-16T08:57:31.520 INFO:tasks.workunit.client.0.smithi031.stdout:2/923: dwrite d6/d20/da1/d11e/f125 [0,4194304] 0 2024-09-16T08:57:31.520 INFO:tasks.workunit.client.0.smithi031.stdout:9/995: symlink d3/d5/da/db/dcb/l15b 0 2024-09-16T08:57:31.520 INFO:tasks.workunit.client.0.smithi031.stdout:9/996: write d3/d5/dd6/f107 [623149,89563] 0 2024-09-16T08:57:31.523 INFO:tasks.workunit.client.0.smithi031.stdout:1/858: getdents d2/d15/d1f/de9/d105 0 2024-09-16T08:57:31.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/859: creat d2/d15/d1f/de9/d1b/f12e x:0 0 0 2024-09-16T08:57:31.528 INFO:tasks.workunit.client.0.smithi031.stdout:1/860: dread - d2/d15/d1f/d43/f11a zero size 2024-09-16T08:57:31.529 INFO:tasks.workunit.client.0.smithi031.stdout:2/924: dread d6/d20/da1/d11e/f125 [0,4194304] 0 2024-09-16T08:57:31.532 INFO:tasks.workunit.client.0.smithi031.stdout:1/861: mknod d2/d15/d1f/de9/d105/db9/c12f 0 2024-09-16T08:57:31.532 INFO:tasks.workunit.client.0.smithi031.stdout:1/862: chown d2/de/d6c/fb1 5642175 1 2024-09-16T08:57:31.533 INFO:tasks.workunit.client.0.smithi031.stdout:9/997: dread d3/d5/f6 [4194304,4194304] 0 2024-09-16T08:57:31.544 INFO:tasks.workunit.client.0.smithi031.stdout:2/925: dread d6/da/d5e/d72/ff7 [0,4194304] 0 2024-09-16T08:57:31.547 INFO:tasks.workunit.client.0.smithi031.stdout:2/926: truncate d6/da/f1e 526771 0 2024-09-16T08:57:31.547 INFO:tasks.workunit.client.0.smithi031.stdout:2/927: chown d6/da/d4b/d4c/df9/fd8 1 1 2024-09-16T08:57:31.549 INFO:tasks.workunit.client.0.smithi031.stdout:2/928: truncate d6/f9 640131 0 2024-09-16T08:57:31.550 INFO:tasks.workunit.client.0.smithi031.stdout:2/929: mkdir d6/d2e/d40/d131 0 2024-09-16T08:57:31.552 INFO:tasks.workunit.client.0.smithi031.stdout:2/930: link d6/d20/c49 d6/da/d5e/c132 0 2024-09-16T08:57:31.552 INFO:tasks.workunit.client.0.smithi031.stdout:2/931: chown d6/da/d4b/d4c/d73/cd3 270 1 2024-09-16T08:57:31.553 INFO:tasks.workunit.client.0.smithi031.stdout:9/998: dwrite d3/d5/d73/f133 [0,4194304] 0 2024-09-16T08:57:31.554 INFO:tasks.workunit.client.0.smithi031.stdout:1/863: dwrite d2/d15/d1f/de9/d1b/f65 [0,4194304] 0 2024-09-16T08:57:31.554 INFO:tasks.workunit.client.0.smithi031.stdout:1/864: fdatasync d2/d15/d32/d87/fd8 0 2024-09-16T08:57:31.555 INFO:tasks.workunit.client.0.smithi031.stdout:2/932: symlink d6/da/d4b/d4c/df9/l133 0 2024-09-16T08:57:31.562 INFO:tasks.workunit.client.0.smithi031.stdout:2/933: dread d6/d2e/d40/d6c/d2c/d53/d60/f35 [0,4194304] 0 2024-09-16T08:57:31.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/959: sync 2024-09-16T08:57:31.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/960: write d2/d8/d11/d66/d9a/fca [927510,3767] 0 2024-09-16T08:57:31.567 INFO:tasks.workunit.client.0.smithi031.stdout:7/961: truncate d2/d8/d11/d5d/f109 69999 0 2024-09-16T08:57:31.571 INFO:tasks.workunit.client.0.smithi031.stdout:9/999: symlink d3/d5/da/d34/d3c/d5e/d62/l15c 0 2024-09-16T08:57:31.572 INFO:tasks.workunit.client.0.smithi031.stdout:1/865: symlink d2/d15/d1f/de9/d1b/d5c/d6b/da9/l130 0 2024-09-16T08:57:31.586 INFO:tasks.workunit.client.0.smithi031.stdout:2/934: dwrite d6/d2e/d40/d6c/f5a [0,4194304] 0 2024-09-16T08:57:31.592 INFO:tasks.workunit.client.0.smithi031.stdout:2/935: unlink d6/da/l45 0 2024-09-16T08:57:31.594 INFO:tasks.workunit.client.0.smithi031.stdout:2/936: rename d6/da/d37/deb to d6/d2e/d40/d6c/d2c/d53/d60/d11b/d134 0 2024-09-16T08:57:31.596 INFO:tasks.workunit.client.0.smithi031.stdout:2/937: mkdir d6/da/d5e/d72/d135 0 2024-09-16T08:57:31.599 INFO:tasks.workunit.client.0.smithi031.stdout:2/938: rename d6/d2e/d40/d58/dae/cf1 to d6/da/d4b/d8f/c136 0 2024-09-16T08:57:31.599 INFO:tasks.workunit.client.0.smithi031.stdout:2/939: dread - d6/d20/da1/dcf/f108 zero size 2024-09-16T08:57:31.599 INFO:tasks.workunit.client.0.smithi031.stdout:2/940: stat d6/d20/d51/ddc/fef 0 2024-09-16T08:57:31.601 INFO:tasks.workunit.client.0.smithi031.stdout:2/941: mkdir d6/da/d5e/d72/d137 0 2024-09-16T08:57:31.603 INFO:tasks.workunit.client.0.smithi031.stdout:2/942: creat d6/d20/d51/f138 x:0 0 0 2024-09-16T08:57:31.603 INFO:tasks.workunit.client.0.smithi031.stdout:2/943: creat d6/da/d37/d8b/dab/f139 x:0 0 0 2024-09-16T08:57:31.616 INFO:tasks.workunit.client.0.smithi031.stdout:2/944: dwrite d6/d2e/ffa [0,4194304] 0 2024-09-16T08:57:31.619 INFO:tasks.workunit.client.0.smithi031.stdout:2/945: rmdir d6/da/d4b/da0/d110 0 2024-09-16T08:57:31.619 INFO:tasks.workunit.client.0.smithi031.stdout:2/946: chown d6/d2e/d40/d6c/d2c/l3c 24 1 2024-09-16T08:57:31.622 INFO:tasks.workunit.client.0.smithi031.stdout:2/947: mknod d6/d20/d51/ddc/c13a 0 2024-09-16T08:57:31.638 INFO:tasks.workunit.client.0.smithi031.stdout:2/948: dwrite d6/d2e/d40/d58/fa2 [4194304,4194304] 0 2024-09-16T08:57:31.640 INFO:tasks.workunit.client.0.smithi031.stdout:2/949: creat d6/da/d4b/f13b x:0 0 0 2024-09-16T08:57:31.640 INFO:tasks.workunit.client.0.smithi031.stdout:2/950: write d6/d2e/d40/d6c/d2c/d53/d60/f93 [1027419,77118] 0 2024-09-16T08:57:31.643 INFO:tasks.workunit.client.0.smithi031.stdout:2/951: rmdir d6/da/d4b/d4c/d68 39 2024-09-16T08:57:31.646 INFO:tasks.workunit.client.0.smithi031.stdout:2/952: mkdir d6/da/d4b/d4c/df9/d13c 0 2024-09-16T08:57:31.649 INFO:tasks.workunit.client.0.smithi031.stdout:2/953: symlink d6/d2e/d40/d6c/d2c/l13d 0 2024-09-16T08:57:31.651 INFO:tasks.workunit.client.0.smithi031.stdout:2/954: mknod d6/d2e/d40/d6c/c13e 0 2024-09-16T08:57:31.652 INFO:tasks.workunit.client.0.smithi031.stdout:2/955: dread d6/da/f27 [0,4194304] 0 2024-09-16T08:57:31.653 INFO:tasks.workunit.client.0.smithi031.stdout:2/956: mkdir d6/d20/d124/d13f 0 2024-09-16T08:57:31.672 INFO:tasks.workunit.client.0.smithi031.stdout:8/867: sync 2024-09-16T08:57:31.672 INFO:tasks.workunit.client.0.smithi031.stdout:0/988: sync 2024-09-16T08:57:31.673 INFO:tasks.workunit.client.0.smithi031.stdout:0/989: stat d6/c3f 0 2024-09-16T08:57:31.673 INFO:tasks.workunit.client.0.smithi031.stdout:8/868: mkdir d4/d2f/dc3/d37/d46/d115 0 2024-09-16T08:57:31.674 INFO:tasks.workunit.client.0.smithi031.stdout:2/957: dwrite d6/da/d4b/d4c/df9/fd8 [0,4194304] 0 2024-09-16T08:57:31.674 INFO:tasks.workunit.client.0.smithi031.stdout:2/958: chown d6/da/d4b/d4c/df9/d13c 19742709 1 2024-09-16T08:57:31.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/990: mkdir d6/dac/d115/d139 0 2024-09-16T08:57:31.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/991: readlink d6/d61/lf4 0 2024-09-16T08:57:31.675 INFO:tasks.workunit.client.0.smithi031.stdout:0/992: chown d6/d12/d13/f11f 97452 1 2024-09-16T08:57:31.677 INFO:tasks.workunit.client.0.smithi031.stdout:2/959: unlink d6/c1f 0 2024-09-16T08:57:31.679 INFO:tasks.workunit.client.0.smithi031.stdout:2/960: truncate d6/d20/da1/dcf/dda/de3/fe8 3101794 0 2024-09-16T08:57:31.683 INFO:tasks.workunit.client.0.smithi031.stdout:2/961: mkdir d6/da/d37/d8b/d140 0 2024-09-16T08:57:31.683 INFO:tasks.workunit.client.0.smithi031.stdout:2/962: fdatasync d6/d2e/d107/ffb 0 2024-09-16T08:57:31.684 INFO:tasks.workunit.client.0.smithi031.stdout:2/963: write d6/da/d37/d42/fbb [892153,121393] 0 2024-09-16T08:57:31.684 INFO:tasks.workunit.client.0.smithi031.stdout:8/869: dread d4/d7/f15 [0,4194304] 0 2024-09-16T08:57:31.688 INFO:tasks.workunit.client.0.smithi031.stdout:2/964: unlink d6/d2e/d40/d58/l7c 0 2024-09-16T08:57:31.690 INFO:tasks.workunit.client.0.smithi031.stdout:2/965: mkdir d6/d2e/d40/d6c/d2c/d53/d60/d141 0 2024-09-16T08:57:31.690 INFO:tasks.workunit.client.0.smithi031.stdout:2/966: write d6/d20/d51/ddc/fe6 [520040,54178] 0 2024-09-16T08:57:31.691 INFO:tasks.workunit.client.0.smithi031.stdout:2/967: creat d6/d2e/d40/d6c/d2c/d53/d60/d141/f142 x:0 0 0 2024-09-16T08:57:31.691 INFO:tasks.workunit.client.0.smithi031.stdout:2/968: chown d6/da/d4b/fa4 2626 1 2024-09-16T08:57:31.693 INFO:tasks.workunit.client.0.smithi031.stdout:2/969: rename d6/da/d5e/d8d/f9e to d6/d20/d124/f143 0 2024-09-16T08:57:31.695 INFO:tasks.workunit.client.0.smithi031.stdout:2/970: mkdir d6/d2e/d40/d6c/d144 0 2024-09-16T08:57:31.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/993: dwrite d6/d61/d91/f9f [0,4194304] 0 2024-09-16T08:57:31.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/994: chown d6/d12/d13/da5/dfa 0 1 2024-09-16T08:57:31.695 INFO:tasks.workunit.client.0.smithi031.stdout:0/995: write d6/d61/d91/ff9 [296174,74569] 0 2024-09-16T08:57:31.696 INFO:tasks.workunit.client.0.smithi031.stdout:2/971: mkdir d6/d20/d51/d145 0 2024-09-16T08:57:31.697 INFO:tasks.workunit.client.0.smithi031.stdout:0/996: mknod d6/d12/d47/c13a 0 2024-09-16T08:57:31.701 INFO:tasks.workunit.client.0.smithi031.stdout:8/870: dwrite d4/d2f/f41 [0,4194304] 0 2024-09-16T08:57:31.701 INFO:tasks.workunit.client.0.smithi031.stdout:8/871: dread - d4/d77/dd3/feb zero size 2024-09-16T08:57:31.719 INFO:tasks.workunit.client.0.smithi031.stdout:2/972: dwrite d6/d20/fcd [0,4194304] 0 2024-09-16T08:57:31.719 INFO:tasks.workunit.client.0.smithi031.stdout:2/973: write d6/da/f17 [8834749,95046] 0 2024-09-16T08:57:31.719 INFO:tasks.workunit.client.0.smithi031.stdout:8/872: dwrite d4/d5b/d8d/dcd/d50/f98 [0,4194304] 0 2024-09-16T08:57:31.719 INFO:tasks.workunit.client.0.smithi031.stdout:8/873: write d4/d2f/dc3/d24/fc6 [632586,123819] 0 2024-09-16T08:57:31.723 INFO:tasks.workunit.client.0.smithi031.stdout:2/974: readlink d6/d2e/d40/d6c/d2c/d53/lca 0 2024-09-16T08:57:31.723 INFO:tasks.workunit.client.0.smithi031.stdout:8/874: dread d4/d2f/dc3/d37/fe3 [0,4194304] 0 2024-09-16T08:57:31.725 INFO:tasks.workunit.client.0.smithi031.stdout:2/975: creat d6/da/d4b/d4c/d68/f146 x:0 0 0 2024-09-16T08:57:31.727 INFO:tasks.workunit.client.0.smithi031.stdout:2/976: rename d6/da/d37/d42/ca6 to d6/d2e/d40/d6c/d2c/d53/d60/d141/c147 0 2024-09-16T08:57:31.727 INFO:tasks.workunit.client.0.smithi031.stdout:2/977: truncate d6/d20/d51/f62 763957 0 2024-09-16T08:57:31.732 INFO:tasks.workunit.client.0.smithi031.stdout:8/875: write d4/f29 [1639705,18513] 0 2024-09-16T08:57:31.732 INFO:tasks.workunit.client.0.smithi031.stdout:2/978: mkdir d6/d20/d148 0 2024-09-16T08:57:31.732 INFO:tasks.workunit.client.0.smithi031.stdout:2/979: write d6/da/d4b/d4c/fe7 [1010121,901] 0 2024-09-16T08:57:31.734 INFO:tasks.workunit.client.0.smithi031.stdout:8/876: symlink d4/d5b/d7b/l116 0 2024-09-16T08:57:31.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/877: fdatasync d4/d9e/fe9 0 2024-09-16T08:57:31.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/878: stat d4/d7/d99/fc1 0 2024-09-16T08:57:31.735 INFO:tasks.workunit.client.0.smithi031.stdout:2/980: mknod d6/d2e/d40/d6c/d2c/d53/d60/d11b/d134/d117/c149 0 2024-09-16T08:57:31.739 INFO:tasks.workunit.client.0.smithi031.stdout:2/981: symlink d6/da/d37/d42/l14a 0 2024-09-16T08:57:31.741 INFO:tasks.workunit.client.0.smithi031.stdout:2/982: mkdir d6/d2e/d40/d6c/d2c/d53/d60/d91/d14b 0 2024-09-16T08:57:31.748 INFO:tasks.workunit.client.0.smithi031.stdout:8/879: dwrite d4/d2f/dc3/d24/f101 [0,4194304] 0 2024-09-16T08:57:31.748 INFO:tasks.workunit.client.0.smithi031.stdout:8/880: chown d4/d9e/d38/f3c 1851419 1 2024-09-16T08:57:31.750 INFO:tasks.workunit.client.0.smithi031.stdout:2/983: dread d6/d2e/d40/d6c/d2c/d53/d60/d91/ff3 [0,4194304] 0 2024-09-16T08:57:31.753 INFO:tasks.workunit.client.0.smithi031.stdout:8/881: symlink d4/d7/d66/dfa/l117 0 2024-09-16T08:57:31.754 INFO:tasks.workunit.client.0.smithi031.stdout:8/882: chown d4/d2f/c107 1482 1 2024-09-16T08:57:31.756 INFO:tasks.workunit.client.0.smithi031.stdout:2/984: link d6/da/d4b/ff2 d6/da/d37/d8b/f14c 0 2024-09-16T08:57:31.760 INFO:tasks.workunit.client.0.smithi031.stdout:2/985: dread d6/d2e/d40/d6c/d2c/d53/d60/d91/ff3 [0,4194304] 0 2024-09-16T08:57:31.761 INFO:tasks.workunit.client.0.smithi031.stdout:2/986: rmdir d6/d20/da1/dcf/dda/de3 39 2024-09-16T08:57:31.765 INFO:tasks.workunit.client.0.smithi031.stdout:2/987: dread d6/da/d37/d42/f71 [0,4194304] 0 2024-09-16T08:57:31.773 INFO:tasks.workunit.client.0.smithi031.stdout:2/988: dread d6/da/d5e/d72/fcb [0,4194304] 0 2024-09-16T08:57:31.774 INFO:tasks.workunit.client.0.smithi031.stdout:2/989: truncate d6/da/d37/f87 506335 0 2024-09-16T08:57:31.777 INFO:tasks.workunit.client.0.smithi031.stdout:2/990: rename d6/d2e/d40/d6c/d2c/l3c to d6/d2e/d40/d58/dae/db1/l14d 0 2024-09-16T08:57:31.779 INFO:tasks.workunit.client.0.smithi031.stdout:2/991: mknod d6/d20/d51/d12e/c14e 0 2024-09-16T08:57:31.781 INFO:tasks.workunit.client.0.smithi031.stdout:2/992: truncate d6/d2e/d40/f94 1057760 0 2024-09-16T08:57:31.784 INFO:tasks.workunit.client.0.smithi031.stdout:2/993: readlink d6/d2e/d40/l10e 0 2024-09-16T08:57:31.791 INFO:tasks.workunit.client.0.smithi031.stdout:2/994: dread d6/d2e/d40/d6c/d2c/d53/f10d [0,4194304] 0 2024-09-16T08:57:31.810 INFO:tasks.workunit.client.0.smithi031.stdout:2/995: dwrite d6/d20/fac [0,4194304] 0 2024-09-16T08:57:31.813 INFO:tasks.workunit.client.0.smithi031.stdout:2/996: creat d6/d20/da1/dcf/dda/de3/d120/f14f x:0 0 0 2024-09-16T08:57:31.814 INFO:tasks.workunit.client.0.smithi031.stdout:2/997: fsync d6/d20/da1/dcf/dda/fee 0 2024-09-16T08:57:31.814 INFO:tasks.workunit.client.0.smithi031.stdout:2/998: creat d6/d2e/d40/d58/f150 x:0 0 0 2024-09-16T08:57:31.814 INFO:tasks.workunit.client.0.smithi031.stdout:2/999: creat d6/da/d4b/d4c/df9/f151 x:0 0 0 2024-09-16T08:57:31.822 INFO:tasks.workunit.client.0.smithi031.stdout:5/841: sync 2024-09-16T08:57:31.822 INFO:tasks.workunit.client.0.smithi031.stdout:5/842: fsync d2/dc/d3d/f86 0 2024-09-16T08:57:31.846 INFO:tasks.workunit.client.0.smithi031.stdout:5/843: write d2/dc/d25/d21/d2e/fa5 [1695757,44438] 0 2024-09-16T08:57:31.847 INFO:tasks.workunit.client.0.smithi031.stdout:5/844: mknod d2/dc/d1d/dbb/c11a 0 2024-09-16T08:57:31.848 INFO:tasks.workunit.client.0.smithi031.stdout:5/845: mkdir d2/dc/d25/da2/d11b 0 2024-09-16T08:57:31.850 INFO:tasks.workunit.client.0.smithi031.stdout:5/846: symlink d2/dc/d25/d21/d81/l11c 0 2024-09-16T08:57:31.850 INFO:tasks.workunit.client.0.smithi031.stdout:5/847: chown d2/dc/d91/fd3 1915426623 1 2024-09-16T08:57:31.850 INFO:tasks.workunit.client.0.smithi031.stdout:5/848: chown d2/dc/d3d/c56 407 1 2024-09-16T08:57:31.851 INFO:tasks.workunit.client.0.smithi031.stdout:5/849: rename d2/dc/d1d/l7b to d2/l11d 0 2024-09-16T08:57:31.865 INFO:tasks.workunit.client.0.smithi031.stdout:5/850: getdents d2 0 2024-09-16T08:57:31.865 INFO:tasks.workunit.client.0.smithi031.stdout:5/851: fsync d2/dc/d25/d21/f45 0 2024-09-16T08:57:31.868 INFO:tasks.workunit.client.0.smithi031.stdout:5/852: truncate d2/dc/d1d/fa1 2468941 0 2024-09-16T08:57:31.870 INFO:tasks.workunit.client.0.smithi031.stdout:5/853: creat d2/dc/d3d/d4c/f11e x:0 0 0 2024-09-16T08:57:31.870 INFO:tasks.workunit.client.0.smithi031.stdout:5/854: getdents d2/dc/d91/dee/d111 0 2024-09-16T08:57:31.872 INFO:tasks.workunit.client.0.smithi031.stdout:5/855: mkdir d2/dc/d1d/d97/d11f 0 2024-09-16T08:57:31.874 INFO:tasks.workunit.client.0.smithi031.stdout:5/856: mkdir d2/dc/d1d/dbb/dfe/d120 0 2024-09-16T08:57:31.874 INFO:tasks.workunit.client.0.smithi031.stdout:5/857: dread - d2/dc/d91/f114 zero size 2024-09-16T08:57:31.874 INFO:tasks.workunit.client.0.smithi031.stdout:5/858: truncate d2/dc/d25/d21/fa3 595094 0 2024-09-16T08:57:31.891 INFO:tasks.workunit.client.0.smithi031.stdout:7/962: sync 2024-09-16T08:57:31.902 INFO:tasks.workunit.client.0.smithi031.stdout:7/963: write d2/f6a [3759448,118014] 0 2024-09-16T08:57:31.903 INFO:tasks.workunit.client.0.smithi031.stdout:7/964: mkdir d2/d8/d11/d5d/d141 0 2024-09-16T08:57:31.903 INFO:tasks.workunit.client.0.smithi031.stdout:7/965: write d2/d8/d11/d81/fa5 [4044801,87011] 0 2024-09-16T08:57:31.903 INFO:tasks.workunit.client.0.smithi031.stdout:7/966: write d2/fb [3282008,71606] 0 2024-09-16T08:57:31.905 INFO:tasks.workunit.client.0.smithi031.stdout:7/967: rename d2/d8/d11/d5d/d7e/db6/d13a/d11c/fb0 to d2/d8/d11/dd5/f142 0 2024-09-16T08:57:31.908 INFO:tasks.workunit.client.0.smithi031.stdout:7/968: readlink d2/d8/d11/l46 0 2024-09-16T08:57:31.910 INFO:tasks.workunit.client.0.smithi031.stdout:7/969: link d2/d8/d11/d66/l97 d2/d8/d11/d5d/d7e/db6/d13a/d114/l143 0 2024-09-16T08:57:31.928 INFO:tasks.workunit.client.0.smithi031.stdout:7/970: dwrite d2/d8/d69/ff7 [0,4194304] 0 2024-09-16T08:57:31.928 INFO:tasks.workunit.client.0.smithi031.stdout:7/971: fdatasync d2/d8/d86/da1/da9/fd3 0 2024-09-16T08:57:31.943 INFO:tasks.workunit.client.0.smithi031.stdout:7/972: dwrite d2/d8/d11/d5d/d7e/fb9 [4194304,4194304] 0 2024-09-16T08:57:31.959 INFO:tasks.workunit.client.0.smithi031.stdout:7/973: dwrite d2/d2d/d6d/fed [0,4194304] 0 2024-09-16T08:57:31.993 INFO:tasks.workunit.client.0.smithi031.stdout:7/974: link d2/d8/d11/d81/f9f d2/d8/d11/d5d/d7e/db6/f144 0 2024-09-16T08:57:32.056 INFO:tasks.workunit.client.0.smithi031.stdout:7/975: creat d2/d8/d11/d5d/d7e/db6/d13a/d114/f145 x:0 0 0 2024-09-16T08:57:32.057 INFO:tasks.workunit.client.0.smithi031.stdout:7/976: stat d2/d8/d11/d5d/d6e/f111 0 2024-09-16T08:57:32.070 INFO:tasks.workunit.client.0.smithi031.stdout:1/866: sync 2024-09-16T08:57:32.070 INFO:tasks.workunit.client.0.smithi031.stdout:0/997: sync 2024-09-16T08:57:32.072 INFO:tasks.workunit.client.0.smithi031.stdout:0/998: mknod d6/d12/d97/c13b 0 2024-09-16T08:57:32.081 INFO:tasks.workunit.client.0.smithi031.stdout:7/977: dwrite d2/d8/d11/d5d/d7e/f84 [0,4194304] 0 2024-09-16T08:57:32.085 INFO:tasks.workunit.client.0.smithi031.stdout:0/999: write d6/d12/d97/d9e/fb8 [2680958,29365] 0 2024-09-16T08:57:32.097 INFO:tasks.workunit.client.0.smithi031.stdout:7/978: dwrite d2/d8/d11/f44 [0,4194304] 0 2024-09-16T08:57:32.100 INFO:tasks.workunit.client.0.smithi031.stdout:7/979: truncate d2/d8/d11/d66/d9a/fde 315419 0 2024-09-16T08:57:32.101 INFO:tasks.workunit.client.0.smithi031.stdout:7/980: truncate d2/f22 1803945 0 2024-09-16T08:57:32.101 INFO:tasks.workunit.client.0.smithi031.stdout:7/981: write d2/f6a [3789568,71803] 0 2024-09-16T08:57:32.108 INFO:tasks.workunit.client.0.smithi031.stdout:7/982: dread d2/d8/f2f [0,4194304] 0 2024-09-16T08:57:32.110 INFO:tasks.workunit.client.0.smithi031.stdout:7/983: rename d2/d8/d86/dd4/ce9 to d2/d8/d11/d81/c146 0 2024-09-16T08:57:32.110 INFO:tasks.workunit.client.0.smithi031.stdout:7/984: write d2/d8/d11/d66/f103 [511315,2474] 0 2024-09-16T08:57:32.110 INFO:tasks.workunit.client.0.smithi031.stdout:7/985: stat d2/d8/d11/d81/c146 0 2024-09-16T08:57:32.112 INFO:tasks.workunit.client.0.smithi031.stdout:7/986: mkdir d2/d8/d11/d5d/d7e/db6/d13a/d114/d147 0 2024-09-16T08:57:32.114 INFO:tasks.workunit.client.0.smithi031.stdout:7/987: creat d2/d8/d86/dd4/df1/f148 x:0 0 0 2024-09-16T08:57:32.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:31 smithi031 ceph-mon[27844]: pgmap v17: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 276 MiB/s rd, 346 MiB/s wr, 529 op/s 2024-09-16T08:57:32.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:31 smithi097 ceph-mon[35478]: pgmap v17: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 276 MiB/s rd, 346 MiB/s wr, 529 op/s 2024-09-16T08:57:32.272 INFO:tasks.workunit.client.0.smithi031.stdout:8/883: sync 2024-09-16T08:57:32.272 INFO:tasks.workunit.client.0.smithi031.stdout:5/859: sync 2024-09-16T08:57:32.272 INFO:tasks.workunit.client.0.smithi031.stdout:8/884: write d4/d7/d99/fb6 [3947121,126054] 0 2024-09-16T08:57:32.272 INFO:tasks.workunit.client.0.smithi031.stdout:8/885: truncate d4/fc5 633842 0 2024-09-16T08:57:32.281 INFO:tasks.workunit.client.0.smithi031.stdout:5/860: write d2/dc/f2d [2623435,63956] 0 2024-09-16T08:57:32.687 INFO:tasks.workunit.client.0.smithi031.stdout:5/861: write d2/dc/d3d/d4c/fbd [470969,110661] 0 2024-09-16T08:57:32.687 INFO:tasks.workunit.client.0.smithi031.stdout:5/862: fdatasync d2/dc/d25/d68/f112 0 2024-09-16T08:57:32.687 INFO:tasks.workunit.client.0.smithi031.stdout:5/863: dread - d2/dc/d91/f114 zero size 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:8/886: symlink d4/d9e/d38/dc8/dee/l118 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:8/887: truncate d4/d2f/dc3/d37/d46/db1/f6a 1482094 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:8/888: rename d4/d9e/c92 to d4/d2d/d6e/c119 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:5/864: dread d2/dc/d25/d68/f6b [0,4194304] 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:5/865: getdents d2/dc/d1d/d97 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:5/866: fsync d2/dc/d91/fdc 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:5/867: fdatasync d2/f11 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:5/868: symlink d2/dc/d25/l121 0 2024-09-16T08:57:32.697 INFO:tasks.workunit.client.0.smithi031.stdout:8/889: dread d4/d2f/dc3/d37/d46/d8c/f58 [0,4194304] 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/869: rename d2/dc/d3d/d4c/lb5 to d2/dc/d25/d21/l122 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/870: dread - d2/dc/d91/fd3 zero size 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/871: readlink d2/dc/d25/l40 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/890: rename d4/d7 to d4/d77/ddc/d11a 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/891: creat d4/d9e/f11b x:0 0 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/892: fsync d4/f6 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/872: rename d2/dc/c5e to d2/dc/d25/da2/d11b/c123 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/893: rename d4/d2d/d6e/fa4 to d4/d5b/d8d/dcd/f11c 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/873: truncate d2/dc/d25/d21/d2e/f72 434393 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/894: mkdir d4/d77/ddc/d11a/d66/de5/d11d 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/895: stat d4/d2f/c107 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:5/874: dread d2/dc/d36/f41 [0,4194304] 0 2024-09-16T08:57:32.698 INFO:tasks.workunit.client.0.smithi031.stdout:8/896: mkdir d4/d2d/d11e 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:8/897: read d4/d9e/d38/f87 [3480482,91222] 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:8/898: creat d4/d2f/dc3/d37/d46/d88/f11f x:0 0 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:8/899: readlink d4/d2f/dc3/d37/d46/db1/lb3 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:8/900: mkdir d4/d77/ddc/d120 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/875: dwrite d2/dc/f1f [0,4194304] 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/876: dread d2/dc/d25/d21/fa3 [0,4194304] 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/877: truncate d2/dc/d25/dc8/ff0 264599 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/878: unlink d2/dc/d25/d21/d2e/f33 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/879: mknod d2/dc/d36/d118/c124 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/880: write d2/dc/d25/d21/f60 [237019,55362] 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/881: fsync d2/dc/d91/fdc 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/882: stat d2/dc/d25/dbf/de4/ced 0 2024-09-16T08:57:32.699 INFO:tasks.workunit.client.0.smithi031.stdout:5/883: readlink d2/dc/d25/lc7 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:5/884: dread d2/f1a [0,4194304] 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:5/885: chown d2/dc/d25/d21/f6d 1935 1 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/901: dwrite d4/d77/ddc/d11a/d99/fc1 [0,4194304] 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/902: write d4/d9e/dea/f10f [1018959,31203] 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:5/886: rename d2/dc/d36/d76/fda to d2/dc/d1d/d97/d11f/f125 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/903: mknod d4/d5b/d8d/dcd/d79/c121 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/904: chown d4/d77/ddc/d11a/d66/dfa/l117 14196 1 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/905: link d4/d77/dd3/feb d4/d2f/dc3/d37/d46/f122 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/906: rename d4/d77/ddc/d11a/f15 to d4/d77/ddc/d11a/d66/dfa/f123 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/907: chown d4/d2f/dc3/d37/d46/l5c 9492851 1 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/908: link d4/d9e/d5e/c84 d4/d2f/dc3/d24/c124 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/909: rename d4/d5b/d8d/dcd/d50/f98 to d4/d5b/d7b/f125 0 2024-09-16T08:57:32.700 INFO:tasks.workunit.client.0.smithi031.stdout:8/910: chown d4/d77/ddc/d11a/d66/de5 1129179462 1 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:8/911: getdents d4/d77/dd3 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/887: dread d2/dc/f1f [0,4194304] 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:8/912: mknod d4/d2f/dc3/dcb/c126 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/888: symlink d2/dc/d36/d118/l126 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/889: stat d2/dc/l67 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/890: rmdir d2/dc/d25/da2/d11b 39 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/891: truncate d2/d4b/f79 511384 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/892: creat d2/dc/d8c/de8/f127 x:0 0 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/893: creat d2/dc/d25/d21/d81/f128 x:0 0 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/894: readlink d2/dc/d25/d21/d2e/l35 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/895: truncate d2/dc/d25/dc8/ff0 426692 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/896: creat d2/dc/d1d/dbb/f129 x:0 0 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/897: creat d2/dc/d25/dad/d115/f12a x:0 0 0 2024-09-16T08:57:32.701 INFO:tasks.workunit.client.0.smithi031.stdout:5/898: readlink d2/dc/d3d/l95 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/899: creat d2/dc/d25/d21/f12b x:0 0 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:8/913: dwrite d4/f6 [0,4194304] 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/900: write d2/dc/f3a [1353574,107299] 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:8/914: mkdir d4/d2d/d6e/d127 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/901: mkdir d2/dc/d91/d12c 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/902: fsync d2/dc/d36/fca 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/903: chown d2/dc/d25/l40 28153 1 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:1/867: sync 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:7/988: sync 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:1/868: write d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/fcf [241772,64852] 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:5/904: creat d2/dc/d25/d21/f12d x:0 0 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:7/989: rename d2/d8/d86/l93 to d2/d8/d11/d5d/d7e/db6/d13a/l149 0 2024-09-16T08:57:32.702 INFO:tasks.workunit.client.0.smithi031.stdout:7/990: dread - d2/d8/d11/d81/fbf zero size 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/991: creat d2/d8/d86/da1/da9/f14a x:0 0 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/992: chown d2/d8/d11/d5d/d6e/c131 25620754 1 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/869: symlink d2/de/l131 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/870: chown d2/d15/d1f/de9/f54 1858310 1 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/871: rmdir d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0 39 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/905: dread d2/dc/f43 [0,4194304] 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/872: creat d2/d15/d1f/de9/d116/f132 x:0 0 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/906: mkdir d2/dc/d25/dbf/d12e 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/993: dread d2/d8/d11/f40 [0,4194304] 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:1/873: mkdir d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/d133 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/907: getdents d2/dc/d25/da2 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/908: fdatasync d2/dc/d36/f4a 0 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/909: chown d2/dc/ffd 85673 1 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:5/910: chown d2/dc/d25/d21/d81 37205 1 2024-09-16T08:57:32.703 INFO:tasks.workunit.client.0.smithi031.stdout:7/994: mknod d2/d8/d86/c14b 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:1/874: rename d2/d15/d1f/de9/d1b/d5c/f66 to d2/d15/d1f/de9/d52/f134 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/911: mknod d2/dc/d25/d68/c12f 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:1/875: mknod d2/d15/d5d/def/c135 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:1/876: creat d2/d15/d1f/de9/d1b/f136 x:0 0 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/912: creat d2/dc/d36/f130 x:0 0 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/995: mkdir d2/d8/d11/d14c 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:1/877: unlink d2/d15/d1f/de9/d52/f6f 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/996: dread d2/fee [0,4194304] 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/913: dwrite d2/dc/d36/fae [0,4194304] 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/997: dwrite d2/d8/d11/d5d/d7e/f119 [0,4194304] 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/998: truncate d2/d8/d86/da1/da9/f14a 43942 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:7/999: chown d2/d8/d11/d5d/f73 3 1 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/914: link d2/f13 d2/f131 0 2024-09-16T08:57:32.704 INFO:tasks.workunit.client.0.smithi031.stdout:5/915: dwrite d2/dc/d25/d21/d2e/fa5 [0,4194304] 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/916: read d2/dc/df3/f106 [134189,73383] 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/917: symlink d2/d100/l132 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/918: dwrite d2/dc/d36/f4a [4194304,4194304] 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/919: write d2/dc/f88 [1173629,49249] 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/920: mkdir d2/dc/d25/d21/d2e/d133 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/921: dread d2/dc/f19 [0,4194304] 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/922: readlink d2/dc/d1d/dbb/ldb 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/923: write d2/dc/ffd [543674,114257] 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/924: readlink d2/dc/d3d/db2/lce 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/925: fdatasync d2/d4b/f8a 0 2024-09-16T08:57:32.705 INFO:tasks.workunit.client.0.smithi031.stdout:5/926: link d2/dc/d25/c69 d2/dc/d91/c134 0 2024-09-16T08:57:32.797 INFO:tasks.workunit.client.0.smithi031.stdout:8/915: sync 2024-09-16T08:57:32.812 INFO:tasks.workunit.client.0.smithi031.stdout:8/916: dwrite d4/fb5 [4194304,4194304] 0 2024-09-16T08:57:32.812 INFO:tasks.workunit.client.0.smithi031.stdout:8/917: fdatasync d4/d2f/dc3/d37/d46/db1/f6a 0 2024-09-16T08:57:32.819 INFO:tasks.workunit.client.0.smithi031.stdout:8/918: dread d4/d2f/dc3/d37/d46/d8c/fd9 [0,4194304] 0 2024-09-16T08:57:32.819 INFO:tasks.workunit.client.0.smithi031.stdout:8/919: chown d4/d2f/dc3/d24/c2c 21043 1 2024-09-16T08:57:32.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/920: creat d4/d9e/d5e/f128 x:0 0 0 2024-09-16T08:57:32.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/921: write d4/d77/dd3/fdb [152711,116352] 0 2024-09-16T08:57:32.820 INFO:tasks.workunit.client.0.smithi031.stdout:8/922: fsync d4/d5b/d8d/dcd/d50/fe7 0 2024-09-16T08:57:32.835 INFO:tasks.workunit.client.0.smithi031.stdout:8/923: dwrite d4/d77/ddc/d11a/fa8 [4194304,4194304] 0 2024-09-16T08:57:32.836 INFO:tasks.workunit.client.0.smithi031.stdout:8/924: creat d4/d9e/daa/f129 x:0 0 0 2024-09-16T08:57:32.836 INFO:tasks.workunit.client.0.smithi031.stdout:8/925: write d4/d9e/d5e/f128 [623926,7436] 0 2024-09-16T08:57:32.837 INFO:tasks.workunit.client.0.smithi031.stdout:8/926: rmdir d4/d9e/d38 39 2024-09-16T08:57:32.837 INFO:tasks.workunit.client.0.smithi031.stdout:8/927: fsync d4/d5b/d7b/f7d 0 2024-09-16T08:57:32.975 INFO:tasks.workunit.client.0.smithi031.stdout:5/927: sync 2024-09-16T08:57:32.975 INFO:tasks.workunit.client.0.smithi031.stdout:1/878: sync 2024-09-16T08:57:32.975 INFO:tasks.workunit.client.0.smithi031.stdout:5/928: creat d2/dc/d25/d21/d81/f135 x:0 0 0 2024-09-16T08:57:32.991 INFO:tasks.workunit.client.0.smithi031.stdout:5/929: dwrite d2/dc/d91/f114 [0,4194304] 0 2024-09-16T08:57:32.993 INFO:tasks.workunit.client.0.smithi031.stdout:5/930: symlink d2/dc/d3d/l136 0 2024-09-16T08:57:32.994 INFO:tasks.workunit.client.0.smithi031.stdout:5/931: stat d2/dc/d3d/lc4 0 2024-09-16T08:57:32.994 INFO:tasks.workunit.client.0.smithi031.stdout:5/932: mkdir d2/dc/d25/dad/d137 0 2024-09-16T08:57:32.995 INFO:tasks.workunit.client.0.smithi031.stdout:5/933: read - d2/dc/d8c/de8/f107 zero size 2024-09-16T08:57:33.008 INFO:tasks.workunit.client.0.smithi031.stdout:5/934: dwrite d2/dc/df3/f10b [0,4194304] 0 2024-09-16T08:57:33.009 INFO:tasks.workunit.client.0.smithi031.stdout:5/935: rmdir d2/d4b 39 2024-09-16T08:57:33.010 INFO:tasks.workunit.client.0.smithi031.stdout:5/936: symlink d2/dc/d1d/d97/l138 0 2024-09-16T08:57:33.013 INFO:tasks.workunit.client.0.smithi031.stdout:5/937: dread d2/dc/d25/d21/f85 [0,4194304] 0 2024-09-16T08:57:33.014 INFO:tasks.workunit.client.0.smithi031.stdout:5/938: truncate d2/d4b/f8a 1451303 0 2024-09-16T08:57:33.037 INFO:tasks.workunit.client.0.smithi031.stdout:5/939: dwrite d2/dc/d25/d21/d81/f128 [0,4194304] 0 2024-09-16T08:57:33.039 INFO:tasks.workunit.client.0.smithi031.stdout:5/940: symlink d2/dc/d25/da2/d11b/l139 0 2024-09-16T08:57:33.056 INFO:tasks.workunit.client.0.smithi031.stdout:5/941: dwrite d2/dc/d36/ff5 [0,4194304] 0 2024-09-16T08:57:33.058 INFO:tasks.workunit.client.0.smithi031.stdout:5/942: creat d2/dc/d25/da2/f13a x:0 0 0 2024-09-16T08:57:33.065 INFO:tasks.workunit.client.0.smithi031.stdout:5/943: write d2/dc/d25/d68/f6e [2029733,89870] 0 2024-09-16T08:57:33.066 INFO:tasks.workunit.client.0.smithi031.stdout:5/944: chown d2/dc/d3d/l3e 12296214 1 2024-09-16T08:57:33.066 INFO:tasks.workunit.client.0.smithi031.stdout:5/945: mkdir d2/d100/d13b 0 2024-09-16T08:57:33.067 INFO:tasks.workunit.client.0.smithi031.stdout:5/946: link d2/dc/d25/dc8/ld0 d2/dc/d3d/db2/l13c 0 2024-09-16T08:57:33.067 INFO:tasks.workunit.client.0.smithi031.stdout:5/947: fdatasync d2/dc/d3d/d4c/f11e 0 2024-09-16T08:57:33.069 INFO:tasks.workunit.client.0.smithi031.stdout:5/948: link d2/dc/d25/d21/f26 d2/dc/d25/d21/d81/f13d 0 2024-09-16T08:57:33.070 INFO:tasks.workunit.client.0.smithi031.stdout:5/949: chown d2/dc/f3a 6 1 2024-09-16T08:57:33.070 INFO:tasks.workunit.client.0.smithi031.stdout:5/950: creat d2/dc/d36/f13e x:0 0 0 2024-09-16T08:57:33.071 INFO:tasks.workunit.client.0.smithi031.stdout:5/951: dread d2/dc/d25/f53 [0,4194304] 0 2024-09-16T08:57:33.073 INFO:tasks.workunit.client.0.smithi031.stdout:5/952: rmdir d2/dc/d91/d12c 0 2024-09-16T08:57:33.075 INFO:tasks.workunit.client.0.smithi031.stdout:5/953: link d2/dc/d36/fca d2/dc/d36/d76/d109/f13f 0 2024-09-16T08:57:33.075 INFO:tasks.workunit.client.0.smithi031.stdout:5/954: write d2/dc/d25/d21/f60 [957542,47985] 0 2024-09-16T08:57:33.075 INFO:tasks.workunit.client.0.smithi031.stdout:5/955: fsync d2/dc/d91/fdc 0 2024-09-16T08:57:33.076 INFO:tasks.workunit.client.0.smithi031.stdout:5/956: symlink d2/dc/d25/dad/d115/l140 0 2024-09-16T08:57:33.078 INFO:tasks.workunit.client.0.smithi031.stdout:5/957: creat d2/d4b/f141 x:0 0 0 2024-09-16T08:57:33.078 INFO:tasks.workunit.client.0.smithi031.stdout:5/958: readlink d2/dc/d25/d21/d81/l11c 0 2024-09-16T08:57:33.079 INFO:tasks.workunit.client.0.smithi031.stdout:5/959: symlink d2/dc/d36/daf/l142 0 2024-09-16T08:57:33.081 INFO:tasks.workunit.client.0.smithi031.stdout:5/960: creat d2/dc/d36/d76/d109/f143 x:0 0 0 2024-09-16T08:57:33.081 INFO:tasks.workunit.client.0.smithi031.stdout:5/961: chown d2/dc/df3/c108 187536802 1 2024-09-16T08:57:33.096 INFO:tasks.workunit.client.0.smithi031.stdout:5/962: dwrite d2/dc/df3/f10b [4194304,4194304] 0 2024-09-16T08:57:33.110 INFO:tasks.workunit.client.0.smithi031.stdout:5/963: dwrite d2/dc/d3d/d4c/fe5 [0,4194304] 0 2024-09-16T08:57:33.110 INFO:tasks.workunit.client.0.smithi031.stdout:5/964: write d2/dc/d3d/fb0 [74354,56767] 0 2024-09-16T08:57:33.125 INFO:tasks.workunit.client.0.smithi031.stdout:5/965: dwrite d2/dc/d1d/d97/ff9 [0,4194304] 0 2024-09-16T08:57:33.134 INFO:tasks.workunit.client.0.smithi031.stdout:5/966: rmdir d2/dc/d25/d68 39 2024-09-16T08:57:33.145 INFO:tasks.workunit.client.0.smithi031.stdout:8/928: sync 2024-09-16T08:57:33.145 INFO:tasks.workunit.client.0.smithi031.stdout:1/879: sync 2024-09-16T08:57:33.145 INFO:tasks.workunit.client.0.smithi031.stdout:1/880: chown d2/d15/d38/f11c 88 1 2024-09-16T08:57:33.147 INFO:tasks.workunit.client.0.smithi031.stdout:1/881: creat d2/d15/d5d/db4/da3/f137 x:0 0 0 2024-09-16T08:57:33.149 INFO:tasks.workunit.client.0.smithi031.stdout:1/882: symlink d2/d15/d1f/de9/d1b/d5c/d6b/dc1/l138 0 2024-09-16T08:57:33.149 INFO:tasks.workunit.client.0.smithi031.stdout:1/883: dread - d2/d15/d1f/d43/f11a zero size 2024-09-16T08:57:33.150 INFO:tasks.workunit.client.0.smithi031.stdout:1/884: truncate d2/d15/d1f/d43/f11a 183390 0 2024-09-16T08:57:33.151 INFO:tasks.workunit.client.0.smithi031.stdout:1/885: mknod d2/d15/d1f/de9/d1b/c139 0 2024-09-16T08:57:33.151 INFO:tasks.workunit.client.0.smithi031.stdout:1/886: stat d2/d15/d1f/de9/d1b/d5c/d6b/c81 0 2024-09-16T08:57:33.153 INFO:tasks.workunit.client.0.smithi031.stdout:1/887: symlink d2/d15/d1f/de9/d1b/d5c/d6b/dc1/dd1/l13a 0 2024-09-16T08:57:33.153 INFO:tasks.workunit.client.0.smithi031.stdout:1/888: readlink d2/d15/d1f/de9/d1b/d5c/d6b/dc1/dd1/lf1 0 2024-09-16T08:57:33.156 INFO:tasks.workunit.client.0.smithi031.stdout:1/889: mkdir d2/d15/d32/d87/d13b 0 2024-09-16T08:57:33.159 INFO:tasks.workunit.client.0.smithi031.stdout:1/890: unlink d2/f14 0 2024-09-16T08:57:33.160 INFO:tasks.workunit.client.0.smithi031.stdout:1/891: mknod d2/d15/d1f/de9/d105/db9/c13c 0 2024-09-16T08:57:33.163 INFO:tasks.workunit.client.0.smithi031.stdout:1/892: rename d2/d15/d32/d128 to d2/d15/d5d/db4/da3/dcb/d13d 0 2024-09-16T08:57:33.164 INFO:tasks.workunit.client.0.smithi031.stdout:1/893: creat d2/d15/d5d/db4/de3/f13e x:0 0 0 2024-09-16T08:57:33.164 INFO:tasks.workunit.client.0.smithi031.stdout:1/894: chown d2/d15/d1f/de9/d67/fbc 82 1 2024-09-16T08:57:33.165 INFO:tasks.workunit.client.0.smithi031.stdout:8/929: dwrite d4/fc5 [0,4194304] 0 2024-09-16T08:57:33.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/895: link d2/d15/d1f/de9/d52/f74 d2/f13f 0 2024-09-16T08:57:33.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/896: chown d2/d15/d1f/de9/d1b/f12e 0 1 2024-09-16T08:57:33.166 INFO:tasks.workunit.client.0.smithi031.stdout:1/897: dread - d2/d15/d1f/f62 zero size 2024-09-16T08:57:33.169 INFO:tasks.workunit.client.0.smithi031.stdout:8/930: creat d4/d77/ddc/d11a/d66/df1/f12a x:0 0 0 2024-09-16T08:57:33.169 INFO:tasks.workunit.client.0.smithi031.stdout:8/931: write d4/d77/ddc/d11a/d66/df1/f110 [891635,106191] 0 2024-09-16T08:57:33.169 INFO:tasks.workunit.client.0.smithi031.stdout:1/898: creat d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/d115/f140 x:0 0 0 2024-09-16T08:57:33.171 INFO:tasks.workunit.client.0.smithi031.stdout:8/932: creat d4/d2f/dc3/d37/f12b x:0 0 0 2024-09-16T08:57:33.171 INFO:tasks.workunit.client.0.smithi031.stdout:8/933: truncate d4/d2f/dc3/d24/fec 1574252 0 2024-09-16T08:57:33.172 INFO:tasks.workunit.client.0.smithi031.stdout:1/899: readlink d2/d15/d5d/db4/da3/laa 0 2024-09-16T08:57:33.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/934: mkdir d4/d2d/d12c 0 2024-09-16T08:57:33.173 INFO:tasks.workunit.client.0.smithi031.stdout:8/935: chown d4/d2f/dc3/d37/d46/d115 767 1 2024-09-16T08:57:33.174 INFO:tasks.workunit.client.0.smithi031.stdout:1/900: creat d2/de/d6c/f141 x:0 0 0 2024-09-16T08:57:33.176 INFO:tasks.workunit.client.0.smithi031.stdout:1/901: mknod d2/d15/d1f/de9/d1b/d5c/d6b/c142 0 2024-09-16T08:57:33.176 INFO:tasks.workunit.client.0.smithi031.stdout:1/902: dread - d2/de/d6c/f141 zero size 2024-09-16T08:57:33.177 INFO:tasks.workunit.client.0.smithi031.stdout:1/903: symlink d2/d15/d1f/de9/d1b/d5c/d6b/dc1/l143 0 2024-09-16T08:57:33.179 INFO:tasks.workunit.client.0.smithi031.stdout:1/904: creat d2/d15/d1f/de9/d1b/d5c/d6b/dc1/df0/f144 x:0 0 0 2024-09-16T08:57:33.180 INFO:tasks.workunit.client.0.smithi031.stdout:1/905: creat d2/d15/d1f/de9/f145 x:0 0 0 2024-09-16T08:57:33.180 INFO:tasks.workunit.client.0.smithi031.stdout:1/906: chown d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0 177 1 2024-09-16T08:57:33.195 INFO:tasks.workunit.client.0.smithi031.stdout:1/907: dwrite d2/d15/d38/f9a [0,4194304] 0 2024-09-16T08:57:33.200 INFO:tasks.workunit.client.0.smithi031.stdout:1/908: mkdir d2/d15/d1f/de9/d52/d146 0 2024-09-16T08:57:33.202 INFO:tasks.workunit.client.0.smithi031.stdout:1/909: symlink d2/d15/d5d/db4/da3/dcb/d13d/l147 0 2024-09-16T08:57:33.204 INFO:tasks.workunit.client.0.smithi031.stdout:1/910: unlink d2/d15/d1f/de9/d1b/f136 0 2024-09-16T08:57:33.217 INFO:tasks.workunit.client.0.smithi031.stdout:1/911: dwrite d2/d15/d38/f47 [0,4194304] 0 2024-09-16T08:57:33.217 INFO:tasks.workunit.client.0.smithi031.stdout:1/912: stat d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda 0 2024-09-16T08:57:33.226 INFO:tasks.workunit.client.0.smithi031.stdout:1/913: dread d2/d15/d1f/de9/d1b/d5c/d6b/f7b [0,4194304] 0 2024-09-16T08:57:33.227 INFO:tasks.workunit.client.0.smithi031.stdout:1/914: creat d2/d15/d1f/de9/f148 x:0 0 0 2024-09-16T08:57:33.229 INFO:tasks.workunit.client.0.smithi031.stdout:1/915: mknod d2/de/c149 0 2024-09-16T08:57:33.229 INFO:tasks.workunit.client.0.smithi031.stdout:1/916: chown d2/d15/d5d/d12b 11844622 1 2024-09-16T08:57:33.245 INFO:tasks.workunit.client.0.smithi031.stdout:1/917: dwrite d2/d15/d1f/de9/d1b/d5c/d6b/f7b [0,4194304] 0 2024-09-16T08:57:33.246 INFO:tasks.workunit.client.0.smithi031.stdout:1/918: unlink d2/d15/d1f/de9/d1b/c6e 0 2024-09-16T08:57:33.246 INFO:tasks.workunit.client.0.smithi031.stdout:1/919: stat d2/d15/d1f/de9/d1b/d5c/f8e 0 2024-09-16T08:57:33.246 INFO:tasks.workunit.client.0.smithi031.stdout:1/920: truncate d2/d15/d38/fc6 377702 0 2024-09-16T08:57:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/921: unlink d2/d15/d38/f9a 0 2024-09-16T08:57:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/922: write d2/d15/d1f/de9/d105/db9/f10f [543190,128800] 0 2024-09-16T08:57:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/923: chown d2/d15/d1f/de9/d1b/d5c/d6b/dc1/fd2 59849 1 2024-09-16T08:57:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/924: fdatasync d2/d15/d1f/de9/d52/fcc 0 2024-09-16T08:57:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/925: truncate d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/d115/f140 540937 0 2024-09-16T08:57:33.248 INFO:tasks.workunit.client.0.smithi031.stdout:1/926: mkdir d2/d15/d1f/de9/d52/d14a 0 2024-09-16T08:57:33.249 INFO:tasks.workunit.client.0.smithi031.stdout:1/927: chown d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea/f8b 0 1 2024-09-16T08:57:33.249 INFO:tasks.workunit.client.0.smithi031.stdout:1/928: creat d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/f14b x:0 0 0 2024-09-16T08:57:33.250 INFO:tasks.workunit.client.0.smithi031.stdout:1/929: rename d2/d15/d1f/de9/l11 to d2/l14c 0 2024-09-16T08:57:33.256 INFO:tasks.workunit.client.0.smithi031.stdout:1/930: dread d2/d15/d1f/de9/d52/fcc [0,4194304] 0 2024-09-16T08:57:33.258 INFO:tasks.workunit.client.0.smithi031.stdout:1/931: rename d2/d15/d1f/de9/d1b/d5c/d6b/dc1/l143 to d2/d15/d1f/d111/l14d 0 2024-09-16T08:57:33.441 INFO:tasks.workunit.client.0.smithi031.stdout:5/967: sync 2024-09-16T08:57:33.441 INFO:tasks.workunit.client.0.smithi031.stdout:5/968: creat d2/dc/d91/f144 x:0 0 0 2024-09-16T08:57:33.441 INFO:tasks.workunit.client.0.smithi031.stdout:5/969: chown d2/dc/f3b 11883 1 2024-09-16T08:57:33.708 INFO:tasks.workunit.client.0.smithi031.stdout:8/936: sync 2024-09-16T08:57:33.709 INFO:tasks.workunit.client.0.smithi031.stdout:1/932: sync 2024-09-16T08:57:33.709 INFO:tasks.workunit.client.0.smithi031.stdout:8/937: fsync d4/d9e/d38/fe0 0 2024-09-16T08:57:33.709 INFO:tasks.workunit.client.0.smithi031.stdout:1/933: chown d2/d15/d1f/de9/d67/f72 3 1 2024-09-16T08:57:33.709 INFO:tasks.workunit.client.0.smithi031.stdout:1/934: fsync d2/d15/d38/f47 0 2024-09-16T08:57:33.709 INFO:tasks.workunit.client.0.smithi031.stdout:8/938: creat d4/d9e/d38/f12d x:0 0 0 2024-09-16T08:57:33.710 INFO:tasks.workunit.client.0.smithi031.stdout:1/935: link d2/d15/d1f/de9/d1b/f3b d2/d15/d1f/de9/d1b/d5c/f14e 0 2024-09-16T08:57:33.713 INFO:tasks.workunit.client.0.smithi031.stdout:1/936: link d2/de/db5/fc2 d2/d15/d1f/d111/f14f 0 2024-09-16T08:57:33.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/937: link d2/d15/d1f/d111/f5b d2/d15/d1f/f150 0 2024-09-16T08:57:33.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/938: fsync d2/de/db5/ff7 0 2024-09-16T08:57:33.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/939: creat d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/d115/f151 x:0 0 0 2024-09-16T08:57:33.716 INFO:tasks.workunit.client.0.smithi031.stdout:1/940: read - d2/d15/d1f/de9/d1b/fb3 zero size 2024-09-16T08:57:33.717 INFO:tasks.workunit.client.0.smithi031.stdout:1/941: chown d2/d15/d1f/de9/l41 11999109 1 2024-09-16T08:57:33.722 INFO:tasks.workunit.client.0.smithi031.stdout:8/939: dwrite d4/d77/ddc/d11a/d66/de5/f113 [0,4194304] 0 2024-09-16T08:57:33.735 INFO:tasks.workunit.client.0.smithi031.stdout:8/940: read d4/f6 [3321951,57601] 0 2024-09-16T08:57:33.736 INFO:tasks.workunit.client.0.smithi031.stdout:8/941: dread d4/d2f/dc3/d37/fc4 [0,4194304] 0 2024-09-16T08:57:33.736 INFO:tasks.workunit.client.0.smithi031.stdout:8/942: chown d4/d9e/d38 4460 1 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/943: read d4/d2f/dc3/dcb/ff0 [254384,77718] 0 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/944: creat d4/d2f/dc3/d37/d46/f12e x:0 0 0 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/945: write d4/d2f/fbc [1802085,40010] 0 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/946: readlink d4/l1d 0 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/947: dread - d4/d9e/d38/fe0 zero size 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/948: readlink d4/l1d 0 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/949: chown d4/d2f/dc3/f23 21971190 1 2024-09-16T08:57:33.738 INFO:tasks.workunit.client.0.smithi031.stdout:8/950: fsync d4/d5b/fd4 0 2024-09-16T08:57:33.739 INFO:tasks.workunit.client.0.smithi031.stdout:8/951: truncate d4/d2f/fbd 145995 0 2024-09-16T08:57:33.742 INFO:tasks.workunit.client.0.smithi031.stdout:8/952: link d4/d9e/d5e/ff5 d4/d9e/f12f 0 2024-09-16T08:57:33.742 INFO:tasks.workunit.client.0.smithi031.stdout:8/953: chown d4/d77/ddc/d11a 279104 1 2024-09-16T08:57:33.742 INFO:tasks.workunit.client.0.smithi031.stdout:1/942: dwrite d2/d15/d1f/de9/f1c [0,4194304] 0 2024-09-16T08:57:33.743 INFO:tasks.workunit.client.0.smithi031.stdout:8/954: symlink d4/d9e/dea/l130 0 2024-09-16T08:57:33.745 INFO:tasks.workunit.client.0.smithi031.stdout:1/943: mknod d2/d15/d1f/de9/d1b/d5c/d6b/c152 0 2024-09-16T08:57:33.747 INFO:tasks.workunit.client.0.smithi031.stdout:1/944: symlink d2/d15/d5d/db4/de3/l153 0 2024-09-16T08:57:33.749 INFO:tasks.workunit.client.0.smithi031.stdout:1/945: unlink d2/d15/d38/f102 0 2024-09-16T08:57:33.749 INFO:tasks.workunit.client.0.smithi031.stdout:1/946: fsync d2/f17 0 2024-09-16T08:57:33.750 INFO:tasks.workunit.client.0.smithi031.stdout:1/947: symlink d2/d15/d1f/l154 0 2024-09-16T08:57:33.751 INFO:tasks.workunit.client.0.smithi031.stdout:1/948: truncate d2/d15/d1f/de9/d52/f113 588341 0 2024-09-16T08:57:33.753 INFO:tasks.workunit.client.0.smithi031.stdout:1/949: mkdir d2/d15/d32/d87/da5/d155 0 2024-09-16T08:57:33.754 INFO:tasks.workunit.client.0.smithi031.stdout:8/955: getdents d4/d9e 0 2024-09-16T08:57:33.754 INFO:tasks.workunit.client.0.smithi031.stdout:8/956: chown d4/d9e/d38/fda 807 1 2024-09-16T08:57:33.755 INFO:tasks.workunit.client.0.smithi031.stdout:1/950: mknod d2/d15/d5d/db4/c156 0 2024-09-16T08:57:33.759 INFO:tasks.workunit.client.0.smithi031.stdout:1/951: link d2/d15/d32/d87/cd6 d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/c157 0 2024-09-16T08:57:33.759 INFO:tasks.workunit.client.0.smithi031.stdout:1/952: chown d2/d15/d1f/de9/d1b/d5c/d6b/f7b 62682799 1 2024-09-16T08:57:33.761 INFO:tasks.workunit.client.0.smithi031.stdout:1/953: mkdir d2/d15/d1f/de9/d52/d146/d158 0 2024-09-16T08:57:33.768 INFO:tasks.workunit.client.0.smithi031.stdout:1/954: getdents d2/d15/d5d/db4 0 2024-09-16T08:57:33.769 INFO:tasks.workunit.client.0.smithi031.stdout:8/957: dwrite d4/d2d/d6e/f10c [0,4194304] 0 2024-09-16T08:57:33.780 INFO:tasks.workunit.client.0.smithi031.stdout:1/955: dread d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0/dda/dea/f8b [0,4194304] 0 2024-09-16T08:57:33.807 INFO:tasks.workunit.client.0.smithi031.stdout:8/958: unlink d4/d2d/d6e/f10c 0 2024-09-16T08:57:33.810 INFO:tasks.workunit.client.0.smithi031.stdout:8/959: dread d4/d77/dd3/fdb [0,4194304] 0 2024-09-16T08:57:33.822 INFO:tasks.workunit.client.0.smithi031.stdout:1/956: mknod d2/d15/d32/d87/d13b/c159 0 2024-09-16T08:57:33.875 INFO:tasks.workunit.client.0.smithi031.stdout:1/957: creat d2/d15/d1f/de9/d52/d146/f15a x:0 0 0 2024-09-16T08:57:33.875 INFO:tasks.workunit.client.0.smithi031.stdout:1/958: chown d2/de 358940 1 2024-09-16T08:57:33.876 INFO:tasks.workunit.client.0.smithi031.stdout:1/959: truncate d2/f17 1258472 0 2024-09-16T08:57:33.895 INFO:tasks.workunit.client.0.smithi031.stdout:8/960: rmdir d4 39 2024-09-16T08:57:33.895 INFO:tasks.workunit.client.0.smithi031.stdout:8/961: write d4/d2f/fbc [2544068,51044] 0 2024-09-16T08:57:33.895 INFO:tasks.workunit.client.0.smithi031.stdout:8/962: symlink d4/d5b/d8d/dcd/l131 0 2024-09-16T08:57:33.896 INFO:tasks.workunit.client.0.smithi031.stdout:8/963: mkdir d4/d5b/d8d/dcd/d50/da3/d10d/d132 0 2024-09-16T08:57:33.897 INFO:tasks.workunit.client.0.smithi031.stdout:8/964: rmdir d4/d2f/dc3/d37/d46 39 2024-09-16T08:57:33.897 INFO:tasks.workunit.client.0.smithi031.stdout:8/965: truncate d4/d2f/dc3/d37/d46/f12e 1029118 0 2024-09-16T08:57:33.899 INFO:tasks.workunit.client.0.smithi031.stdout:8/966: mkdir d4/d2d/d11e/d133 0 2024-09-16T08:57:33.899 INFO:tasks.workunit.client.0.smithi031.stdout:8/967: fsync d4/d9e/d38/f3c 0 2024-09-16T08:57:33.900 INFO:tasks.workunit.client.0.smithi031.stdout:8/968: rename d4/fb5 to d4/d2f/dc3/d37/d46/f134 0 2024-09-16T08:57:33.900 INFO:tasks.workunit.client.0.smithi031.stdout:8/969: truncate d4/d2d/d6e/ff3 1030613 0 2024-09-16T08:57:33.901 INFO:tasks.workunit.client.0.smithi031.stdout:8/970: rmdir d4/d5b/d8d/dcd/d50/da3/d10d 39 2024-09-16T08:57:33.908 INFO:tasks.workunit.client.0.smithi031.stdout:1/960: rmdir d2/d15/d1f/de9/d1b/d5c/d6b/da9/dc0 39 2024-09-16T08:57:33.908 INFO:tasks.workunit.client.0.smithi031.stdout:1/961: fdatasync d2/d15/d1f/de9/d1b/fa6 0 2024-09-16T08:57:33.908 INFO:tasks.workunit.client.0.smithi031.stdout:1/962: stat d2/d15/d1f/de9/f5 0 2024-09-16T08:57:33.910 INFO:tasks.workunit.client.0.smithi031.stdout:8/971: mknod d4/d9e/d38/c135 0 2024-09-16T08:57:33.917 INFO:tasks.workunit.client.0.smithi031.stdout:1/963: write d2/d15/d32/d87/fa0 [2494396,90780] 0 2024-09-16T08:57:33.918 INFO:tasks.workunit.client.0.smithi031.stdout:5/970: sync 2024-09-16T08:57:33.918 INFO:tasks.workunit.client.0.smithi031.stdout:5/971: fsync d2/dc/d91/f10f 0 2024-09-16T08:57:33.918 INFO:tasks.workunit.client.0.smithi031.stdout:5/972: write d2/dc/d25/d21/f12d [627324,99039] 0 2024-09-16T08:57:33.919 INFO:tasks.workunit.client.0.smithi031.stdout:1/964: rename d2/d15/d1f/de9/d52/d146/d158 to d2/d15/dff/d15b 0 2024-09-16T08:57:33.920 INFO:tasks.workunit.client.0.smithi031.stdout:1/965: truncate d2/f3e 662318 0 2024-09-16T08:57:33.920 INFO:tasks.workunit.client.0.smithi031.stdout:8/972: read d4/d2f/dc3/d37/f76 [2708479,99723] 0 2024-09-16T08:57:33.920 INFO:tasks.workunit.client.0.smithi031.stdout:5/973: symlink d2/dc/d36/d76/l145 0 2024-09-16T08:57:33.922 INFO:tasks.workunit.client.0.smithi031.stdout:1/966: rmdir d2/d15/d1f/de9/d105 39 2024-09-16T08:57:33.924 INFO:tasks.workunit.client.0.smithi031.stdout:8/973: creat d4/d5b/d7b/f136 x:0 0 0 2024-09-16T08:57:33.924 INFO:tasks.workunit.client.0.smithi031.stdout:8/974: creat d4/d5b/d7b/f137 x:0 0 0 2024-09-16T08:57:33.927 INFO:tasks.workunit.client.0.smithi031.stdout:8/975: link d4/d2d/f10a d4/d2f/dc3/d37/d46/d8c/f138 0 2024-09-16T08:57:33.934 INFO:tasks.workunit.client.0.smithi031.stdout:1/967: dread d2/d15/d1f/de9/d52/f74 [0,4194304] 0 2024-09-16T08:57:33.947 INFO:tasks.workunit.client.0.smithi031.stdout:1/968: read d2/d15/d1f/de9/d52/fd3 [1089389,22494] 0 2024-09-16T08:57:33.953 INFO:tasks.workunit.client.0.smithi031.stdout:8/976: dwrite d4/d5b/d8d/f102 [0,4194304] 0 2024-09-16T08:57:33.955 INFO:tasks.workunit.client.0.smithi031.stdout:5/974: dwrite d2/dc/d91/fb4 [0,4194304] 0 2024-09-16T08:57:33.955 INFO:tasks.workunit.client.0.smithi031.stdout:8/977: mkdir d4/d9e/daa/d139 0 2024-09-16T08:57:33.957 INFO:tasks.workunit.client.0.smithi031.stdout:5/975: creat d2/d100/d13b/f146 x:0 0 0 2024-09-16T08:57:33.958 INFO:tasks.workunit.client.0.smithi031.stdout:1/969: write d2/d15/d1f/de9/d1b/f3b [3901677,17686] 0 2024-09-16T08:57:33.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/978: symlink d4/d77/ddc/d11a/d66/l13a 0 2024-09-16T08:57:33.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/979: chown d4/d77/ddc/d11a/d99/fb6 0 1 2024-09-16T08:57:33.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/980: readlink d4/d9e/d5e/l63 0 2024-09-16T08:57:33.959 INFO:tasks.workunit.client.0.smithi031.stdout:8/981: fdatasync d4/d2f/dc3/dcb/fb2 0 2024-09-16T08:57:33.960 INFO:tasks.workunit.client.0.smithi031.stdout:8/982: write d4/d5b/d8d/dcd/d50/fe7 [710955,105539] 0 2024-09-16T08:57:33.961 INFO:tasks.workunit.client.0.smithi031.stdout:8/983: symlink d4/d77/ddc/l13b 0 2024-09-16T08:57:33.963 INFO:tasks.workunit.client.0.smithi031.stdout:8/984: unlink d4/d77/ddc/d11a/d99/fc1 0 2024-09-16T08:57:33.965 INFO:tasks.workunit.client.0.smithi031.stdout:8/985: creat d4/d2d/d11e/f13c x:0 0 0 2024-09-16T08:57:33.966 INFO:tasks.workunit.client.0.smithi031.stdout:8/986: creat d4/d77/f13d x:0 0 0 2024-09-16T08:57:33.966 INFO:tasks.workunit.client.0.smithi031.stdout:8/987: chown c1 4178918 1 2024-09-16T08:57:33.968 INFO:tasks.workunit.client.0.smithi031.stdout:8/988: mknod d4/c13e 0 2024-09-16T08:57:33.975 INFO:tasks.workunit.client.0.smithi031.stdout:8/989: write d4/d5b/d7b/f7d [768938,64446] 0 2024-09-16T08:57:33.975 INFO:tasks.workunit.client.0.smithi031.stdout:8/990: read - d4/d2d/d11e/f13c zero size 2024-09-16T08:57:33.976 INFO:tasks.workunit.client.0.smithi031.stdout:8/991: mknod d4/d5b/c13f 0 2024-09-16T08:57:33.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/992: truncate d4/d5b/d8d/dcd/f5d 8309149 0 2024-09-16T08:57:33.977 INFO:tasks.workunit.client.0.smithi031.stdout:8/993: fdatasync d4/d77/dd3/feb 0 2024-09-16T08:57:33.978 INFO:tasks.workunit.client.0.smithi031.stdout:8/994: mkdir d4/d2d/d11e/d133/d140 0 2024-09-16T08:57:33.979 INFO:tasks.workunit.client.0.smithi031.stdout:8/995: rename d4/d9e/dea to d4/d5b/d7b/d141 0 2024-09-16T08:57:33.980 INFO:tasks.workunit.client.0.smithi031.stdout:8/996: dread d4/fb4 [4194304,4194304] 0 2024-09-16T08:57:33.980 INFO:tasks.workunit.client.0.smithi031.stdout:5/976: dwrite d2/dc/d25/d21/f60 [0,4194304] 0 2024-09-16T08:57:33.981 INFO:tasks.workunit.client.0.smithi031.stdout:1/970: dwrite d2/d15/d1f/f62 [0,4194304] 0 2024-09-16T08:57:33.982 INFO:tasks.workunit.client.0.smithi031.stdout:5/977: creat d2/dc/d25/dc8/f147 x:0 0 0 2024-09-16T08:57:33.985 INFO:tasks.workunit.client.0.smithi031.stdout:1/971: getdents d2/d15/d1f/de9/d1b/d5c/d6b/dc1/dd1 0 2024-09-16T08:57:33.985 INFO:tasks.workunit.client.0.smithi031.stdout:1/972: creat d2/d15/d32/d87/f15c x:0 0 0 2024-09-16T08:57:33.985 INFO:tasks.workunit.client.0.smithi031.stdout:1/973: dread - d2/d15/d5d/db4/f126 zero size 2024-09-16T08:57:33.987 INFO:tasks.workunit.client.0.smithi031.stdout:5/978: mknod d2/d4b/c148 0 2024-09-16T08:57:33.990 INFO:tasks.workunit.client.0.smithi031.stdout:8/997: read d4/d77/ddc/d11a/f28 [262762,53285] 0 2024-09-16T08:57:33.990 INFO:tasks.workunit.client.0.smithi031.stdout:5/979: creat d2/dc/d25/d68/f149 x:0 0 0 2024-09-16T08:57:33.990 INFO:tasks.workunit.client.0.smithi031.stdout:5/980: write d2/dc/d3d/d4c/f62 [1428915,60033] 0 2024-09-16T08:57:33.991 INFO:tasks.workunit.client.0.smithi031.stdout:8/998: mknod d4/d2d/d6e/d10b/c142 0 2024-09-16T08:57:33.992 INFO:tasks.workunit.client.0.smithi031.stdout:1/974: write d2/d15/d1f/d111/faf [1633020,27213] 0 2024-09-16T08:57:33.992 INFO:tasks.workunit.client.0.smithi031.stdout:5/981: mknod d2/c14a 0 2024-09-16T08:57:33.993 INFO:tasks.workunit.client.0.smithi031.stdout:8/999: symlink d4/d9e/l143 0 2024-09-16T08:57:33.995 INFO:tasks.workunit.client.0.smithi031.stdout:1/975: truncate d2/d15/d1f/de9/d67/f72 652023 0 2024-09-16T08:57:33.995 INFO:tasks.workunit.client.0.smithi031.stdout:1/976: write d2/d15/d1f/de9/d1b/f12e [117861,107189] 0 2024-09-16T08:57:33.997 INFO:tasks.workunit.client.0.smithi031.stdout:1/977: rename d2/de/f2a to d2/d15/d9c/f15d 0 2024-09-16T08:57:34.012 INFO:tasks.workunit.client.0.smithi031.stdout:1/978: dwrite d2/d15/d38/fc6 [0,4194304] 0 2024-09-16T08:57:34.013 INFO:tasks.workunit.client.0.smithi031.stdout:5/982: truncate d2/dc/d25/d21/f12d 329264 0 2024-09-16T08:57:34.014 INFO:tasks.workunit.client.0.smithi031.stdout:1/979: fdatasync d2/d15/d1f/de9/f5 0 2024-09-16T08:57:34.014 INFO:tasks.workunit.client.0.smithi031.stdout:1/980: chown d2/d15/d1f/de9/d1b/d5c/ff2 71923897 1 2024-09-16T08:57:34.016 INFO:tasks.workunit.client.0.smithi031.stdout:5/983: mknod d2/dc/d1d/c14b 0 2024-09-16T08:57:34.016 INFO:tasks.workunit.client.0.smithi031.stdout:5/984: write d2/dc/d25/d21/d2e/fa8 [1082761,73790] 0 2024-09-16T08:57:34.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/985: symlink d2/dc/d25/da2/d11b/l14c 0 2024-09-16T08:57:34.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/986: write d2/dc/d8c/fc9 [2476532,29620] 0 2024-09-16T08:57:34.018 INFO:tasks.workunit.client.0.smithi031.stdout:5/987: stat d2/dc/f75 0 2024-09-16T08:57:34.019 INFO:tasks.workunit.client.0.smithi031.stdout:5/988: chown d2/d4b/fdd 26 1 2024-09-16T08:57:34.019 INFO:tasks.workunit.client.0.smithi031.stdout:5/989: chown d2/dc/d25/d21/c9d 11069 1 2024-09-16T08:57:34.019 INFO:tasks.workunit.client.0.smithi031.stdout:1/981: mknod d2/d15/d1f/de9/c15e 0 2024-09-16T08:57:34.022 INFO:tasks.workunit.client.0.smithi031.stdout:1/982: unlink d2/d15/d1f/d43/ld4 0 2024-09-16T08:57:34.023 INFO:tasks.workunit.client.0.smithi031.stdout:1/983: mkdir d2/d15/d1f/de9/d1b/d5c/d6b/dc1/d15f 0 2024-09-16T08:57:34.024 INFO:tasks.workunit.client.0.smithi031.stdout:5/990: dread d2/dc/d25/d21/d2e/fa8 [0,4194304] 0 2024-09-16T08:57:34.053 INFO:tasks.workunit.client.0.smithi031.stdout:1/984: dwrite d2/de/ff6 [0,4194304] 0 2024-09-16T08:57:34.053 INFO:tasks.workunit.client.0.smithi031.stdout:1/985: readlink d2/l14c 0 2024-09-16T08:57:34.053 INFO:tasks.workunit.client.0.smithi031.stdout:5/991: dwrite d2/dc/d1d/d97/fb9 [0,4194304] 0 2024-09-16T08:57:34.054 INFO:tasks.workunit.client.0.smithi031.stdout:1/986: mkdir d2/d15/d1f/d160 0 2024-09-16T08:57:34.054 INFO:tasks.workunit.client.0.smithi031.stdout:1/987: dread - d2/d15/d1f/d111/f14f zero size 2024-09-16T08:57:34.058 INFO:tasks.workunit.client.0.smithi031.stdout:1/988: rename d2/d15/d1f/de9/d105/db9 to d2/d15/d5d/db4/de3/d161 0 2024-09-16T08:57:34.058 INFO:tasks.workunit.client.0.smithi031.stdout:1/989: write d2/d15/d5d/def/f11f [463837,2286] 0 2024-09-16T08:57:34.058 INFO:tasks.workunit.client.0.smithi031.stdout:1/990: fsync d2/d15/d1f/de9/fa2 0 2024-09-16T08:57:34.058 INFO:tasks.workunit.client.0.smithi031.stdout:1/991: chown d2/d15/d1f/de9/c15e 2466 1 2024-09-16T08:57:34.058 INFO:tasks.workunit.client.0.smithi031.stdout:1/992: mkdir d2/d15/d5d/db4/de3/d162 0 2024-09-16T08:57:34.060 INFO:tasks.workunit.client.0.smithi031.stdout:1/993: mkdir d2/d15/d5d/def/d163 0 2024-09-16T08:57:34.061 INFO:tasks.workunit.client.0.smithi031.stdout:1/994: creat d2/d15/f164 x:0 0 0 2024-09-16T08:57:34.062 INFO:tasks.workunit.client.0.smithi031.stdout:5/992: write d2/dc/d1d/d97/fb9 [2731253,130106] 0 2024-09-16T08:57:34.063 INFO:tasks.workunit.client.0.smithi031.stdout:1/995: truncate d2/d15/d1f/de9/d1b/d5c/d6b/dc1/fd2 83369 0 2024-09-16T08:57:34.064 INFO:tasks.workunit.client.0.smithi031.stdout:5/993: read d2/d4b/f79 [286744,94545] 0 2024-09-16T08:57:34.066 INFO:tasks.workunit.client.0.smithi031.stdout:1/996: mkdir d2/d15/d32/d165 0 2024-09-16T08:57:34.066 INFO:tasks.workunit.client.0.smithi031.stdout:5/994: creat d2/dc/d1d/d97/d11f/f14d x:0 0 0 2024-09-16T08:57:34.068 INFO:tasks.workunit.client.0.smithi031.stdout:1/997: mkdir d2/d15/d5d/db4/de3/d162/d166 0 2024-09-16T08:57:34.068 INFO:tasks.workunit.client.0.smithi031.stdout:1/998: write d2/d15/d1f/de9/d116/f132 [956830,45794] 0 2024-09-16T08:57:34.068 INFO:tasks.workunit.client.0.smithi031.stdout:1/999: read - d2/d15/d1f/de9/d1b/d5c/d6b/dc1/df0/f144 zero size 2024-09-16T08:57:34.069 INFO:tasks.workunit.client.0.smithi031.stdout:5/995: truncate d2/dc/d36/ff5 1412221 0 2024-09-16T08:57:34.069 INFO:tasks.workunit.client.0.smithi031.stdout:5/996: truncate d2/f11 2108804 0 2024-09-16T08:57:34.069 INFO:tasks.workunit.client.0.smithi031.stdout:5/997: fsync d2/dc/d25/d21/f26 0 2024-09-16T08:57:34.086 INFO:tasks.workunit.client.0.smithi031.stdout:5/998: dwrite d2/dc/d1d/d97/d11f/f125 [4194304,4194304] 0 2024-09-16T08:57:34.086 INFO:tasks.workunit.client.0.smithi031.stdout:5/999: readlink d2/dc/d36/daf/l142 0 2024-09-16T08:57:34.089 INFO:tasks.workunit.client.0.smithi031.stderr:+ rm -rf -- ./tmp.sDrlXt8OLN 2024-09-16T08:57:34.108 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:33 smithi097 ceph-mon[35478]: pgmap v18: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 169 MiB/s rd, 212 MiB/s wr, 334 op/s 2024-09-16T08:57:34.122 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:33 smithi031 ceph-mon[27844]: pgmap v18: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 169 MiB/s rd, 212 MiB/s wr, 334 op/s 2024-09-16T08:57:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:34 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:35.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:34 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:36.117 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:35 smithi031 ceph-mon[27844]: pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 205 MiB/s rd, 263 MiB/s wr, 457 op/s 2024-09-16T08:57:36.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:35 smithi097 ceph-mon[35478]: pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 205 MiB/s rd, 263 MiB/s wr, 457 op/s 2024-09-16T08:57:38.074 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:37 smithi031 ceph-mon[27844]: pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 104 MiB/s rd, 132 MiB/s wr, 269 op/s 2024-09-16T08:57:38.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:37 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:38.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:37 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:38.075 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:37 smithi031 ceph-mon[27844]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-09-16T08:57:38.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:37 smithi097 ceph-mon[35478]: pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 104 MiB/s rd, 132 MiB/s wr, 269 op/s 2024-09-16T08:57:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:37 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:37 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:37 smithi097 ceph-mon[35478]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-09-16T08:57:38.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:38 smithi031 ceph-mon[27844]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-09-16T08:57:39.097 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T08:57:39.097 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-16T08:57:39.159 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:38 smithi097 ceph-mon[35478]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-09-16T08:57:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:39 smithi031 ceph-mon[27844]: pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 104 MiB/s rd, 132 MiB/s wr, 269 op/s 2024-09-16T08:57:40.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:39 smithi097 ceph-mon[35478]: pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 104 MiB/s rd, 132 MiB/s wr, 269 op/s 2024-09-16T08:57:41.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:41 smithi031 ceph-mon[27844]: pgmap v22: 65 pgs: 65 active+clean; 1.8 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 106 MiB/s rd, 135 MiB/s wr, 362 op/s 2024-09-16T08:57:42.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:41 smithi097 ceph-mon[35478]: pgmap v22: 65 pgs: 65 active+clean; 1.8 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 106 MiB/s rd, 135 MiB/s wr, 362 op/s 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: Reconfiguring ceph-exporter.smithi097 (dependencies changed)... 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: Deploying daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:57:44.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:43 smithi097 ceph-mon[35478]: pgmap v23: 65 pgs: 65 active+clean; 1.8 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 38 MiB/s rd, 54 MiB/s wr, 216 op/s 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: Reconfiguring ceph-exporter.smithi097 (dependencies changed)... 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: Deploying daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:57:44.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:43 smithi031 ceph-mon[27844]: pgmap v23: 65 pgs: 65 active+clean; 1.8 GiB data, 6.6 GiB used, 530 GiB / 536 GiB avail; 38 MiB/s rd, 54 MiB/s wr, 216 op/s 2024-09-16T08:57:46.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:45 smithi031 ceph-mon[27844]: pgmap v24: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 38 MiB/s rd, 55 MiB/s wr, 339 op/s 2024-09-16T08:57:46.152 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:45 smithi097 ceph-mon[35478]: pgmap v24: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 38 MiB/s rd, 55 MiB/s wr, 339 op/s 2024-09-16T08:57:47.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 -- 172.21.15.31:0/2141976070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 msgr2=0x7f93d8102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:47.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 --2- 172.21.15.31:0/2141976070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 0x7f93d8102190 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009880 tx=0x7f93c802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:57:47.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 -- 172.21.15.31:0/2141976070 shutdown_connections 2024-09-16T08:57:47.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 --2- 172.21.15.31:0/2141976070 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 0x7f93d8103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:47.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 --2- 172.21.15.31:0/2141976070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 0x7f93d8102190 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:47.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 -- 172.21.15.31:0/2141976070 >> 172.21.15.31:0/2141976070 conn(0x7f93d80fd540 msgr2=0x7f93d80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:47.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 -- 172.21.15.31:0/2141976070 shutdown_connections 2024-09-16T08:57:47.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.887+0000 7f93dd346640 1 -- 172.21.15.31:0/2141976070 wait complete. 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.888+0000 7f93dd346640 1 Processor -- start 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.888+0000 7f93dd346640 1 -- start start 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93dd346640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 0x7f93d81977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93dd346640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 0x7f93d8197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93dd346640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d81982f0 con 0x7f93d8101d90 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93dd346640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d8198460 con 0x7f93d8102f90 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93d67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 0x7f93d8197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93d67fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 0x7f93d8197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:44514/0 (socket says 172.21.15.31:44514) 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93d67fc640 1 -- 172.21.15.31:0/2661988696 learned_addr learned my addr 172.21.15.31:0/2661988696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93d6ffd640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 0x7f93d81977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:47.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93d67fc640 1 -- 172.21.15.31:0/2661988696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 msgr2=0x7f93d81977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93d67fc640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 0x7f93d81977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.889+0000 7f93d67fc640 1 -- 172.21.15.31:0/2661988696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8009560 con 0x7f93d8102f90 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.890+0000 7f93d67fc640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 0x7f93d8197d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00d8d0 tx=0x7f93cc00dd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.890+0000 7f93bffff640 1 -- 172.21.15.31:0/2661988696 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93cc00bea0 con 0x7f93d8102f90 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.890+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d819cf00 con 0x7f93d8102f90 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.890+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d819d4a0 con 0x7f93d8102f90 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.890+0000 7f93bffff640 1 -- 172.21.15.31:0/2661988696 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93cc009c00 con 0x7f93d8102f90 2024-09-16T08:57:47.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.890+0000 7f93bffff640 1 -- 172.21.15.31:0/2661988696 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93cc00f7b0 con 0x7f93d8102f90 2024-09-16T08:57:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.892+0000 7f93bffff640 1 -- 172.21.15.31:0/2661988696 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f93cc00f960 con 0x7f93d8102f90 2024-09-16T08:57:47.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.892+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c4005280 con 0x7f93d8102f90 2024-09-16T08:57:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.893+0000 7f93bffff640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f93c0073d30 0x7f93c00761f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.893+0000 7f93bffff640 1 -- 172.21.15.31:0/2661988696 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f93cc094c60 con 0x7f93d8102f90 2024-09-16T08:57:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.893+0000 7f93d6ffd640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f93c0073d30 0x7f93c00761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:47.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.894+0000 7f93d6ffd640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f93c0073d30 0x7f93c00761f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006010 tx=0x7f93c80314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:47.898 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:47.898+0000 7f93bffff640 1 -- 172.21.15.31:0/2661988696 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f93cc05cbc0 con 0x7f93d8102f90 2024-09-16T08:57:48.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:47 smithi031 ceph-mon[27844]: pgmap v25: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 215 op/s 2024-09-16T08:57:48.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:47 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:48.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:47 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:48.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:47 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:57:48.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:47 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:48.053 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T08:57:48.054 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T08:57:48.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.190+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93c4000bf0 con 0x7f93c0073d30 2024-09-16T08:57:48.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.191+0000 7f93bffff640 1 -- 172.21.15.31:0/2661988696 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f93c4000bf0 con 0x7f93c0073d30 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.193+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f93c0073d30 msgr2=0x7f93c00761f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.193+0000 7f93dd346640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f93c0073d30 0x7f93c00761f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f93c8006010 tx=0x7f93c80314e0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 msgr2=0x7f93d8197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 0x7f93d8197d20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00d8d0 tx=0x7f93cc00dd90 comp rx=0 tx=0).stop 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 shutdown_connections 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f93c0073d30 0x7f93c00761f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f93d8102f90 0x7f93d8197d20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 --2- 172.21.15.31:0/2661988696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93d8101d90 0x7f93d81977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 >> 172.21.15.31:0/2661988696 conn(0x7f93d80fd540 msgr2=0x7f93d80ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 shutdown_connections 2024-09-16T08:57:48.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.194+0000 7f93dd346640 1 -- 172.21.15.31:0/2661988696 wait complete. 2024-09-16T08:57:48.204 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:57:48.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:47 smithi097 ceph-mon[35478]: pgmap v25: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 215 op/s 2024-09-16T08:57:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:47 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:47 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:47 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:57:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:47 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 -- 172.21.15.31:0/1775227904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 msgr2=0x7fe568101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/1775227904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568101200 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009880 tx=0x7fe55c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 -- 172.21.15.31:0/1775227904 shutdown_connections 2024-09-16T08:57:48.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/1775227904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568101200 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/1775227904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5680ffb80 0x7fe5680fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 -- 172.21.15.31:0/1775227904 >> 172.21.15.31:0/1775227904 conn(0x7fe5680fb310 msgr2=0x7fe5680fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:48.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 -- 172.21.15.31:0/1775227904 shutdown_connections 2024-09-16T08:57:48.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.288+0000 7fe56dcb5640 1 -- 172.21.15.31:0/1775227904 wait complete. 2024-09-16T08:57:48.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.289+0000 7fe56dcb5640 1 Processor -- start 2024-09-16T08:57:48.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.289+0000 7fe56dcb5640 1 -- start start 2024-09-16T08:57:48.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.289+0000 7fe56dcb5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5680ffb80 0x7fe5681977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:48.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.289+0000 7fe56dcb5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:48.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.289+0000 7fe56dcb5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5681982f0 con 0x7fe568100d80 2024-09-16T08:57:48.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.289+0000 7fe56dcb5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe568198460 con 0x7fe5680ffb80 2024-09-16T08:57:48.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe566ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:48.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe566ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59868/0 (socket says 172.21.15.31:59868) 2024-09-16T08:57:48.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe566ffd640 1 -- 172.21.15.31:0/3898714245 learned_addr learned my addr 172.21.15.31:0/3898714245 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:48.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe5677fe640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5680ffb80 0x7fe5681977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:48.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe566ffd640 1 -- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5680ffb80 msgr2=0x7fe5681977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe566ffd640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5680ffb80 0x7fe5681977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe566ffd640 1 -- 172.21.15.31:0/3898714245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c009560 con 0x7fe568100d80 2024-09-16T08:57:48.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe566ffd640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568197d20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005520 tx=0x7fe55c005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:48.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe564ff9640 1 -- 172.21.15.31:0/3898714245 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55c004aa0 con 0x7fe568100d80 2024-09-16T08:57:48.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56819cea0 con 0x7fe568100d80 2024-09-16T08:57:48.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe564ff9640 1 -- 172.21.15.31:0/3898714245 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe55c004c00 con 0x7fe568100d80 2024-09-16T08:57:48.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.290+0000 7fe564ff9640 1 -- 172.21.15.31:0/3898714245 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe55c003b10 con 0x7fe568100d80 2024-09-16T08:57:48.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.291+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56819d3e0 con 0x7fe568100d80 2024-09-16T08:57:48.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.292+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe568104bd0 con 0x7fe568100d80 2024-09-16T08:57:48.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.292+0000 7fe564ff9640 1 -- 172.21.15.31:0/3898714245 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7fe55c003c70 con 0x7fe568100d80 2024-09-16T08:57:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.293+0000 7fe564ff9640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fe550073d40 0x7fe550076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.293+0000 7fe564ff9640 1 -- 172.21.15.31:0/3898714245 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fe55c0b6100 con 0x7fe568100d80 2024-09-16T08:57:48.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.293+0000 7fe5677fe640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fe550073d40 0x7fe550076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:48.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.294+0000 7fe5677fe640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fe550073d40 0x7fe550076200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe568100be0 tx=0x7fe55800b390 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:48.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.299+0000 7fe564ff9640 1 -- 172.21.15.31:0/3898714245 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7fe55c0797f0 con 0x7fe568100d80 2024-09-16T08:57:48.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.527+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe56802c480 con 0x7fe550073d40 2024-09-16T08:57:48.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.528+0000 7fe564ff9640 1 -- 172.21.15.31:0/3898714245 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fe56802c480 con 0x7fe550073d40 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fe550073d40 msgr2=0x7fe550076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fe550073d40 0x7fe550076200 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe568100be0 tx=0x7fe55800b390 comp rx=0 tx=0).stop 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 msgr2=0x7fe568197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568197d20 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fe55c005520 tx=0x7fe55c005550 comp rx=0 tx=0).stop 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 shutdown_connections 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fe550073d40 0x7fe550076200 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe568100d80 0x7fe568197d20 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.530+0000 7fe56dcb5640 1 --2- 172.21.15.31:0/3898714245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5680ffb80 0x7fe5681977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.531+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 >> 172.21.15.31:0/3898714245 conn(0x7fe5680fb310 msgr2=0x7fe5680fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.531+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 shutdown_connections 2024-09-16T08:57:48.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.531+0000 7fe56dcb5640 1 -- 172.21.15.31:0/3898714245 wait complete. 2024-09-16T08:57:48.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.645+0000 7f1a3fc82640 1 -- 172.21.15.31:0/1819317161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 msgr2=0x7f1a38071830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.645+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/1819317161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 0x7f1a38071830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1a34013e90 tx=0x7f1a34032a30 comp rx=0 tx=0).stop 2024-09-16T08:57:48.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.646+0000 7f1a3fc82640 1 -- 172.21.15.31:0/1819317161 shutdown_connections 2024-09-16T08:57:48.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.646+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/1819317161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 0x7f1a381074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.646+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/1819317161 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 0x7f1a38071830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.646+0000 7f1a3fc82640 1 -- 172.21.15.31:0/1819317161 >> 172.21.15.31:0/1819317161 conn(0x7f1a3806cc10 msgr2=0x7f1a3806f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:48.647 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.647+0000 7f1a3fc82640 1 -- 172.21.15.31:0/1819317161 shutdown_connections 2024-09-16T08:57:48.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.647+0000 7f1a3fc82640 1 -- 172.21.15.31:0/1819317161 wait complete. 2024-09-16T08:57:48.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.647+0000 7f1a3fc82640 1 Processor -- start 2024-09-16T08:57:48.649 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3fc82640 1 -- start start 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3fc82640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 0x7f1a3819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3fc82640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 0x7f1a3819c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3fc82640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3819c6b0 con 0x7f1a38071d70 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3fc82640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3819c820 con 0x7f1a38071430 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3d21d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 0x7f1a3819c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3da1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 0x7f1a3819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3d21d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 0x7f1a3819c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59876/0 (socket says 172.21.15.31:59876) 2024-09-16T08:57:48.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.648+0000 7f1a3d21d640 1 -- 172.21.15.31:0/2841380472 learned_addr learned my addr 172.21.15.31:0/2841380472 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:48.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.649+0000 7f1a3d21d640 1 -- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 msgr2=0x7f1a3819bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.649+0000 7f1a3d21d640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 0x7f1a3819bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.649+0000 7f1a3d21d640 1 -- 172.21.15.31:0/2841380472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34013b70 con 0x7f1a38071d70 2024-09-16T08:57:48.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.649+0000 7f1a3da1e640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 0x7f1a3819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:57:48.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.650+0000 7f1a3d21d640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 0x7f1a3819c0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e970 tx=0x7f1a2800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:48.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.650+0000 7f1a2effd640 1 -- 172.21.15.31:0/2841380472 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a28009700 con 0x7f1a38071d70 2024-09-16T08:57:48.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.650+0000 7f1a2effd640 1 -- 172.21.15.31:0/2841380472 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a280054e0 con 0x7f1a38071d70 2024-09-16T08:57:48.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.650+0000 7f1a2effd640 1 -- 172.21.15.31:0/2841380472 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a28009700 con 0x7f1a38071d70 2024-09-16T08:57:48.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.650+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a381a12c0 con 0x7f1a38071d70 2024-09-16T08:57:48.652 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.650+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a381a17c0 con 0x7f1a38071d70 2024-09-16T08:57:48.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.652+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a38195d30 con 0x7f1a38071d70 2024-09-16T08:57:48.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.653+0000 7f1a2effd640 1 -- 172.21.15.31:0/2841380472 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f1a2800cc90 con 0x7f1a38071d70 2024-09-16T08:57:48.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.654+0000 7f1a2effd640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1a24073e30 0x7f1a240762f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:48.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.654+0000 7f1a3da1e640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1a24073e30 0x7f1a240762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:48.655 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.654+0000 7f1a2effd640 1 -- 172.21.15.31:0/2841380472 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1a28014070 con 0x7f1a38071d70 2024-09-16T08:57:48.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.655+0000 7f1a3da1e640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1a24073e30 0x7f1a240762f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a34004dd0 tx=0x7f1a34000bc0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:48.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.664+0000 7f1a2effd640 1 -- 172.21.15.31:0/2841380472 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f1a280588a0 con 0x7f1a38071d70 2024-09-16T08:57:48.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:48 smithi031 ceph-mon[27844]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:57:48.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:48 smithi031 ceph-mon[27844]: Upgrade: Need to upgrade myself (mgr.smithi097.mpashf) 2024-09-16T08:57:48.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:48 smithi031 ceph-mon[27844]: Upgrade: Need to upgrade myself (mgr.smithi097.mpashf) 2024-09-16T08:57:48.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.964+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a3802c4f0 con 0x7f1a24073e30 2024-09-16T08:57:48.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.974+0000 7f1a2effd640 1 -- 172.21.15.31:0/2841380472 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1a3802c4f0 con 0x7f1a24073e30 2024-09-16T08:57:48.975 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:57:48.975 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (7m) 23s ago 9m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 starting - - - - 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 starting - - - - 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (9m) 23s ago 9m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (8m) 35s ago 8m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 23s ago 9m 84.2M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (3m) 23s ago 3m 13.4M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (3m) 23s ago 3m 187M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:57:48.976 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (3m) 35s ago 3m 22.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (2m) 35s ago 2m 15.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:9283,8765,8443 running (10m) 23s ago 10m 299M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 1c78052ec7c9 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (66s) 35s ago 8m 496M - 19.3.0-4917-g8293d73f 425e316ba8eb 6aa6df83241b 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (10m) 23s ago 10m 53.4M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (8m) 35s ago 8m 40.5M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (9m) 23s ago 9m 15.5M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (8m) 35s ago 8m 16.3M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:57:48.977 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 23s ago 6m 186M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:57:48.978 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 23s ago 6m 207M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:57:48.978 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 23s ago 5m 156M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:57:48.978 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (5m) 35s ago 5m 183M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:57:48.978 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (5m) 35s ago 5m 206M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:57:48.978 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (4m) 35s ago 4m 205M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:57:48.978 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 starting - - - - 2024-09-16T08:57:48.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1a24073e30 msgr2=0x7f1a240762f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1a24073e30 0x7f1a240762f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1a34004dd0 tx=0x7f1a34000bc0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 msgr2=0x7f1a3819c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:48.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 0x7f1a3819c0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800e970 tx=0x7f1a2800ee30 comp rx=0 tx=0).stop 2024-09-16T08:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 shutdown_connections 2024-09-16T08:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1a24073e30 0x7f1a240762f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a38071d70 0x7f1a3819c0e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 --2- 172.21.15.31:0/2841380472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1a38071430 0x7f1a3819bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.977+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 >> 172.21.15.31:0/2841380472 conn(0x7f1a3806cc10 msgr2=0x7f1a3806eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.978+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 shutdown_connections 2024-09-16T08:57:48.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:48.978+0000 7f1a3fc82640 1 -- 172.21.15.31:0/2841380472 wait complete. 2024-09-16T08:57:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 -- 172.21.15.31:0/801885780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 msgr2=0x7f31280feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 --2- 172.21.15.31:0/801885780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 0x7f31280feca0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f311c009880 tx=0x7f311c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 -- 172.21.15.31:0/801885780 shutdown_connections 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 --2- 172.21.15.31:0/801885780 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 0x7f31280fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 --2- 172.21.15.31:0/801885780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 0x7f31280feca0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 -- 172.21.15.31:0/801885780 >> 172.21.15.31:0/801885780 conn(0x7f31280fa010 msgr2=0x7f31280fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 -- 172.21.15.31:0/801885780 shutdown_connections 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.090+0000 7f312d9a1640 1 -- 172.21.15.31:0/801885780 wait complete. 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f312d9a1640 1 Processor -- start 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f312d9a1640 1 -- start start 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f312d9a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 0x7f3128195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f312d9a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 0x7f31281959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f312d9a1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3128195f80 con 0x7f31280fe8a0 2024-09-16T08:57:49.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f312d9a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31281960f0 con 0x7f31280ffaa0 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f3127fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 0x7f31281959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f312c99f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 0x7f3128195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f3127fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 0x7f31281959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:44566/0 (socket says 172.21.15.31:44566) 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.091+0000 7f3127fff640 1 -- 172.21.15.31:0/3017021707 learned_addr learned my addr 172.21.15.31:0/3017021707 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.092+0000 7f3127fff640 1 -- 172.21.15.31:0/3017021707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 msgr2=0x7f3128195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.092+0000 7f3127fff640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 0x7f3128195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.092+0000 7f3127fff640 1 -- 172.21.15.31:0/3017021707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c009560 con 0x7f31280ffaa0 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.092+0000 7f3127fff640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 0x7f31281959b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f311400e970 tx=0x7f311400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:49.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.092+0000 7f3125ffb640 1 -- 172.21.15.31:0/3017021707 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f311400cc90 con 0x7f31280ffaa0 2024-09-16T08:57:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.092+0000 7f3125ffb640 1 -- 172.21.15.31:0/3017021707 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f311400cdf0 con 0x7f31280ffaa0 2024-09-16T08:57:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.092+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312819ab90 con 0x7f31280ffaa0 2024-09-16T08:57:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.093+0000 7f3125ffb640 1 -- 172.21.15.31:0/3017021707 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3114010430 con 0x7f31280ffaa0 2024-09-16T08:57:49.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.093+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312819b160 con 0x7f31280ffaa0 2024-09-16T08:57:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.094+0000 7f3125ffb640 1 -- 172.21.15.31:0/3017021707 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f3114005020 con 0x7f31280ffaa0 2024-09-16T08:57:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.094+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31281074b0 con 0x7f31280ffaa0 2024-09-16T08:57:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.094+0000 7f3125ffb640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3118073d40 0x7f3118076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.095+0000 7f3125ffb640 1 -- 172.21.15.31:0/3017021707 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3114014070 con 0x7f31280ffaa0 2024-09-16T08:57:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.095+0000 7f312c99f640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3118073d40 0x7f3118076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.095+0000 7f312c99f640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3118073d40 0x7f3118076200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f311c009230 tx=0x7f311c0314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:49.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.099+0000 7f3125ffb640 1 -- 172.21.15.31:0/3017021707 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f3114058680 con 0x7f31280ffaa0 2024-09-16T08:57:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:48 smithi097 ceph-mon[35478]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:57:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:48 smithi097 ceph-mon[35478]: Upgrade: Need to upgrade myself (mgr.smithi097.mpashf) 2024-09-16T08:57:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:48 smithi097 ceph-mon[35478]: Upgrade: Need to upgrade myself (mgr.smithi097.mpashf) 2024-09-16T08:57:49.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.435+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f31281076a0 con 0x7f31280ffaa0 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.436+0000 7f3125ffb640 1 -- 172.21.15.31:0/3017021707 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+836 (secure 0 0 0) 0x7f311405b7a0 con 0x7f31280ffaa0 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 1, 2024-09-16T08:57:49.437 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 13, 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:57:49.438 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:57:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3118073d40 msgr2=0x7f3118076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3118073d40 0x7f3118076200 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f311c009230 tx=0x7f311c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 msgr2=0x7f31281959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 0x7f31281959b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f311400e970 tx=0x7f311400ed70 comp rx=0 tx=0).stop 2024-09-16T08:57:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 shutdown_connections 2024-09-16T08:57:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3118073d40 0x7f3118076200 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f31280ffaa0 0x7f31281959b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 --2- 172.21.15.31:0/3017021707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31280fe8a0 0x7f3128195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 >> 172.21.15.31:0/3017021707 conn(0x7f31280fa010 msgr2=0x7f31280fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 shutdown_connections 2024-09-16T08:57:49.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.439+0000 7f312d9a1640 1 -- 172.21.15.31:0/3017021707 wait complete. 2024-09-16T08:57:49.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 -- 172.21.15.31:0/468654172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958100d90 msgr2=0x7fc958101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 --2- 172.21.15.31:0/468654172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958100d90 0x7fc958101210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009880 tx=0x7fc94c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 -- 172.21.15.31:0/468654172 shutdown_connections 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 --2- 172.21.15.31:0/468654172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958100d90 0x7fc958101210 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 --2- 172.21.15.31:0/468654172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9580ffb90 0x7fc9580fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 -- 172.21.15.31:0/468654172 >> 172.21.15.31:0/468654172 conn(0x7fc9580fb360 msgr2=0x7fc9580fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 -- 172.21.15.31:0/468654172 shutdown_connections 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.536+0000 7fc95db94640 1 -- 172.21.15.31:0/468654172 wait complete. 2024-09-16T08:57:49.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.537+0000 7fc95db94640 1 Processor -- start 2024-09-16T08:57:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.537+0000 7fc95db94640 1 -- start start 2024-09-16T08:57:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.537+0000 7fc95db94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9580ffb90 0x7fc958195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.537+0000 7fc95db94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100d90 0x7fc958195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.537+0000 7fc95db94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958196120 con 0x7fc9580ffb90 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.537+0000 7fc95db94640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958196290 con 0x7fc958100d90 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc9577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9580ffb90 0x7fc958195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc9577fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9580ffb90 0x7fc958195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59922/0 (socket says 172.21.15.31:59922) 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc9577fe640 1 -- 172.21.15.31:0/1785973300 learned_addr learned my addr 172.21.15.31:0/1785973300 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc956ffd640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100d90 0x7fc958195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc9577fe640 1 -- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100d90 msgr2=0x7fc958195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc9577fe640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100d90 0x7fc958195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc9577fe640 1 -- 172.21.15.31:0/1785973300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c009560 con 0x7fc9580ffb90 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc9577fe640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9580ffb90 0x7fc958195610 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc94800d720 tx=0x7fc94800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc954ff9640 1 -- 172.21.15.31:0/1785973300 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9480041f0 con 0x7fc9580ffb90 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95819ad30 con 0x7fc9580ffb90 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95819b270 con 0x7fc9580ffb90 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.538+0000 7fc954ff9640 1 -- 172.21.15.31:0/1785973300 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc948004350 con 0x7fc9580ffb90 2024-09-16T08:57:49.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.539+0000 7fc954ff9640 1 -- 172.21.15.31:0/1785973300 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc948003df0 con 0x7fc9580ffb90 2024-09-16T08:57:49.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.540+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9580fff90 con 0x7fc9580ffb90 2024-09-16T08:57:49.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.540+0000 7fc954ff9640 1 -- 172.21.15.31:0/1785973300 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7fc948009820 con 0x7fc9580ffb90 2024-09-16T08:57:49.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.541+0000 7fc954ff9640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc940073d40 0x7fc940076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.541+0000 7fc954ff9640 1 -- 172.21.15.31:0/1785973300 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc948094980 con 0x7fc9580ffb90 2024-09-16T08:57:49.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.541+0000 7fc956ffd640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc940073d40 0x7fc940076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.543 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.542+0000 7fc956ffd640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc940073d40 0x7fc940076200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc94c000c00 tx=0x7fc94c0314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:49.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.546+0000 7fc954ff9640 1 -- 172.21.15.31:0/1785973300 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7fc948058070 con 0x7fc9580ffb90 2024-09-16T08:57:49.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.887+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc958108870 con 0x7fc9580ffb90 2024-09-16T08:57:49.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.888+0000 7fc954ff9640 1 -- 172.21.15.31:0/1785973300 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7fc948020750 con 0x7fc9580ffb90 2024-09-16T08:57:49.889 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:57:49.889 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:57:49.890 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:57:49.891 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:57:49.892 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:49.893 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:57:49.893 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:57:49.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.891+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc940073d40 msgr2=0x7fc940076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.891+0000 7fc95db94640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc940073d40 0x7fc940076200 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc94c000c00 tx=0x7fc94c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.891+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9580ffb90 msgr2=0x7fc958195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.891+0000 7fc95db94640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9580ffb90 0x7fc958195610 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc94800d720 tx=0x7fc94800dbe0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.891+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 shutdown_connections 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.891+0000 7fc95db94640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc940073d40 0x7fc940076200 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.892+0000 7fc95db94640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100d90 0x7fc958195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.892+0000 7fc95db94640 1 --2- 172.21.15.31:0/1785973300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9580ffb90 0x7fc958195610 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.892+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 >> 172.21.15.31:0/1785973300 conn(0x7fc9580fb360 msgr2=0x7fc958104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.892+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 shutdown_connections 2024-09-16T08:57:49.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.892+0000 7fc95db94640 1 -- 172.21.15.31:0/1785973300 wait complete. 2024-09-16T08:57:49.964 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='client.24507 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:49.964 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:49.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: pgmap v26: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 215 op/s 2024-09-16T08:57:49.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: Upgrade: Updating mgr.smithi031.dcmqpv 2024-09-16T08:57:49.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:49.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:57:49.965 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:57:49.966 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:57:49.966 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:49.966 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: Deploying daemon mgr.smithi031.dcmqpv on smithi031 2024-09-16T08:57:49.966 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:49.966 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:49.966 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:49 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3017021707' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:57:49.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 -- 172.21.15.31:0/2174902651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 msgr2=0x7fc4bc100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 --2- 172.21.15.31:0/2174902651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc100170 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009880 tx=0x7fc4ac02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 -- 172.21.15.31:0/2174902651 shutdown_connections 2024-09-16T08:57:49.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 --2- 172.21.15.31:0/2174902651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc100170 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 --2- 172.21.15.31:0/2174902651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4bc0feaf0 0x7fc4bc0feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 -- 172.21.15.31:0/2174902651 >> 172.21.15.31:0/2174902651 conn(0x7fc4bc0fa280 msgr2=0x7fc4bc0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 -- 172.21.15.31:0/2174902651 shutdown_connections 2024-09-16T08:57:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.989+0000 7fc4c351e640 1 -- 172.21.15.31:0/2174902651 wait complete. 2024-09-16T08:57:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c351e640 1 Processor -- start 2024-09-16T08:57:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c351e640 1 -- start start 2024-09-16T08:57:49.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c351e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4bc0feaf0 0x7fc4bc0687d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c351e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc19a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c351e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc19a960 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c351e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4bc19aad0 con 0x7fc4bc0feaf0 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c0ab9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc19a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c0ab9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc19a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59942/0 (socket says 172.21.15.31:59942) 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.990+0000 7fc4c0ab9640 1 -- 172.21.15.31:0/3807944635 learned_addr learned my addr 172.21.15.31:0/3807944635 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c12ba640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4bc0feaf0 0x7fc4bc0687d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c0ab9640 1 -- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4bc0feaf0 msgr2=0x7fc4bc0687d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c0ab9640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4bc0feaf0 0x7fc4bc0687d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c0ab9640 1 -- 172.21.15.31:0/3807944635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac009560 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c12ba640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4bc0feaf0 0x7fc4bc0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c0ab9640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc19a390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0054f0 tx=0x7fc4ac005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4b27fc640 1 -- 172.21.15.31:0/3807944635 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4ac004aa0 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4b27fc640 1 -- 172.21.15.31:0/3807944635 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4ac004c00 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4bc19ad50 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4b27fc640 1 -- 172.21.15.31:0/3807944635 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4ac003ac0 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.996 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.991+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4bc19b240 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.993+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4bc18e670 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.994+0000 7fc4b27fc640 1 -- 172.21.15.31:0/3807944635 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7fc4ac031c70 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.995+0000 7fc4b27fc640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc4a8073d40 0x7fc4a8076200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:49.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.995+0000 7fc4b27fc640 1 -- 172.21.15.31:0/3807944635 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fc4ac0b6340 con 0x7fc4bc0ffcf0 2024-09-16T08:57:49.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.995+0000 7fc4c12ba640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc4a8073d40 0x7fc4a8076200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:49.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:49.996+0000 7fc4c12ba640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc4a8073d40 0x7fc4a8076200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc068d10 tx=0x7fc4b800b410 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:50.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.000+0000 7fc4b27fc640 1 -- 172.21.15.31:0/3807944635 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7fc4ac079910 con 0x7fc4bc0ffcf0 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='client.24507 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: pgmap v26: 65 pgs: 65 active+clean; 1.1 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 215 op/s 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: Upgrade: Updating mgr.smithi031.dcmqpv 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:57:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:57:50.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:57:50.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: Deploying daemon mgr.smithi031.dcmqpv on smithi031 2024-09-16T08:57:50.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:50.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:57:50.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:49 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3017021707' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:57:50.276 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.275+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4bc02c450 con 0x7fc4a8073d40 2024-09-16T08:57:50.277 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.276+0000 7fc4b27fc640 1 -- 172.21.15.31:0/3807944635 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7fc4bc02c450 con 0x7fc4a8073d40 2024-09-16T08:57:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:57:50.277 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:57:50.277 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:57:50.277 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:57:50.277 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-09-16T08:57:50.278 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "1/23 daemons upgraded", 2024-09-16T08:57:50.278 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T08:57:50.278 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:57:50.278 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:57:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc4a8073d40 msgr2=0x7fc4a8076200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc4a8073d40 0x7fc4a8076200 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc068d10 tx=0x7fc4b800b410 comp rx=0 tx=0).stop 2024-09-16T08:57:50.280 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 msgr2=0x7fc4bc19a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc19a390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0054f0 tx=0x7fc4ac005520 comp rx=0 tx=0).stop 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 shutdown_connections 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fc4a8073d40 0x7fc4a8076200 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4bc0ffcf0 0x7fc4bc19a390 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 --2- 172.21.15.31:0/3807944635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc4bc0feaf0 0x7fc4bc0687d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.279+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 >> 172.21.15.31:0/3807944635 conn(0x7fc4bc0fa280 msgr2=0x7fc4bc1033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.280+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 shutdown_connections 2024-09-16T08:57:50.281 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.280+0000 7fc4c351e640 1 -- 172.21.15.31:0/3807944635 wait complete. 2024-09-16T08:57:50.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 -- 172.21.15.31:0/2240414365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 msgr2=0x7f019c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:50.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 --2- 172.21.15.31:0/2240414365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c102a70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f018c0098b0 tx=0x7f018c02a4d0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 -- 172.21.15.31:0/2240414365 shutdown_connections 2024-09-16T08:57:50.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 --2- 172.21.15.31:0/2240414365 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102fb0 0x7f019c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 --2- 172.21.15.31:0/2240414365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c102a70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 -- 172.21.15.31:0/2240414365 >> 172.21.15.31:0/2240414365 conn(0x7f019c0fa280 msgr2=0x7f019c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:50.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 -- 172.21.15.31:0/2240414365 shutdown_connections 2024-09-16T08:57:50.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 -- 172.21.15.31:0/2240414365 wait complete. 2024-09-16T08:57:50.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 Processor -- start 2024-09-16T08:57:50.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.378+0000 7f01a24ac640 1 -- start start 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f01a24ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f01a24ac640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102fb0 0x7f019c0fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f01a24ac640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0feb70 con 0x7f019c100680 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f01a24ac640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0fece0 con 0x7f019c102fb0 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f019bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f019bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c0fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:59954/0 (socket says 172.21.15.31:59954) 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f019bfff640 1 -- 172.21.15.31:0/3446700808 learned_addr learned my addr 172.21.15.31:0/3446700808 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f019b7fe640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102fb0 0x7f019c0fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f019bfff640 1 -- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102fb0 msgr2=0x7f019c0fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f019bfff640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102fb0 0x7f019c0fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.379+0000 7f019bfff640 1 -- 172.21.15.31:0/3446700808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c009560 con 0x7f019c100680 2024-09-16T08:57:50.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.380+0000 7f019bfff640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c0fffe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a9f0 tx=0x7f018c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:50.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.380+0000 7f01997fa640 1 -- 172.21.15.31:0/3446700808 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f018c004aa0 con 0x7f019c100680 2024-09-16T08:57:50.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.380+0000 7f01997fa640 1 -- 172.21.15.31:0/3446700808 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f018c005d50 con 0x7f019c100680 2024-09-16T08:57:50.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.380+0000 7f01997fa640 1 -- 172.21.15.31:0/3446700808 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f018c0039b0 con 0x7f019c100680 2024-09-16T08:57:50.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.380+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c0fef60 con 0x7f019c100680 2024-09-16T08:57:50.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.380+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c0ff450 con 0x7f019c100680 2024-09-16T08:57:50.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.381+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019c108810 con 0x7f019c100680 2024-09-16T08:57:50.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.382+0000 7f01997fa640 1 -- 172.21.15.31:0/3446700808 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 25) v1 ==== 97037+0+0 (secure 0 0 0) 0x7f018c031c00 con 0x7f019c100680 2024-09-16T08:57:50.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.383+0000 7f01997fa640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f0184073d30 0x7f01840761f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:57:50.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.383+0000 7f01997fa640 1 -- 172.21.15.31:0/3446700808 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f018c0b6370 con 0x7f019c100680 2024-09-16T08:57:50.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.383+0000 7f019b7fe640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f0184073d30 0x7f01840761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:57:50.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.384+0000 7f019b7fe640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f0184073d30 0x7f01840761f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f019c0ffd60 tx=0x7f0190005f90 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:57:50.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.387+0000 7f01997fa640 1 -- 172.21.15.31:0/3446700808 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f018c079b00 con 0x7f019c100680 2024-09-16T08:57:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.788+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f019c1089d0 con 0x7f019c100680 2024-09-16T08:57:50.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.789+0000 7f01997fa640 1 -- 172.21.15.31:0/3446700808 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f018c07cc20 con 0x7f019c100680 2024-09-16T08:57:50.791 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:57:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f0184073d30 msgr2=0x7f01840761f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f0184073d30 0x7f01840761f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f019c0ffd60 tx=0x7f0190005f90 comp rx=0 tx=0).stop 2024-09-16T08:57:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 msgr2=0x7f019c0fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:57:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c0fffe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a9f0 tx=0x7f018c0055a0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 shutdown_connections 2024-09-16T08:57:50.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f0184073d30 0x7f01840761f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f019c102fb0 0x7f019c0fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 --2- 172.21.15.31:0/3446700808 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f019c100680 0x7f019c0fffe0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:57:50.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 >> 172.21.15.31:0/3446700808 conn(0x7f019c0fa280 msgr2=0x7f019c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:57:50.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 shutdown_connections 2024-09-16T08:57:50.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:57:50.794+0000 7f01a24ac640 1 -- 172.21.15.31:0/3446700808 wait complete. 2024-09-16T08:57:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:50 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/1785973300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:57:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:50 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3446700808' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:57:51.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:50 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/1785973300' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:57:51.018 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:50 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3446700808' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:57:52.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:51 smithi031 ceph-mon[27844]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:52.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:51 smithi031 ceph-mon[27844]: pgmap v27: 65 pgs: 65 active+clean; 398 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 357 op/s 2024-09-16T08:57:52.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:51 smithi097 ceph-mon[35478]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:57:52.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:51 smithi097 ceph-mon[35478]: pgmap v27: 65 pgs: 65 active+clean; 398 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 357 op/s 2024-09-16T08:57:54.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:53 smithi097 ceph-mon[35478]: pgmap v28: 65 pgs: 65 active+clean; 398 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.6 MiB/s wr, 265 op/s 2024-09-16T08:57:54.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:53 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:54.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:53 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:54.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:53 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:54.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:53 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:54.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:53 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:53 smithi031 ceph-mon[27844]: pgmap v28: 65 pgs: 65 active+clean; 398 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.6 MiB/s wr, 265 op/s 2024-09-16T08:57:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:53 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:53 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:53 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:57:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:53 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:53 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:56.187 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:55 smithi097 ceph-mon[35478]: pgmap v29: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.4 MiB/s wr, 447 op/s 2024-09-16T08:57:56.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:55 smithi031 ceph-mon[27844]: pgmap v29: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.4 MiB/s wr, 447 op/s 2024-09-16T08:57:58.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:57 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:58.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:57 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:58.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:57 smithi097 ceph-mon[35478]: pgmap v30: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 324 op/s 2024-09-16T08:57:58.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:57 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:58.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:57 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:57:58.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:57 smithi031 ceph-mon[27844]: pgmap v30: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 324 op/s 2024-09-16T08:58:00.045 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:57:59 smithi031 ceph-mon[27844]: pgmap v31: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 324 op/s 2024-09-16T08:58:00.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:57:59 smithi097 ceph-mon[35478]: pgmap v31: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 1.6 MiB/s wr, 324 op/s 2024-09-16T08:58:01.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:00 smithi097 ceph-mon[35478]: Standby manager daemon smithi031.dcmqpv restarted 2024-09-16T08:58:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:00 smithi097 ceph-mon[35478]: Standby manager daemon smithi031.dcmqpv started 2024-09-16T08:58:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:00 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/crt"}]: dispatch 2024-09-16T08:58:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:00 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:58:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:00 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/key"}]: dispatch 2024-09-16T08:58:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:00 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:58:01.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:00 smithi031 ceph-mon[27844]: Standby manager daemon smithi031.dcmqpv restarted 2024-09-16T08:58:01.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:00 smithi031 ceph-mon[27844]: Standby manager daemon smithi031.dcmqpv started 2024-09-16T08:58:01.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:00 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/crt"}]: dispatch 2024-09-16T08:58:01.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:00 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:58:01.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:00 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi031.dcmqpv/key"}]: dispatch 2024-09-16T08:58:01.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:00 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:58:02.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:01 smithi031 ceph-mon[27844]: pgmap v32: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 2.3 MiB/s wr, 439 op/s 2024-09-16T08:58:02.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:01 smithi031 ceph-mon[27844]: mgrmap e26: smithi097.mpashf(active, since 58s), standbys: smithi031.dcmqpv 2024-09-16T08:58:02.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:01 smithi097 ceph-mon[35478]: pgmap v32: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 51 KiB/s rd, 2.3 MiB/s wr, 439 op/s 2024-09-16T08:58:02.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:01 smithi097 ceph-mon[35478]: mgrmap e26: smithi097.mpashf(active, since 58s), standbys: smithi031.dcmqpv 2024-09-16T08:58:04.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:03 smithi031 ceph-mon[27844]: pgmap v33: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.5 MiB/s wr, 297 op/s 2024-09-16T08:58:04.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:03 smithi097 ceph-mon[35478]: pgmap v33: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.5 MiB/s wr, 297 op/s 2024-09-16T08:58:05.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:04 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:04 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:05 smithi031 ceph-mon[27844]: pgmap v34: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.2 MiB/s wr, 469 op/s 2024-09-16T08:58:06.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:05 smithi097 ceph-mon[35478]: pgmap v34: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 2.2 MiB/s wr, 469 op/s 2024-09-16T08:58:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:07 smithi031 ceph-mon[27844]: pgmap v35: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 286 op/s 2024-09-16T08:58:08.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:07 smithi097 ceph-mon[35478]: pgmap v35: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 286 op/s 2024-09-16T08:58:10.064 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:09 smithi097 ceph-mon[35478]: pgmap v36: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 286 op/s 2024-09-16T08:58:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:09 smithi031 ceph-mon[27844]: pgmap v36: 65 pgs: 65 active+clean; 301 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.3 MiB/s wr, 286 op/s 2024-09-16T08:58:12.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:11 smithi031 ceph-mon[27844]: pgmap v37: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 2.5 MiB/s wr, 360 op/s 2024-09-16T08:58:12.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:11 smithi097 ceph-mon[35478]: pgmap v37: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 35 KiB/s rd, 2.5 MiB/s wr, 360 op/s 2024-09-16T08:58:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:13 smithi031 ceph-mon[27844]: pgmap v38: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-09-16T08:58:14.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:13 smithi097 ceph-mon[35478]: pgmap v38: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-09-16T08:58:16.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:15 smithi031 ceph-mon[27844]: pgmap v39: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 2.8 MiB/s wr, 368 op/s 2024-09-16T08:58:16.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:15 smithi097 ceph-mon[35478]: pgmap v39: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 2.8 MiB/s wr, 368 op/s 2024-09-16T08:58:17.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:17 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:17.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:17 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:17.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:17 smithi031 ceph-mon[27844]: pgmap v40: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-16T08:58:17.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:17 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:17.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:17 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:18.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:17 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:17 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:17 smithi097 ceph-mon[35478]: pgmap v40: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-16T08:58:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:17 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:17 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:20.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:19 smithi031 ceph-mon[27844]: pgmap v41: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-16T08:58:20.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:19 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:20.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:19 smithi097 ceph-mon[35478]: pgmap v41: 65 pgs: 65 active+clean; 318 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 2.1 MiB/s wr, 197 op/s 2024-09-16T08:58:20.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:19 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:20.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/3583622489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18103030 msgr2=0x7f7a18105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:20.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/3583622489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18103030 0x7f7a18105420 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009880 tx=0x7f7a0c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:58:20.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/3583622489 shutdown_connections 2024-09-16T08:58:20.909 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/3583622489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18103030 0x7f7a18105420 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:20.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/3583622489 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18100700 0x7f7a18102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:20.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/3583622489 >> 172.21.15.31:0/3583622489 conn(0x7f7a180fa280 msgr2=0x7f7a180fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:20.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/3583622489 shutdown_connections 2024-09-16T08:58:20.910 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.907+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/3583622489 wait complete. 2024-09-16T08:58:20.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a1d0bd640 1 Processor -- start 2024-09-16T08:58:20.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a1d0bd640 1 -- start start 2024-09-16T08:58:20.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a1d0bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18100700 0x7f7a18100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:20.911 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a1d0bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18103030 0x7f7a180fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a1d0bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180febf0 con 0x7f7a18100700 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a1d0bd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a180fed60 con 0x7f7a18103030 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a16d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18100700 0x7f7a18100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a16d9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18100700 0x7f7a18100060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55742/0 (socket says 172.21.15.31:55742) 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a16d9d640 1 -- 172.21.15.31:0/1868651575 learned_addr learned my addr 172.21.15.31:0/1868651575 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.908+0000 7f7a1659c640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18103030 0x7f7a180fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f7a16d9d640 1 -- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18103030 msgr2=0x7f7a180fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f7a16d9d640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18103030 0x7f7a180fe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f7a16d9d640 1 -- 172.21.15.31:0/1868651575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c009560 con 0x7f7a18100700 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f7a16d9d640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18100700 0x7f7a18100060 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800e970 tx=0x7f7a0800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:20.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f79fffff640 1 -- 172.21.15.31:0/1868651575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a0800cc90 con 0x7f7a18100700 2024-09-16T08:58:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a180ff040 con 0x7f7a18100700 2024-09-16T08:58:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f79fffff640 1 -- 172.21.15.31:0/1868651575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a0800cdf0 con 0x7f7a18100700 2024-09-16T08:58:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f79fffff640 1 -- 172.21.15.31:0/1868651575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a08018810 con 0x7f7a18100700 2024-09-16T08:58:20.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.909+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a1819d710 con 0x7f7a18100700 2024-09-16T08:58:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.910+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a18100ae0 con 0x7f7a18100700 2024-09-16T08:58:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.911+0000 7f79fffff640 1 -- 172.21.15.31:0/1868651575 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f7a08010c10 con 0x7f7a18100700 2024-09-16T08:58:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.911+0000 7f79fffff640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f7a0007ace0 0x7f7a0007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.912+0000 7f79fffff640 1 -- 172.21.15.31:0/1868651575 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7a08014070 con 0x7f7a18100700 2024-09-16T08:58:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.912+0000 7f7a1659c640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f7a0007ace0 0x7f7a0007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:20.914 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.912+0000 7f7a1659c640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f7a0007ace0 0x7f7a0007d1a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009230 tx=0x7f7a0c0314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:20.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:20.919+0000 7f79fffff640 1 -- 172.21.15.31:0/1868651575 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f7a08064db0 con 0x7f7a18100700 2024-09-16T08:58:21.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.189+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a18103a50 con 0x7f7a0007ace0 2024-09-16T08:58:21.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.191+0000 7f79fffff640 1 -- 172.21.15.31:0/1868651575 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f7a18103a50 con 0x7f7a0007ace0 2024-09-16T08:58:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f7a0007ace0 msgr2=0x7f7a0007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f7a0007ace0 0x7f7a0007d1a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009230 tx=0x7f7a0c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18100700 msgr2=0x7f7a18100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18100700 0x7f7a18100060 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7a0800e970 tx=0x7f7a0800ee30 comp rx=0 tx=0).stop 2024-09-16T08:58:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 shutdown_connections 2024-09-16T08:58:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f7a0007ace0 0x7f7a0007d1a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a18103030 0x7f7a180fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 --2- 172.21.15.31:0/1868651575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a18100700 0x7f7a18100060 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 >> 172.21.15.31:0/1868651575 conn(0x7f7a180fa280 msgr2=0x7f7a180fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 shutdown_connections 2024-09-16T08:58:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.193+0000 7f7a1d0bd640 1 -- 172.21.15.31:0/1868651575 wait complete. 2024-09-16T08:58:21.206 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:58:21.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 -- 172.21.15.31:0/287157961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c102fb0 msgr2=0x7f3b5c1053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 --2- 172.21.15.31:0/287157961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c102fb0 0x7f3b5c1053a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3b500098b0 tx=0x7f3b5002a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 -- 172.21.15.31:0/287157961 shutdown_connections 2024-09-16T08:58:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 --2- 172.21.15.31:0/287157961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c102fb0 0x7f3b5c1053a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 --2- 172.21.15.31:0/287157961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5c100680 0x7f3b5c102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 -- 172.21.15.31:0/287157961 >> 172.21.15.31:0/287157961 conn(0x7f3b5c0fa280 msgr2=0x7f3b5c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 -- 172.21.15.31:0/287157961 shutdown_connections 2024-09-16T08:58:21.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.288+0000 7f3b62130640 1 -- 172.21.15.31:0/287157961 wait complete. 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.289+0000 7f3b62130640 1 Processor -- start 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.289+0000 7f3b62130640 1 -- start start 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.289+0000 7f3b62130640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c100680 0x7f3b5c0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.289+0000 7f3b62130640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5c102fb0 0x7f3b5c0fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.289+0000 7f3b62130640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0feb70 con 0x7f3b5c100680 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.289+0000 7f3b62130640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0fece0 con 0x7f3b5c102fb0 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c100680 0x7f3b5c0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c100680 0x7f3b5c0fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:55760/0 (socket says 172.21.15.31:55760) 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5b7fe640 1 -- 172.21.15.31:0/2332640913 learned_addr learned my addr 172.21.15.31:0/2332640913 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5affd640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5c102fb0 0x7f3b5c0fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5b7fe640 1 -- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5c102fb0 msgr2=0x7f3b5c0fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5b7fe640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5c102fb0 0x7f3b5c0fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5b7fe640 1 -- 172.21.15.31:0/2332640913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b50009560 con 0x7f3b5c100680 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b5b7fe640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c100680 0x7f3b5c0fffe0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d6f0 tx=0x7f3b4c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b58ff9640 1 -- 172.21.15.31:0/2332640913 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b4c009a30 con 0x7f3b5c100680 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b58ff9640 1 -- 172.21.15.31:0/2332640913 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b4c0054e0 con 0x7f3b5c100680 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.290+0000 7f3b58ff9640 1 -- 172.21.15.31:0/2332640913 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b4c010460 con 0x7f3b5c100680 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.291+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c0fefc0 con 0x7f3b5c100680 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.291+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c19d780 con 0x7f3b5c100680 2024-09-16T08:58:21.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.292+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b48005280 con 0x7f3b5c100680 2024-09-16T08:58:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.294+0000 7f3b58ff9640 1 -- 172.21.15.31:0/2332640913 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f3b4c0046c0 con 0x7f3b5c100680 2024-09-16T08:58:21.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.295+0000 7f3b58ff9640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3b4407ad30 0x7f3b4407d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.295+0000 7f3b58ff9640 1 -- 172.21.15.31:0/2332640913 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f3b4c09d540 con 0x7f3b5c100680 2024-09-16T08:58:21.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.296+0000 7f3b5affd640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3b4407ad30 0x7f3b4407d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:21.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.297+0000 7f3b5affd640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3b4407ad30 0x7f3b4407d1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009880 tx=0x7f3b500314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:21.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.302+0000 7f3b58ff9640 1 -- 172.21.15.31:0/2332640913 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f3b4c060b90 con 0x7f3b5c100680 2024-09-16T08:58:21.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.622+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b48000bf0 con 0x7f3b4407ad30 2024-09-16T08:58:21.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.623+0000 7f3b58ff9640 1 -- 172.21.15.31:0/2332640913 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f3b48000bf0 con 0x7f3b4407ad30 2024-09-16T08:58:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3b4407ad30 msgr2=0x7f3b4407d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3b4407ad30 0x7f3b4407d1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009880 tx=0x7f3b500314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c100680 msgr2=0x7f3b5c0fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c100680 0x7f3b5c0fffe0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00d6f0 tx=0x7f3b4c00dbb0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 shutdown_connections 2024-09-16T08:58:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f3b4407ad30 0x7f3b4407d1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b5c102fb0 0x7f3b5c0fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 --2- 172.21.15.31:0/2332640913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b5c100680 0x7f3b5c0fffe0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 >> 172.21.15.31:0/2332640913 conn(0x7f3b5c0fa280 msgr2=0x7f3b5c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 shutdown_connections 2024-09-16T08:58:21.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.626+0000 7f3b62130640 1 -- 172.21.15.31:0/2332640913 wait complete. 2024-09-16T08:58:21.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 -- 172.21.15.31:0/766745475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 msgr2=0x7f5648100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 --2- 172.21.15.31:0/766745475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f5648100170 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f563c009c20 tx=0x7f563c02a970 comp rx=0 tx=0).stop 2024-09-16T08:58:21.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 -- 172.21.15.31:0/766745475 shutdown_connections 2024-09-16T08:58:21.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 --2- 172.21.15.31:0/766745475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f5648100170 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 --2- 172.21.15.31:0/766745475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56480feaf0 0x7f56480feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 -- 172.21.15.31:0/766745475 >> 172.21.15.31:0/766745475 conn(0x7f56480fa280 msgr2=0x7f56480fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 -- 172.21.15.31:0/766745475 shutdown_connections 2024-09-16T08:58:21.730 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.727+0000 7f564eeee640 1 -- 172.21.15.31:0/766745475 wait complete. 2024-09-16T08:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f564eeee640 1 Processor -- start 2024-09-16T08:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f564eeee640 1 -- start start 2024-09-16T08:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f564eeee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56480feaf0 0x7f5648068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f564eeee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f564819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f564eeee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564819a9a0 con 0x7f56480ffcf0 2024-09-16T08:58:21.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f564eeee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564819aae0 con 0x7f56480feaf0 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f5647fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f564819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f5647fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f564819a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50450/0 (socket says 172.21.15.31:50450) 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f5647fff640 1 -- 172.21.15.31:0/2829155170 learned_addr learned my addr 172.21.15.31:0/2829155170 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.728+0000 7f564cc8a640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56480feaf0 0x7f5648068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f5647fff640 1 -- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56480feaf0 msgr2=0x7f5648068790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f5647fff640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56480feaf0 0x7f5648068790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f5647fff640 1 -- 172.21.15.31:0/2829155170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c009900 con 0x7f56480ffcf0 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f564cc8a640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56480feaf0 0x7f5648068790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f5647fff640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f564819a3d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f563c006010 tx=0x7f563c02f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f5645ffb640 1 -- 172.21.15.31:0/2829155170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f563c030840 con 0x7f56480ffcf0 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564819ad30 con 0x7f56480ffcf0 2024-09-16T08:58:21.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f5645ffb640 1 -- 172.21.15.31:0/2829155170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f563c02f9f0 con 0x7f56480ffcf0 2024-09-16T08:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f5645ffb640 1 -- 172.21.15.31:0/2829155170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f563c039680 con 0x7f56480ffcf0 2024-09-16T08:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.729+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564819b220 con 0x7f56480ffcf0 2024-09-16T08:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.731+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564818e670 con 0x7f56480ffcf0 2024-09-16T08:58:21.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.732+0000 7f5645ffb640 1 -- 172.21.15.31:0/2829155170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f563c00a040 con 0x7f56480ffcf0 2024-09-16T08:58:21.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.732+0000 7f5645ffb640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f563007ace0 0x7f563007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:21.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.732+0000 7f5645ffb640 1 -- 172.21.15.31:0/2829155170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f563c02c040 con 0x7f56480ffcf0 2024-09-16T08:58:21.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.732+0000 7f564cc8a640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f563007ace0 0x7f563007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:21.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.733+0000 7f564cc8a640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f563007ace0 0x7f563007d1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56380058c0 tx=0x7f5638005850 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:21.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.739+0000 7f5645ffb640 1 -- 172.21.15.31:0/2829155170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f563c082540 con 0x7f56480ffcf0 2024-09-16T08:58:21.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:21 smithi031 ceph-mon[27844]: pgmap v42: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 3.0 MiB/s wr, 622 op/s 2024-09-16T08:58:21.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.961+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f564802c450 con 0x7f563007ace0 2024-09-16T08:58:21.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.970+0000 7f5645ffb640 1 -- 172.21.15.31:0/2829155170 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f564802c450 con 0x7f563007ace0 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (7m) 5s ago 9m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (45s) 5s ago 10m 7490k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (35s) 25s ago 9m 7356k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (10m) 5s ago 10m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (8m) 25s ago 8m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (7m) 5s ago 9m 86.9M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (3m) 5s ago 3m 13.7M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (3m) 5s ago 3m 228M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (3m) 25s ago 3m 23.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (3m) 25s ago 3m 15.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:58:21.973 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (29s) 5s ago 11m 87.2M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (99s) 25s ago 8m 508M - 19.3.0-4917-g8293d73f 425e316ba8eb 6aa6df83241b 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (11m) 5s ago 11m 53.8M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (8m) 25s ago 8m 42.4M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (9m) 5s ago 9m 15.5M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (8m) 25s ago 8m 16.5M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 5s ago 7m 200M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 5s ago 6m 221M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (6m) 5s ago 6m 176M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (6m) 25s ago 6m 236M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (5m) 25s ago 5m 205M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (5m) 25s ago 5m 194M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:58:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (39s) 5s ago 9m 44.5M - 2.43.0 a07b618ecd1d 0f6707427d62 2024-09-16T08:58:21.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f563007ace0 msgr2=0x7f563007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f563007ace0 0x7f563007d1a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f56380058c0 tx=0x7f5638005850 comp rx=0 tx=0).stop 2024-09-16T08:58:21.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 msgr2=0x7f564819a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:21.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f564819a3d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f563c006010 tx=0x7f563c02f5f0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 shutdown_connections 2024-09-16T08:58:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f563007ace0 0x7f563007d1a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56480ffcf0 0x7f564819a3d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 --2- 172.21.15.31:0/2829155170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56480feaf0 0x7f5648068790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 >> 172.21.15.31:0/2829155170 conn(0x7f56480fa280 msgr2=0x7f56481033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 shutdown_connections 2024-09-16T08:58:21.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:21.973+0000 7f564eeee640 1 -- 172.21.15.31:0/2829155170 wait complete. 2024-09-16T08:58:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 -- 172.21.15.31:0/3849730955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fedb0 msgr2=0x7fd0980ff210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/3849730955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fedb0 0x7fd0980ff210 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009880 tx=0x7fd08c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 -- 172.21.15.31:0/3849730955 shutdown_connections 2024-09-16T08:58:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/3849730955 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fedb0 0x7fd0980ff210 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/3849730955 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0980fe470 0x7fd0980fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 -- 172.21.15.31:0/3849730955 >> 172.21.15.31:0/3849730955 conn(0x7fd0980fa280 msgr2=0x7fd0980fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 -- 172.21.15.31:0/3849730955 shutdown_connections 2024-09-16T08:58:22.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.063+0000 7fd09e2bc640 1 -- 172.21.15.31:0/3849730955 wait complete. 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.064+0000 7fd09e2bc640 1 Processor -- start 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.064+0000 7fd09e2bc640 1 -- start start 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.064+0000 7fd09e2bc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fe470 0x7fd0981978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.064+0000 7fd09e2bc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0980fedb0 0x7fd098197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.064+0000 7fd09e2bc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0981983c0 con 0x7fd0980fe470 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.064+0000 7fd09e2bc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd098198530 con 0x7fd0980fedb0 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd097fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fe470 0x7fd0981978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd097fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fe470 0x7fd0981978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50468/0 (socket says 172.21.15.31:50468) 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd097fff640 1 -- 172.21.15.31:0/2899429346 learned_addr learned my addr 172.21.15.31:0/2899429346 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd0977fe640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0980fedb0 0x7fd098197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd097fff640 1 -- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0980fedb0 msgr2=0x7fd098197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd097fff640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0980fedb0 0x7fd098197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd097fff640 1 -- 172.21.15.31:0/2899429346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd088009490 con 0x7fd0980fe470 2024-09-16T08:58:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd097fff640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fe470 0x7fd0981978b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ea10 tx=0x7fd08800eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:22.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd0957fa640 1 -- 172.21.15.31:0/2899429346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08800cd50 con 0x7fd0980fe470 2024-09-16T08:58:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c009560 con 0x7fd0980fe470 2024-09-16T08:58:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd0957fa640 1 -- 172.21.15.31:0/2899429346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd08800ceb0 con 0x7fd0980fe470 2024-09-16T08:58:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd0957fa640 1 -- 172.21.15.31:0/2899429346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd088004a60 con 0x7fd0980fe470 2024-09-16T08:58:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.066+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09819d2c0 con 0x7fd0980fe470 2024-09-16T08:58:22.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.067+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd098191950 con 0x7fd0980fe470 2024-09-16T08:58:22.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.068+0000 7fd0957fa640 1 -- 172.21.15.31:0/2899429346 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 103129+0+0 (secure 0 0 0) 0x7fd088004d50 con 0x7fd0980fe470 2024-09-16T08:58:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.069+0000 7fd0957fa640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fd08007ace0 0x7fd08007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.069+0000 7fd0957fa640 1 -- 172.21.15.31:0/2899429346 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fd088014070 con 0x7fd0980fe470 2024-09-16T08:58:22.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.070+0000 7fd0977fe640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fd08007ace0 0x7fd08007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:22.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.070+0000 7fd0977fe640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fd08007ace0 0x7fd08007d1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd08c000c00 tx=0x7fd08c0314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:22.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.074+0000 7fd0957fa640 1 -- 172.21.15.31:0/2899429346 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7fd0880612d0 con 0x7fd0980fe470 2024-09-16T08:58:22.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:21 smithi097 ceph-mon[35478]: pgmap v42: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 32 KiB/s rd, 3.0 MiB/s wr, 622 op/s 2024-09-16T08:58:22.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.439+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd09810ac70 con 0x7fd0980fe470 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.440+0000 7fd0957fa640 1 -- 172.21.15.31:0/2899429346 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7fd0880643f0 con 0x7fd0980fe470 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:58:22.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 12, 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:58:22.442 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.443+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fd08007ace0 msgr2=0x7fd08007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.443+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fd08007ace0 0x7fd08007d1a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd08c000c00 tx=0x7fd08c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.444+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fe470 msgr2=0x7fd0981978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.444+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fe470 0x7fd0981978b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd08800ea10 tx=0x7fd08800eed0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.444+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 shutdown_connections 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.444+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7fd08007ace0 0x7fd08007d1a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.444+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0980fedb0 0x7fd098197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.444+0000 7fd09e2bc640 1 --2- 172.21.15.31:0/2899429346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd0980fe470 0x7fd0981978b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.444+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 >> 172.21.15.31:0/2899429346 conn(0x7fd0980fa280 msgr2=0x7fd098106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:22.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.445+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 shutdown_connections 2024-09-16T08:58:22.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.445+0000 7fd09e2bc640 1 -- 172.21.15.31:0/2899429346 wait complete. 2024-09-16T08:58:22.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.571+0000 7f1e58982640 1 -- 172.21.15.31:0/1959094538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54108820 msgr2=0x7f1e54108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.572+0000 7f1e58982640 1 --2- 172.21.15.31:0/1959094538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54108820 0x7f1e54108c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009880 tx=0x7f1e4402a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.572+0000 7f1e58982640 1 -- 172.21.15.31:0/1959094538 shutdown_connections 2024-09-16T08:58:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.572+0000 7f1e58982640 1 --2- 172.21.15.31:0/1959094538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e541091f0 0x7f1e540712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.572+0000 7f1e58982640 1 --2- 172.21.15.31:0/1959094538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e54108820 0x7f1e54108c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.572+0000 7f1e58982640 1 -- 172.21.15.31:0/1959094538 >> 172.21.15.31:0/1959094538 conn(0x7f1e5406cc10 msgr2=0x7f1e5406f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.573+0000 7f1e58982640 1 -- 172.21.15.31:0/1959094538 shutdown_connections 2024-09-16T08:58:22.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.573+0000 7f1e58982640 1 -- 172.21.15.31:0/1959094538 wait complete. 2024-09-16T08:58:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.574+0000 7f1e58982640 1 Processor -- start 2024-09-16T08:58:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.574+0000 7f1e58982640 1 -- start start 2024-09-16T08:58:22.575 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.574+0000 7f1e58982640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e54108820 0x7f1e5419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.575+0000 7f1e58982640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e541091f0 0x7f1e5419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.575+0000 7f1e58982640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5419ea50 con 0x7f1e54108820 2024-09-16T08:58:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.575+0000 7f1e58982640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5419ebc0 con 0x7f1e541091f0 2024-09-16T08:58:22.576 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.575+0000 7f1e51d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e541091f0 0x7f1e5419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.575+0000 7f1e51d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e541091f0 0x7f1e5419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:41996/0 (socket says 172.21.15.31:41996) 2024-09-16T08:58:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.575+0000 7f1e51d9b640 1 -- 172.21.15.31:0/37678154 learned_addr learned my addr 172.21.15.31:0/37678154 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:22.577 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.576+0000 7f1e5259c640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e54108820 0x7f1e5419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.577+0000 7f1e51d9b640 1 -- 172.21.15.31:0/37678154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e54108820 msgr2=0x7f1e5419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.577+0000 7f1e51d9b640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e54108820 0x7f1e5419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.577+0000 7f1e51d9b640 1 -- 172.21.15.31:0/37678154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e44009560 con 0x7f1e541091f0 2024-09-16T08:58:22.578 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.577+0000 7f1e51d9b640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e541091f0 0x7f1e5419e480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00b3f0 tx=0x7f1e3c00b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.577+0000 7f1e4b7fe640 1 -- 172.21.15.31:0/37678154 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c011800 con 0x7f1e541091f0 2024-09-16T08:58:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.577+0000 7f1e4b7fe640 1 -- 172.21.15.31:0/37678154 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e3c011e20 con 0x7f1e541091f0 2024-09-16T08:58:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.577+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e541a3660 con 0x7f1e541091f0 2024-09-16T08:58:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.578+0000 7f1e4b7fe640 1 -- 172.21.15.31:0/37678154 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c00f2e0 con 0x7f1e541091f0 2024-09-16T08:58:22.579 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.578+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e541a3c30 con 0x7f1e541091f0 2024-09-16T08:58:22.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.580+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e54110f90 con 0x7f1e541091f0 2024-09-16T08:58:22.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.580+0000 7f1e4b7fe640 1 -- 172.21.15.31:0/37678154 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f1e3c011960 con 0x7f1e541091f0 2024-09-16T08:58:22.582 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.581+0000 7f1e4b7fe640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1e4007ad20 0x7f1e4007d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.582+0000 7f1e4b7fe640 1 -- 172.21.15.31:0/37678154 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f1e3c09d620 con 0x7f1e541091f0 2024-09-16T08:58:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.582+0000 7f1e5259c640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1e4007ad20 0x7f1e4007d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:22.583 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.583+0000 7f1e5259c640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1e4007ad20 0x7f1e4007d1e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402aac0 tx=0x7f1e440314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:22.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.589+0000 7f1e4b7fe640 1 -- 172.21.15.31:0/37678154 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f1e3c060bc0 con 0x7f1e541091f0 2024-09-16T08:58:22.912 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.911+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1e54111120 con 0x7f1e541091f0 2024-09-16T08:58:22.913 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.912+0000 7f1e4b7fe640 1 -- 172.21.15.31:0/37678154 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f1e3c015750 con 0x7f1e541091f0 2024-09-16T08:58:22.913 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:58:22.913 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:58:22.913 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:58:22.913 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:58:22.914 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:58:22.915 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:22.916 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1e4007ad20 msgr2=0x7f1e4007d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1e4007ad20 0x7f1e4007d1e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1e4402aac0 tx=0x7f1e440314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e541091f0 msgr2=0x7f1e5419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e541091f0 0x7f1e5419e480 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00b3f0 tx=0x7f1e3c00b8b0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 shutdown_connections 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f1e4007ad20 0x7f1e4007d1e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e541091f0 0x7f1e5419e480 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.917 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 --2- 172.21.15.31:0/37678154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e54108820 0x7f1e5419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 >> 172.21.15.31:0/37678154 conn(0x7f1e5406cc10 msgr2=0x7f1e5410b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 shutdown_connections 2024-09-16T08:58:22.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:22.915+0000 7f1e58982640 1 -- 172.21.15.31:0/37678154 wait complete. 2024-09-16T08:58:23.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 -- 172.21.15.31:0/3186792881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c100680 msgr2=0x7f704c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 --2- 172.21.15.31:0/3186792881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c100680 0x7f704c102a70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f70400098b0 tx=0x7f704002a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 -- 172.21.15.31:0/3186792881 shutdown_connections 2024-09-16T08:58:23.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 --2- 172.21.15.31:0/3186792881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704c102fb0 0x7f704c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 --2- 172.21.15.31:0/3186792881 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c100680 0x7f704c102a70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 -- 172.21.15.31:0/3186792881 >> 172.21.15.31:0/3186792881 conn(0x7f704c0fa280 msgr2=0x7f704c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:23.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:22 smithi031 ceph-mon[27844]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:23.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:22 smithi031 ceph-mon[27844]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:23.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:22 smithi031 ceph-mon[27844]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:23.005 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:22 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2899429346' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:58:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 -- 172.21.15.31:0/3186792881 shutdown_connections 2024-09-16T08:58:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.003+0000 7f7052ac0640 1 -- 172.21.15.31:0/3186792881 wait complete. 2024-09-16T08:58:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f7052ac0640 1 Processor -- start 2024-09-16T08:58:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f7052ac0640 1 -- start start 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f7052ac0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704c100680 0x7f704c0fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f7052ac0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c102fb0 0x7f704c0fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f7052ac0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0feb70 con 0x7f704c102fb0 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f7052ac0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704c0fece0 con 0x7f704c100680 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f704bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c102fb0 0x7f704c0fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f704bfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c102fb0 0x7f704c0fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50498/0 (socket says 172.21.15.31:50498) 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f704bfff640 1 -- 172.21.15.31:0/1191205164 learned_addr learned my addr 172.21.15.31:0/1191205164 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.004+0000 7f705085c640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704c100680 0x7f704c0fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f704bfff640 1 -- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704c100680 msgr2=0x7f704c0fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f704bfff640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704c100680 0x7f704c0fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f704bfff640 1 -- 172.21.15.31:0/1191205164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009560 con 0x7f704c102fb0 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f704bfff640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c102fb0 0x7f704c0fe630 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f703800e940 tx=0x7f703800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f7049ffb640 1 -- 172.21.15.31:0/1191205164 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7038009700 con 0x7f704c102fb0 2024-09-16T08:58:23.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f7049ffb640 1 -- 172.21.15.31:0/1191205164 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70380054e0 con 0x7f704c102fb0 2024-09-16T08:58:23.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704c0fefc0 con 0x7f704c102fb0 2024-09-16T08:58:23.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f7049ffb640 1 -- 172.21.15.31:0/1191205164 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7038010430 con 0x7f704c102fb0 2024-09-16T08:58:23.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.005+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704c19d750 con 0x7f704c102fb0 2024-09-16T08:58:23.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.007+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704c108810 con 0x7f704c102fb0 2024-09-16T08:58:23.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.007+0000 7f7049ffb640 1 -- 172.21.15.31:0/1191205164 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f7038005020 con 0x7f704c102fb0 2024-09-16T08:58:23.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.008+0000 7f7049ffb640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f703c07ad30 0x7f703c07d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:23.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.008+0000 7f7049ffb640 1 -- 172.21.15.31:0/1191205164 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7038014070 con 0x7f704c102fb0 2024-09-16T08:58:23.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.008+0000 7f705085c640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f703c07ad30 0x7f703c07d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:23.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.009+0000 7f705085c640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f703c07ad30 0x7f703c07d1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7040006010 tx=0x7f70400314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:22 smithi097 ceph-mon[35478]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:22 smithi097 ceph-mon[35478]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:22 smithi097 ceph-mon[35478]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:22 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2899429346' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:58:23.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.013+0000 7f7049ffb640 1 -- 172.21.15.31:0/1191205164 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f70380615e0 con 0x7f704c102fb0 2024-09-16T08:58:23.259 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.258+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 --> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f704c1039d0 con 0x7f703c07ad30 2024-09-16T08:58:23.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.260+0000 7f7049ffb640 1 -- 172.21.15.31:0/1191205164 <== mgr.24453 v2:172.21.15.97:6828/3476759025 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f704c1039d0 con 0x7f703c07ad30 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/23 daemons upgraded", 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mgr daemons", 2024-09-16T08:58:23.261 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:58:23.262 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:58:23.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f703c07ad30 msgr2=0x7f703c07d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f703c07ad30 0x7f703c07d1f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7040006010 tx=0x7f70400314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c102fb0 msgr2=0x7f704c0fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c102fb0 0x7f704c0fe630 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f703800e940 tx=0x7f703800ee00 comp rx=0 tx=0).stop 2024-09-16T08:58:23.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 shutdown_connections 2024-09-16T08:58:23.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f703c07ad30 0x7f703c07d1f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.263 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f704c102fb0 0x7f704c0fe630 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 --2- 172.21.15.31:0/1191205164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f704c100680 0x7f704c0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 >> 172.21.15.31:0/1191205164 conn(0x7f704c0fa280 msgr2=0x7f704c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:23.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 shutdown_connections 2024-09-16T08:58:23.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.262+0000 7f7052ac0640 1 -- 172.21.15.31:0/1191205164 wait complete. 2024-09-16T08:58:23.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 -- 172.21.15.31:0/760284240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fe4d0 msgr2=0x7f9d5c0fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/760284240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fe4d0 0x7f9d5c0fe8d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009880 tx=0x7f9d4c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 -- 172.21.15.31:0/760284240 shutdown_connections 2024-09-16T08:58:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/760284240 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d5c0fee10 0x7f9d5c0ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/760284240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fe4d0 0x7f9d5c0fe8d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 -- 172.21.15.31:0/760284240 >> 172.21.15.31:0/760284240 conn(0x7f9d5c0fa280 msgr2=0x7f9d5c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 -- 172.21.15.31:0/760284240 shutdown_connections 2024-09-16T08:58:23.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.357+0000 7f9d61ffa640 1 -- 172.21.15.31:0/760284240 wait complete. 2024-09-16T08:58:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.358+0000 7f9d61ffa640 1 Processor -- start 2024-09-16T08:58:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.358+0000 7f9d61ffa640 1 -- start start 2024-09-16T08:58:23.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.358+0000 7f9d61ffa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d5c0fe4d0 0x7f9d5c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.358+0000 7f9d61ffa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fee10 0x7f9d5c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.358+0000 7f9d61ffa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c198350 con 0x7f9d5c0fee10 2024-09-16T08:58:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.358+0000 7f9d61ffa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c1984c0 con 0x7f9d5c0fe4d0 2024-09-16T08:58:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fee10 0x7f9d5c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fee10 0x7f9d5c197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:50518/0 (socket says 172.21.15.31:50518) 2024-09-16T08:58:23.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5affd640 1 -- 172.21.15.31:0/3465599079 learned_addr learned my addr 172.21.15.31:0/3465599079 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5b7fe640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d5c0fe4d0 0x7f9d5c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5affd640 1 -- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d5c0fe4d0 msgr2=0x7f9d5c197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5affd640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d5c0fe4d0 0x7f9d5c197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5affd640 1 -- 172.21.15.31:0/3465599079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009560 con 0x7f9d5c0fee10 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d5affd640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fee10 0x7f9d5c197d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d900 tx=0x7f9d5000ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d58ff9640 1 -- 172.21.15.31:0/3465599079 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d50009c00 con 0x7f9d5c0fee10 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d58ff9640 1 -- 172.21.15.31:0/3465599079 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d5000b660 con 0x7f9d5c0fee10 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c19cf60 con 0x7f9d5c0fee10 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.359+0000 7f9d58ff9640 1 -- 172.21.15.31:0/3465599079 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5000f680 con 0x7f9d5c0fee10 2024-09-16T08:58:23.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.360+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c19d480 con 0x7f9d5c0fee10 2024-09-16T08:58:23.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.361+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c0fe8d0 con 0x7f9d5c0fee10 2024-09-16T08:58:23.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.362+0000 7f9d58ff9640 1 -- 172.21.15.31:0/3465599079 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 26) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f9d50010ab0 con 0x7f9d5c0fee10 2024-09-16T08:58:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.363+0000 7f9d58ff9640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9d4407ace0 0x7f9d4407d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.363+0000 7f9d58ff9640 1 -- 172.21.15.31:0/3465599079 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f9d5009d740 con 0x7f9d5c0fee10 2024-09-16T08:58:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.363+0000 7f9d5b7fe640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9d4407ace0 0x7f9d4407d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:23.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.364+0000 7f9d5b7fe640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9d4407ace0 0x7f9d4407d1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c000c00 tx=0x7f9d4c0314e0 comp rx=0 tx=0).ready entity=mgr.24453 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:23.368 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.368+0000 7f9d58ff9640 1 -- 172.21.15.31:0/3465599079 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f9d50060e30 con 0x7f9d5c0fee10 2024-09-16T08:58:23.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.729+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9d5c10aab0 con 0x7f9d5c0fee10 2024-09-16T08:58:23.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.729+0000 7f9d58ff9640 1 -- 172.21.15.31:0/3465599079 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f9d50063f50 con 0x7f9d5c0fee10 2024-09-16T08:58:23.731 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9d4407ace0 msgr2=0x7f9d4407d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9d4407ace0 0x7f9d4407d1a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c000c00 tx=0x7f9d4c0314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fee10 msgr2=0x7f9d5c197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fee10 0x7f9d5c197d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000d900 tx=0x7f9d5000ddc0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 shutdown_connections 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:6828/3476759025,v1:172.21.15.97:6829/3476759025] conn(0x7f9d4407ace0 0x7f9d4407d1a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d5c0fee10 0x7f9d5c197d80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 --2- 172.21.15.31:0/3465599079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d5c0fe4d0 0x7f9d5c197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 >> 172.21.15.31:0/3465599079 conn(0x7f9d5c0fa280 msgr2=0x7f9d5c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 shutdown_connections 2024-09-16T08:58:23.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:23.732+0000 7f9d61ffa640 1 -- 172.21.15.31:0/3465599079 wait complete. 2024-09-16T08:58:24.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:23 smithi031 ceph-mon[27844]: pgmap v43: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.8 MiB/s wr, 548 op/s 2024-09-16T08:58:24.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:23 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/37678154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:58:24.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:23 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/3465599079' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:58:24.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:23 smithi097 ceph-mon[35478]: pgmap v43: 65 pgs: 65 active+clean; 323 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.8 MiB/s wr, 548 op/s 2024-09-16T08:58:24.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:23 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/37678154' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:58:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:23 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/3465599079' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:58:24.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:24 smithi031 ceph-mon[27844]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:25.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:24 smithi097 ceph-mon[35478]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:25 smithi031 ceph-mon[27844]: pgmap v44: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.6 MiB/s wr, 915 op/s 2024-09-16T08:58:26.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:25 smithi097 ceph-mon[35478]: pgmap v44: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 41 KiB/s rd, 2.6 MiB/s wr, 915 op/s 2024-09-16T08:58:28.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:27 smithi097 ceph-mon[35478]: pgmap v45: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 791 op/s 2024-09-16T08:58:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:27 smithi031 ceph-mon[27844]: pgmap v45: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.7 MiB/s wr, 791 op/s 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr fail", "who": "smithi097.mpashf"}]: dispatch 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr fail", "who": "smithi097.mpashf"}]: dispatch 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:58:29.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd='[{"prefix": "mgr fail", "who": "smithi097.mpashf"}]': finished 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: mgrmap e27: smithi031.dcmqpv(active, starting, since 0.00872207s) 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:58:29.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:58:29.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:29 smithi031 ceph-mon[27844]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:58:29.751 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:29.751 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 172.21.15.97:0/4232871526' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr fail", "who": "smithi097.mpashf"}]: dispatch 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "mgr fail", "who": "smithi097.mpashf"}]: dispatch 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.24453 ' entity='mgr.smithi097.mpashf' cmd='[{"prefix": "mgr fail", "who": "smithi097.mpashf"}]': finished 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: mgrmap e27: smithi031.dcmqpv(active, starting, since 0.00872207s) 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:58:29.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T08:58:29.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:29 smithi097 ceph-mon[35478]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:58:30.597 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:58:30.597 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:30.597 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:58:30.597 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:58:30.597 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"}]: dispatch 2024-09-16T08:58:30.598 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: mgrmap e28: smithi031.dcmqpv(active, since 1.0142s) 2024-09-16T08:58:30.598 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:58:30.598 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:30 smithi097 ceph-mon[35478]: pgmap v3: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"}]: dispatch 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"}]: dispatch 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: mgrmap e28: smithi031.dcmqpv(active, since 1.0142s) 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:58:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:30 smithi031 ceph-mon[27844]: pgmap v3: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:58:31.697 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:31 smithi097 ceph-mon[35478]: mgrmap e29: smithi031.dcmqpv(active, since 2s) 2024-09-16T08:58:31.697 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:31 smithi097 ceph-mon[35478]: pgmap v4: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:58:31.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:31 smithi031 ceph-mon[27844]: mgrmap e29: smithi031.dcmqpv(active, since 2s) 2024-09-16T08:58:31.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:31 smithi031 ceph-mon[27844]: pgmap v4: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:58:32.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:32 smithi097 ceph-mon[35478]: [16/Sep/2024:08:58:32] ENGINE Bus STARTING 2024-09-16T08:58:32.770 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:32 smithi031 ceph-mon[27844]: [16/Sep/2024:08:58:32] ENGINE Bus STARTING 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: [16/Sep/2024:08:58:32] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: [16/Sep/2024:08:58:32] ENGINE Error in HTTPServer.serve 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Traceback (most recent call last): 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._run(expiration_interval) 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:58:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s = self.context.wrap_socket( 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: return self.sslsocket_class._create( 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self.do_handshake() 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._sslobj.do_handshake() 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: [16/Sep/2024:08:58:32] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: [16/Sep/2024:08:58:32] ENGINE Bus STARTED 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: pgmap v5: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: mgrmap e30: smithi031.dcmqpv(active, since 4s) 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:33.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:33 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: [16/Sep/2024:08:58:32] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: [16/Sep/2024:08:58:32] ENGINE Error in HTTPServer.serve 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Traceback (most recent call last): 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._run(expiration_interval) 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:58:33.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self.do_handshake() 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: [16/Sep/2024:08:58:32] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: [16/Sep/2024:08:58:32] ENGINE Bus STARTED 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: pgmap v5: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:58:33.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: mgrmap e30: smithi031.dcmqpv(active, since 4s) 2024-09-16T08:58:33.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:33.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:33 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:34 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:34 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:34 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:58:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:34 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:58:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:34 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:58:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:34 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:58:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:34 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:58:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:34 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:34 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:34 smithi031 ceph-mon[27844]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:58:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:34 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:58:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:34 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:58:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:34 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:58:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:34 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/955123988' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:58:35.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:35 smithi031 ceph-mon[27844]: pgmap v6: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s wr, 112 op/s 2024-09-16T08:58:35.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:35 smithi031 ceph-mon[27844]: mgrmap e31: smithi031.dcmqpv(active, since 6s), standbys: smithi097.mpashf 2024-09-16T08:58:35.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:35 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"}]: dispatch 2024-09-16T08:58:36.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:35 smithi097 ceph-mon[35478]: pgmap v6: 65 pgs: 65 active+clean; 281 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 6.4 KiB/s wr, 112 op/s 2024-09-16T08:58:36.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:35 smithi097 ceph-mon[35478]: mgrmap e31: smithi031.dcmqpv(active, since 6s), standbys: smithi097.mpashf 2024-09-16T08:58:36.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:35 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"}]: dispatch 2024-09-16T08:58:37.855 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:37 smithi031 ceph-mon[27844]: pgmap v7: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.3 MiB/s wr, 518 op/s 2024-09-16T08:58:38.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:37 smithi097 ceph-mon[35478]: pgmap v7: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 52 KiB/s rd, 1.3 MiB/s wr, 518 op/s 2024-09-16T08:58:40.064 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:39 smithi097 ceph-mon[35478]: pgmap v8: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.0 MiB/s wr, 402 op/s 2024-09-16T08:58:40.064 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:39 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:40.064 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:39 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:40.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:39 smithi031 ceph-mon[27844]: pgmap v8: 65 pgs: 65 active+clean; 273 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.0 MiB/s wr, 402 op/s 2024-09-16T08:58:40.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:39 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:40.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:39 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:41 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:41 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:41 smithi031 ceph-mon[27844]: pgmap v9: 65 pgs: 65 active+clean; 261 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 1.7 MiB/s wr, 749 op/s 2024-09-16T08:58:41.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:41 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:41.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:41 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:41.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:41 smithi097 ceph-mon[35478]: pgmap v9: 65 pgs: 65 active+clean; 261 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 49 KiB/s rd, 1.7 MiB/s wr, 749 op/s 2024-09-16T08:58:43.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:43 smithi097 ceph-mon[35478]: pgmap v10: 65 pgs: 65 active+clean; 261 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.5 MiB/s wr, 685 op/s 2024-09-16T08:58:43.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:43 smithi031 ceph-mon[27844]: pgmap v10: 65 pgs: 65 active+clean; 261 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.5 MiB/s wr, 685 op/s 2024-09-16T08:58:44.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:44 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:44.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:44 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:58:45.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:45 smithi097 ceph-mon[35478]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.6 MiB/s wr, 743 op/s 2024-09-16T08:58:45.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:45 smithi031 ceph-mon[27844]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45 KiB/s rd, 1.6 MiB/s wr, 743 op/s 2024-09-16T08:58:47.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:47 smithi097 ceph-mon[35478]: pgmap v12: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.3 MiB/s wr, 995 op/s 2024-09-16T08:58:47.872 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:47 smithi031 ceph-mon[27844]: pgmap v12: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.3 MiB/s wr, 995 op/s 2024-09-16T08:58:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: pgmap v13: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.5 MiB/s wr, 741 op/s 2024-09-16T08:58:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:50 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: pgmap v13: 65 pgs: 65 active+clean; 250 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1.5 MiB/s wr, 741 op/s 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:50 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:58:50.719 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-09-16T08:58:50.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-16T08:58:51.071 DEBUG:teuthology.parallel:result is None 2024-09-16T08:58:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:58:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:58:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:58:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:58:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:58:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:58:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-09-16T08:58:51.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:51 smithi031 ceph-mon[27844]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-09-16T08:58:51.431 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:58:51.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:58:51.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:58:51.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:58:51.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:58:51.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:58:51.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Reconfiguring prometheus.smithi031 (dependencies changed)... 2024-09-16T08:58:51.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:51 smithi097 ceph-mon[35478]: Reconfiguring daemon prometheus.smithi031 on smithi031 2024-09-16T08:58:52.139 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:52 smithi031 ceph-mon[27844]: pgmap v14: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 2.2 MiB/s wr, 960 op/s 2024-09-16T08:58:52.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:52 smithi097 ceph-mon[35478]: pgmap v14: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 39 KiB/s rd, 2.2 MiB/s wr, 960 op/s 2024-09-16T08:58:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.875+0000 7f732e6c4640 1 -- 172.21.15.31:0/1702026776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 msgr2=0x7f732810b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:53.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.875+0000 7f732e6c4640 1 --2- 172.21.15.31:0/1702026776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 0x7f732810b560 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7318009880 tx=0x7f731802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.876+0000 7f732e6c4640 1 -- 172.21.15.31:0/1702026776 shutdown_connections 2024-09-16T08:58:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.876+0000 7f732e6c4640 1 --2- 172.21.15.31:0/1702026776 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 0x7f732810b560 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.876+0000 7f732e6c4640 1 --2- 172.21.15.31:0/1702026776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 0x7f73280715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:53.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.876+0000 7f732e6c4640 1 -- 172.21.15.31:0/1702026776 >> 172.21.15.31:0/1702026776 conn(0x7f732806c9f0 msgr2=0x7f732806ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.878+0000 7f732e6c4640 1 -- 172.21.15.31:0/1702026776 shutdown_connections 2024-09-16T08:58:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.880+0000 7f732e6c4640 1 -- 172.21.15.31:0/1702026776 wait complete. 2024-09-16T08:58:53.881 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.880+0000 7f732e6c4640 1 Processor -- start 2024-09-16T08:58:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732e6c4640 1 -- start start 2024-09-16T08:58:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732e6c4640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 0x7f73281a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732e6c4640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 0x7f73281a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732e6c4640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281a4d70 con 0x7f7328071bc0 2024-09-16T08:58:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732e6c4640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73281a4ee0 con 0x7f73280711f0 2024-09-16T08:58:53.886 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732d6c2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 0x7f73281a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:53.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732d6c2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 0x7f73281a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:44250/0 (socket says 172.21.15.31:44250) 2024-09-16T08:58:53.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.884+0000 7f732d6c2640 1 -- 172.21.15.31:0/279116642 learned_addr learned my addr 172.21.15.31:0/279116642 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:53.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.886+0000 7f732cec1640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 0x7f73281a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:53.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.888+0000 7f732d6c2640 1 -- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 msgr2=0x7f73281a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:53.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.888+0000 7f732d6c2640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 0x7f73281a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:53.889 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.888+0000 7f732d6c2640 1 -- 172.21.15.31:0/279116642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7318009560 con 0x7f73280711f0 2024-09-16T08:58:53.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.889+0000 7f732d6c2640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 0x7f73281a4260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f732400c8f0 tx=0x7f732400cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:53.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.890+0000 7f731e7fc640 1 -- 172.21.15.31:0/279116642 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7324007b80 con 0x7f73280711f0 2024-09-16T08:58:53.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.890+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732810ed80 con 0x7f73280711f0 2024-09-16T08:58:53.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.890+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732810f320 con 0x7f73280711f0 2024-09-16T08:58:53.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.890+0000 7f731e7fc640 1 -- 172.21.15.31:0/279116642 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7324007ce0 con 0x7f73280711f0 2024-09-16T08:58:53.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.890+0000 7f731e7fc640 1 -- 172.21.15.31:0/279116642 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7324018800 con 0x7f73280711f0 2024-09-16T08:58:53.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.896+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c005280 con 0x7f73280711f0 2024-09-16T08:58:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.898+0000 7f731e7fc640 1 -- 172.21.15.31:0/279116642 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 31) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f7324018960 con 0x7f73280711f0 2024-09-16T08:58:53.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.899+0000 7f731e7fc640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f731407ae00 0x7f731407d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:53.901 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.899+0000 7f731e7fc640 1 -- 172.21.15.31:0/279116642 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f732409dc00 con 0x7f73280711f0 2024-09-16T08:58:53.902 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.901+0000 7f732cec1640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f731407ae00 0x7f731407d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:53.903 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.902+0000 7f732cec1640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f731407ae00 0x7f731407d2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7318006010 tx=0x7f73180314e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:53.905 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:53.905+0000 7f731e7fc640 1 -- 172.21.15.31:0/279116642 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f73240610d0 con 0x7f73280711f0 2024-09-16T08:58:54.159 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:54 smithi031 ceph-mon[27844]: pgmap v15: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.5 MiB/s wr, 575 op/s 2024-09-16T08:58:54.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.158+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 --> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f730c000bf0 con 0x7f731407ae00 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.160+0000 7f731e7fc640 1 -- 172.21.15.31:0/279116642 <== mgr.14772 v2:172.21.15.31:6800/2307887580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f730c000bf0 con 0x7f731407ae00 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f731407ae00 msgr2=0x7f731407d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f731407ae00 0x7f731407d2c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7318006010 tx=0x7f73180314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 msgr2=0x7f73281a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 0x7f73281a4260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f732400c8f0 tx=0x7f732400cdb0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 shutdown_connections 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f731407ae00 0x7f731407d2c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7328071bc0 0x7f73281a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 --2- 172.21.15.31:0/279116642 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73280711f0 0x7f73281a4260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 >> 172.21.15.31:0/279116642 conn(0x7f732806c9f0 msgr2=0x7f73281098d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 shutdown_connections 2024-09-16T08:58:54.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.162+0000 7f732e6c4640 1 -- 172.21.15.31:0/279116642 wait complete. 2024-09-16T08:58:54.191 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:58:54.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 -- 172.21.15.31:0/2179834115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 msgr2=0x7fac900ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 --2- 172.21.15.31:0/2179834115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 0x7fac900ff270 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fac84009880 tx=0x7fac8402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 -- 172.21.15.31:0/2179834115 shutdown_connections 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 --2- 172.21.15.31:0/2179834115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 0x7fac900ff270 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 --2- 172.21.15.31:0/2179834115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac900fe4d0 0x7fac900fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 -- 172.21.15.31:0/2179834115 >> 172.21.15.31:0/2179834115 conn(0x7fac900fa280 msgr2=0x7fac900fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 -- 172.21.15.31:0/2179834115 shutdown_connections 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.263+0000 7fac959d5640 1 -- 172.21.15.31:0/2179834115 wait complete. 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac959d5640 1 Processor -- start 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac959d5640 1 -- start start 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac959d5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac900fe4d0 0x7fac901978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac959d5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 0x7fac90197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac959d5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac901983e0 con 0x7fac900fee10 2024-09-16T08:58:54.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac959d5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90198550 con 0x7fac900fe4d0 2024-09-16T08:58:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 0x7fac90197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac8e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 0x7fac90197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60008/0 (socket says 172.21.15.31:60008) 2024-09-16T08:58:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.264+0000 7fac8e7fc640 1 -- 172.21.15.31:0/1088909194 learned_addr learned my addr 172.21.15.31:0/1088909194 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:54.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac8effd640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac900fe4d0 0x7fac901978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:54.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac8effd640 1 -- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 msgr2=0x7fac90197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac8effd640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 0x7fac90197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac8effd640 1 -- 172.21.15.31:0/1088909194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac84009560 con 0x7fac900fe4d0 2024-09-16T08:58:54.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac8effd640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac900fe4d0 0x7fac901978d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac8000e970 tx=0x7fac8000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:54.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac949d3640 1 -- 172.21.15.31:0/1088909194 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac80009620 con 0x7fac900fe4d0 2024-09-16T08:58:54.268 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9019cfa0 con 0x7fac900fe4d0 2024-09-16T08:58:54.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac949d3640 1 -- 172.21.15.31:0/1088909194 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac800044f0 con 0x7fac900fe4d0 2024-09-16T08:58:54.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9019d460 con 0x7fac900fe4d0 2024-09-16T08:58:54.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.265+0000 7fac949d3640 1 -- 172.21.15.31:0/1088909194 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac800049d0 con 0x7fac900fe4d0 2024-09-16T08:58:54.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.267+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac900fe8d0 con 0x7fac900fe4d0 2024-09-16T08:58:54.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.267+0000 7fac949d3640 1 -- 172.21.15.31:0/1088909194 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 31) v1 ==== 103129+0+0 (secure 0 0 0) 0x7fac8000cc90 con 0x7fac900fe4d0 2024-09-16T08:58:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.268+0000 7fac949d3640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fac7807ace0 0x7fac7807d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.268+0000 7fac949d3640 1 -- 172.21.15.31:0/1088909194 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fac80014070 con 0x7fac900fe4d0 2024-09-16T08:58:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.268+0000 7fac8e7fc640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fac7807ace0 0x7fac7807d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:54.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.268+0000 7fac8e7fc640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fac7807ace0 0x7fac7807d1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fac90198da0 tx=0x7fac840314e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:54.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.272+0000 7fac949d3640 1 -- 172.21.15.31:0/1088909194 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7fac800613d0 con 0x7fac900fe4d0 2024-09-16T08:58:54.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:54 smithi097 ceph-mon[35478]: pgmap v15: 65 pgs: 65 active+clean; 235 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.5 MiB/s wr, 575 op/s 2024-09-16T08:58:54.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.548+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 --> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac9002c570 con 0x7fac7807ace0 2024-09-16T08:58:54.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.549+0000 7fac949d3640 1 -- 172.21.15.31:0/1088909194 <== mgr.14772 v2:172.21.15.31:6800/2307887580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fac9002c570 con 0x7fac7807ace0 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fac7807ace0 msgr2=0x7fac7807d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fac7807ace0 0x7fac7807d1a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fac90198da0 tx=0x7fac840314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac900fe4d0 msgr2=0x7fac901978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac900fe4d0 0x7fac901978d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac8000e970 tx=0x7fac8000ee30 comp rx=0 tx=0).stop 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 shutdown_connections 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fac7807ace0 0x7fac7807d1a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac900fee10 0x7fac90197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 --2- 172.21.15.31:0/1088909194 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac900fe4d0 0x7fac901978d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 >> 172.21.15.31:0/1088909194 conn(0x7fac900fa280 msgr2=0x7fac90106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 shutdown_connections 2024-09-16T08:58:54.554 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.553+0000 7fac959d5640 1 -- 172.21.15.31:0/1088909194 wait complete. 2024-09-16T08:58:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.654+0000 7f82f89a2640 1 -- 172.21.15.31:0/1437329428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4102f60 msgr2=0x7f82f4105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.654+0000 7f82f89a2640 1 --2- 172.21.15.31:0/1437329428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4102f60 0x7f82f4105350 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f82e80098b0 tx=0x7f82e802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.655+0000 7f82f89a2640 1 -- 172.21.15.31:0/1437329428 shutdown_connections 2024-09-16T08:58:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.655+0000 7f82f89a2640 1 --2- 172.21.15.31:0/1437329428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4102f60 0x7f82f4105350 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.655+0000 7f82f89a2640 1 --2- 172.21.15.31:0/1437329428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f4100630 0x7f82f4102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.655+0000 7f82f89a2640 1 -- 172.21.15.31:0/1437329428 >> 172.21.15.31:0/1437329428 conn(0x7f82f40fa230 msgr2=0x7f82f40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:54.656 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.655+0000 7f82f89a2640 1 -- 172.21.15.31:0/1437329428 shutdown_connections 2024-09-16T08:58:54.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.655+0000 7f82f89a2640 1 -- 172.21.15.31:0/1437329428 wait complete. 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.655+0000 7f82f89a2640 1 Processor -- start 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f89a2640 1 -- start start 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f89a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4100630 0x7f82f40fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f89a2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f4102f60 0x7f82f40fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f89a2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f41004d0 con 0x7f82f4100630 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f89a2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40feb50 con 0x7f82f4102f60 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4100630 0x7f82f40fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4100630 0x7f82f40fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60014/0 (socket says 172.21.15.31:60014) 2024-09-16T08:58:54.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f259c640 1 -- 172.21.15.31:0/2218634440 learned_addr learned my addr 172.21.15.31:0/2218634440 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f1d9b640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f4102f60 0x7f82f40fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f259c640 1 -- 172.21.15.31:0/2218634440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f4102f60 msgr2=0x7f82f40fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f259c640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f4102f60 0x7f82f40fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.656+0000 7f82f259c640 1 -- 172.21.15.31:0/2218634440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8009560 con 0x7f82f4100630 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.657+0000 7f82f259c640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4100630 0x7f82f40fff90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f82e400c8a0 tx=0x7f82e400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.657+0000 7f82e37fe640 1 -- 172.21.15.31:0/2218634440 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e4007b80 con 0x7f82f4100630 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.657+0000 7f82e37fe640 1 -- 172.21.15.31:0/2218634440 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f82e4007ce0 con 0x7f82f4100630 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.657+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f40fee30 con 0x7f82f4100630 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.657+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f40ff370 con 0x7f82f4100630 2024-09-16T08:58:54.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.657+0000 7f82e37fe640 1 -- 172.21.15.31:0/2218634440 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82e4018810 con 0x7f82f4100630 2024-09-16T08:58:54.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.658+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f4100a10 con 0x7f82f4100630 2024-09-16T08:58:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.659+0000 7f82e37fe640 1 -- 172.21.15.31:0/2218634440 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f82e4018970 con 0x7f82f4100630 2024-09-16T08:58:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.660+0000 7f82e37fe640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f82dc07ad20 0x7f82dc07d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.660+0000 7f82e37fe640 1 -- 172.21.15.31:0/2218634440 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f82e409dde0 con 0x7f82f4100630 2024-09-16T08:58:54.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.660+0000 7f82f1d9b640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f82dc07ad20 0x7f82dc07d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:54.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.661+0000 7f82f1d9b640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f82dc07ad20 0x7f82dc07d1e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009880 tx=0x7f82e80314e0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:54.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:54.665+0000 7f82e37fe640 1 -- 172.21.15.31:0/2218634440 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f82e4061230 con 0x7f82f4100630 2024-09-16T08:58:55.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.006+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 --> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f82f4103980 con 0x7f82dc07ad20 2024-09-16T08:58:55.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.029+0000 7f82e37fe640 1 -- 172.21.15.31:0/2218634440 <== mgr.14772 v2:172.21.15.31:6800/2307887580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f82f4103980 con 0x7f82dc07ad20 2024-09-16T08:58:55.030 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (8m) 15s ago 10m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (78s) 15s ago 10m 7566k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (68s) 21s ago 9m 7436k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (10m) 15s ago 10m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (9m) 21s ago 9m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (8m) 15s ago 10m 85.9M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (4m) 15s ago 4m 14.1M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (4m) 15s ago 4m 236M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (4m) 21s ago 4m 25.2M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (4m) 21s ago 4m 16.0M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (62s) 15s ago 11m 493M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T08:58:55.031 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (2m) 21s ago 9m 149M - 19.3.0-4917-g8293d73f 425e316ba8eb 6aa6df83241b 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (11m) 15s ago 11m 57.5M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (9m) 21s ago 9m 45.0M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (10m) 15s ago 10m 15.6M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (9m) 21s ago 9m 16.4M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (7m) 15s ago 7m 219M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (7m) 15s ago 7m 247M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (6m) 15s ago 6m 205M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (6m) 21s ago 6m 258M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (6m) 21s ago 6m 228M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (5m) 21s ago 5m 213M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:58:55.032 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (72s) 15s ago 9m 47.0M - 2.43.0 a07b618ecd1d 0f6707427d62 2024-09-16T08:58:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.032+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f82dc07ad20 msgr2=0x7f82dc07d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.032+0000 7f82f89a2640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f82dc07ad20 0x7f82dc07d1e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009880 tx=0x7f82e80314e0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.032+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4100630 msgr2=0x7f82f40fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.032+0000 7f82f89a2640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4100630 0x7f82f40fff90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f82e400c8a0 tx=0x7f82e400cd60 comp rx=0 tx=0).stop 2024-09-16T08:58:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.033+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 shutdown_connections 2024-09-16T08:58:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.033+0000 7f82f89a2640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f82dc07ad20 0x7f82dc07d1e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.033+0000 7f82f89a2640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f82f4102f60 0x7f82f40fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.033+0000 7f82f89a2640 1 --2- 172.21.15.31:0/2218634440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82f4100630 0x7f82f40fff90 secure :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f82e400c8a0 tx=0x7f82e400cd60 comp rx=0 tx=0).stop 2024-09-16T08:58:55.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.033+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 >> 172.21.15.31:0/2218634440 conn(0x7f82f40fa230 msgr2=0x7f82f40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.034+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 shutdown_connections 2024-09-16T08:58:55.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.035+0000 7f82f89a2640 1 -- 172.21.15.31:0/2218634440 wait complete. 2024-09-16T08:58:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 -- 172.21.15.31:0/995566214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 msgr2=0x7f0b001053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.149 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 --2- 172.21.15.31:0/995566214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b001053a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80098b0 tx=0x7f0ae802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 -- 172.21.15.31:0/995566214 shutdown_connections 2024-09-16T08:58:55.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 --2- 172.21.15.31:0/995566214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b001053a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 --2- 172.21.15.31:0/995566214 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b00100680 0x7f0b00102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 -- 172.21.15.31:0/995566214 >> 172.21.15.31:0/995566214 conn(0x7f0b000fa280 msgr2=0x7f0b000fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:55.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 -- 172.21.15.31:0/995566214 shutdown_connections 2024-09-16T08:58:55.150 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.148+0000 7f0b050db640 1 -- 172.21.15.31:0/995566214 wait complete. 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.149+0000 7f0b050db640 1 Processor -- start 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.149+0000 7f0b050db640 1 -- start start 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.149+0000 7f0b050db640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b00100680 0x7f0b000fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.149+0000 7f0b050db640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b000fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.149+0000 7f0b050db640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000feb70 con 0x7f0b00102fb0 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.149+0000 7f0b050db640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b000fece0 con 0x7f0b00100680 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.149+0000 7f0afe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b000fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0afe59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b000fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60038/0 (socket says 172.21.15.31:60038) 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0afe59c640 1 -- 172.21.15.31:0/2304443687 learned_addr learned my addr 172.21.15.31:0/2304443687 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:55.152 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0afed9d640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b00100680 0x7f0b000fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0afe59c640 1 -- 172.21.15.31:0/2304443687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b00100680 msgr2=0x7f0b000fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0afe59c640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b00100680 0x7f0b000fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0afe59c640 1 -- 172.21.15.31:0/2304443687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8009560 con 0x7f0b00102fb0 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0afe59c640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b000fe630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ae802a9f0 tx=0x7f0ae80055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0af7fff640 1 -- 172.21.15.31:0/2304443687 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae8004aa0 con 0x7f0b00102fb0 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0af7fff640 1 -- 172.21.15.31:0/2304443687 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ae8004c00 con 0x7f0b00102fb0 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b000fef60 con 0x7f0b00102fb0 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0af7fff640 1 -- 172.21.15.31:0/2304443687 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae8003a80 con 0x7f0b00102fb0 2024-09-16T08:58:55.153 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.150+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b000ff400 con 0x7f0b00102fb0 2024-09-16T08:58:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.151+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b00100a60 con 0x7f0b00102fb0 2024-09-16T08:58:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.152+0000 7f0af7fff640 1 -- 172.21.15.31:0/2304443687 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f0ae8003d40 con 0x7f0b00102fb0 2024-09-16T08:58:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.153+0000 7f0af7fff640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f0aec07ace0 0x7f0aec07d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.153+0000 7f0af7fff640 1 -- 172.21.15.31:0/2304443687 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0ae80bee90 con 0x7f0b00102fb0 2024-09-16T08:58:55.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.153+0000 7f0afed9d640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f0aec07ace0 0x7f0aec07d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:55.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.153+0000 7f0afed9d640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f0aec07ace0 0x7f0aec07d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0af0005fd0 tx=0x7f0af000b390 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:55.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.159+0000 7f0af7fff640 1 -- 172.21.15.31:0/2304443687 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f0ae8082350 con 0x7f0b00102fb0 2024-09-16T08:58:55.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:55 smithi031 ceph-mon[27844]: from='client.24585 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:55.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:55 smithi097 ceph-mon[35478]: from='client.24585 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:55.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.522+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0b00108800 con 0x7f0b00102fb0 2024-09-16T08:58:55.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.522+0000 7f0af7fff640 1 -- 172.21.15.31:0/2304443687 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+731 (secure 0 0 0) 0x7f0ae8085470 con 0x7f0b00102fb0 2024-09-16T08:58:55.523 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:58:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:58:55.523 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:58:55.524 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 12, 2024-09-16T08:58:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T08:58:55.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:58:55.525 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:58:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f0aec07ace0 msgr2=0x7f0aec07d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f0aec07ace0 0x7f0aec07d1a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0af0005fd0 tx=0x7f0af000b390 comp rx=0 tx=0).stop 2024-09-16T08:58:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 msgr2=0x7f0b000fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b000fe630 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0ae802a9f0 tx=0x7f0ae80055a0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 shutdown_connections 2024-09-16T08:58:55.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f0aec07ace0 0x7f0aec07d1a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b00102fb0 0x7f0b000fe630 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 --2- 172.21.15.31:0/2304443687 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b00100680 0x7f0b000fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 >> 172.21.15.31:0/2304443687 conn(0x7f0b000fa280 msgr2=0x7f0b000fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 shutdown_connections 2024-09-16T08:58:55.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.525+0000 7f0b050db640 1 -- 172.21.15.31:0/2304443687 wait complete. 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.619+0000 7ff5e7dda640 1 -- 172.21.15.31:0/1819036783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 msgr2=0x7ff5e01053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.619+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/1819036783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e01053a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00098b0 tx=0x7ff5d002a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.619+0000 7ff5e7dda640 1 -- 172.21.15.31:0/1819036783 shutdown_connections 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.619+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/1819036783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e01053a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.619+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/1819036783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0100680 0x7ff5e0102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.619+0000 7ff5e7dda640 1 -- 172.21.15.31:0/1819036783 >> 172.21.15.31:0/1819036783 conn(0x7ff5e00fa280 msgr2=0x7ff5e00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.620+0000 7ff5e7dda640 1 -- 172.21.15.31:0/1819036783 shutdown_connections 2024-09-16T08:58:55.621 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.620+0000 7ff5e7dda640 1 -- 172.21.15.31:0/1819036783 wait complete. 2024-09-16T08:58:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.620+0000 7ff5e7dda640 1 Processor -- start 2024-09-16T08:58:55.622 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e7dda640 1 -- start start 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e7dda640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0100680 0x7ff5e01956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e7dda640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e0195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e7dda640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0196200 con 0x7ff5e0102fb0 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e7dda640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e0196370 con 0x7ff5e0100680 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5375640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e0195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5375640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e0195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60054/0 (socket says 172.21.15.31:60054) 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5375640 1 -- 172.21.15.31:0/459764414 learned_addr learned my addr 172.21.15.31:0/459764414 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5b76640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0100680 0x7ff5e01956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5375640 1 -- 172.21.15.31:0/459764414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0100680 msgr2=0x7ff5e01956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5375640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0100680 0x7ff5e01956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:55.623 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5375640 1 -- 172.21.15.31:0/459764414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0009560 con 0x7ff5e0102fb0 2024-09-16T08:58:55.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.621+0000 7ff5e5b76640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0100680 0x7ff5e01956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:58:55.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.622+0000 7ff5e5375640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e0195c30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0004650 tx=0x7ff5d00055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:55.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.622+0000 7ff5d6ffd640 1 -- 172.21.15.31:0/459764414 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d0004aa0 con 0x7ff5e0102fb0 2024-09-16T08:58:55.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.622+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e019adb0 con 0x7ff5e0102fb0 2024-09-16T08:58:55.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.622+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e019b2a0 con 0x7ff5e0102fb0 2024-09-16T08:58:55.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.622+0000 7ff5d6ffd640 1 -- 172.21.15.31:0/459764414 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5d0005d50 con 0x7ff5e0102fb0 2024-09-16T08:58:55.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.622+0000 7ff5d6ffd640 1 -- 172.21.15.31:0/459764414 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5d0003960 con 0x7ff5e0102fb0 2024-09-16T08:58:55.625 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.623+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c4005280 con 0x7ff5e0102fb0 2024-09-16T08:58:55.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.625+0000 7ff5d6ffd640 1 -- 172.21.15.31:0/459764414 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 103129+0+0 (secure 0 0 0) 0x7ff5d0004c00 con 0x7ff5e0102fb0 2024-09-16T08:58:55.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.626+0000 7ff5d6ffd640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7ff5cc07ace0 0x7ff5cc07d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:55.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.626+0000 7ff5d6ffd640 1 -- 172.21.15.31:0/459764414 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff5d00befc0 con 0x7ff5e0102fb0 2024-09-16T08:58:55.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.626+0000 7ff5e5b76640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7ff5cc07ace0 0x7ff5cc07d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:55.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.627+0000 7ff5e5b76640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7ff5cc07ace0 0x7ff5cc07d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009fc0 tx=0x7ff5dc005f90 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:55.633 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:55.633+0000 7ff5d6ffd640 1 -- 172.21.15.31:0/459764414 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7ff5d00824c0 con 0x7ff5e0102fb0 2024-09-16T08:58:55.804 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-16T08:58:55.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T08:58:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.041+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff5c4005800 con 0x7ff5e0102fb0 2024-09-16T08:58:56.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.042+0000 7ff5d6ffd640 1 -- 172.21.15.31:0/459764414 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7ff5d00855e0 con 0x7ff5e0102fb0 2024-09-16T08:58:56.043 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:58:56.043 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:58:56.043 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:58:56.043 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:58:56.044 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:58:56.045 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:56.046 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:58:56.047 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:58:56.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7ff5cc07ace0 msgr2=0x7ff5cc07d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7ff5cc07ace0 0x7ff5cc07d1a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff5dc009fc0 tx=0x7ff5dc005f90 comp rx=0 tx=0).stop 2024-09-16T08:58:56.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 msgr2=0x7ff5e0195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e0195c30 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0004650 tx=0x7ff5d00055a0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 shutdown_connections 2024-09-16T08:58:56.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7ff5cc07ace0 0x7ff5cc07d1a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5e0102fb0 0x7ff5e0195c30 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 --2- 172.21.15.31:0/459764414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5e0100680 0x7ff5e01956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 >> 172.21.15.31:0/459764414 conn(0x7ff5e00fa280 msgr2=0x7ff5e00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:56.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 shutdown_connections 2024-09-16T08:58:56.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.045+0000 7ff5e7dda640 1 -- 172.21.15.31:0/459764414 wait complete. 2024-09-16T08:58:56.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:56 smithi031 ceph-mon[27844]: pgmap v16: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 621 op/s 2024-09-16T08:58:56.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:56 smithi031 ceph-mon[27844]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:56.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:56 smithi031 ceph-mon[27844]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:56.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:56 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/2304443687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:58:56.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:56 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/459764414' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:58:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 -- 172.21.15.31:0/2942148739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 msgr2=0x7fb9d4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 --2- 172.21.15.31:0/2942148739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d4101210 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4009880 tx=0x7fb9c402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 -- 172.21.15.31:0/2942148739 shutdown_connections 2024-09-16T08:58:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 --2- 172.21.15.31:0/2942148739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d4101210 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 --2- 172.21.15.31:0/2942148739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40ffb90 0x7fb9d40fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 -- 172.21.15.31:0/2942148739 >> 172.21.15.31:0/2942148739 conn(0x7fb9d40fb360 msgr2=0x7fb9d40fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 -- 172.21.15.31:0/2942148739 shutdown_connections 2024-09-16T08:58:56.133 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.131+0000 7fb9db615640 1 -- 172.21.15.31:0/2942148739 wait complete. 2024-09-16T08:58:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9db615640 1 Processor -- start 2024-09-16T08:58:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9db615640 1 -- start start 2024-09-16T08:58:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9db615640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40ffb90 0x7fb9d4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9db615640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9db615640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d4072de0 con 0x7fb9d4100d90 2024-09-16T08:58:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9db615640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d4072f50 con 0x7fb9d40ffb90 2024-09-16T08:58:56.134 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9d8bb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9d8bb0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d40728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60070/0 (socket says 172.21.15.31:60070) 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9d8bb0640 1 -- 172.21.15.31:0/2838441888 learned_addr learned my addr 172.21.15.31:0/2838441888 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.132+0000 7fb9d93b1640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40ffb90 0x7fb9d4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9d8bb0640 1 -- 172.21.15.31:0/2838441888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40ffb90 msgr2=0x7fb9d4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9d8bb0640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40ffb90 0x7fb9d4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9d8bb0640 1 -- 172.21.15.31:0/2838441888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4009560 con 0x7fb9d4100d90 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9d8bb0640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d40728a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40054f0 tx=0x7fb9c4005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9ca7fc640 1 -- 172.21.15.31:0/2838441888 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c4004b60 con 0x7fb9d4100d90 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9ca7fc640 1 -- 172.21.15.31:0/2838441888 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9c4004cc0 con 0x7fb9d4100d90 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d40731d0 con 0x7fb9d4100d90 2024-09-16T08:58:56.135 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9ca7fc640 1 -- 172.21.15.31:0/2838441888 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9c4003b50 con 0x7fb9d4100d90 2024-09-16T08:58:56.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.133+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d40736c0 con 0x7fb9d4100d90 2024-09-16T08:58:56.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.134+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d40fff90 con 0x7fb9d4100d90 2024-09-16T08:58:56.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.135+0000 7fb9ca7fc640 1 -- 172.21.15.31:0/2838441888 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 103129+0+0 (secure 0 0 0) 0x7fb9c4003cb0 con 0x7fb9d4100d90 2024-09-16T08:58:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.136+0000 7fb9ca7fc640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fb9c007ace0 0x7fb9c007d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.136+0000 7fb9ca7fc640 1 -- 172.21.15.31:0/2838441888 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb9c40bef20 con 0x7fb9d4100d90 2024-09-16T08:58:56.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.136+0000 7fb9d93b1640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fb9c007ace0 0x7fb9c007d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:56.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.137+0000 7fb9d93b1640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fb9c007ace0 0x7fb9c007d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d000b410 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:56.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.141+0000 7fb9ca7fc640 1 -- 172.21.15.31:0/2838441888 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7fb9c40823a0 con 0x7fb9d4100d90 2024-09-16T08:58:56.281 DEBUG:teuthology.parallel:result is None 2024-09-16T08:58:56.281 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T08:58:56.314 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T08:58:56.314 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-16T08:58:56.337 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:56 smithi097 ceph-mon[35478]: pgmap v16: 65 pgs: 65 active+clean; 237 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 1.7 MiB/s wr, 621 op/s 2024-09-16T08:58:56.337 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:56 smithi097 ceph-mon[35478]: from='client.24589 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:56.337 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:56 smithi097 ceph-mon[35478]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:56.337 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:56 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2304443687' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:58:56.337 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:56 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/459764414' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:58:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.342+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 --> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9d402c450 con 0x7fb9c007ace0 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.343+0000 7fb9ca7fc640 1 -- 172.21.15.31:0/2838441888 <== mgr.14772 v2:172.21.15.31:6800/2307887580 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb9d402c450 con 0x7fb9c007ace0 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T08:58:56.344 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T08:58:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/23 daemons upgraded", 2024-09-16T08:58:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-09-16T08:58:56.345 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:58:56.345 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:58:56.346 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-16T08:58:56.346 DEBUG:teuthology.parallel:result is None 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fb9c007ace0 msgr2=0x7fb9c007d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fb9c007ace0 0x7fb9c007d1a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0005fd0 tx=0x7fb9d000b410 comp rx=0 tx=0).stop 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 msgr2=0x7fb9d40728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d40728a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40054f0 tx=0x7fb9c4005520 comp rx=0 tx=0).stop 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 shutdown_connections 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7fb9c007ace0 0x7fb9c007d1a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9d4100d90 0x7fb9d40728a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 --2- 172.21.15.31:0/2838441888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9d40ffb90 0x7fb9d4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.345+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 >> 172.21.15.31:0/2838441888 conn(0x7fb9d40fb360 msgr2=0x7fb9d4104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:56.347 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.346+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 shutdown_connections 2024-09-16T08:58:56.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.346+0000 7fb9db615640 1 -- 172.21.15.31:0/2838441888 wait complete. 2024-09-16T08:58:56.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 -- 172.21.15.31:0/3164759741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170102d60 msgr2=0x7f3170105150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 --2- 172.21.15.31:0/3164759741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170102d60 0x7f3170105150 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f31600098b0 tx=0x7f316002a4d0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 -- 172.21.15.31:0/3164759741 shutdown_connections 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 --2- 172.21.15.31:0/3164759741 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170102d60 0x7f3170105150 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 --2- 172.21.15.31:0/3164759741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3170100430 0x7f3170102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 -- 172.21.15.31:0/3164759741 >> 172.21.15.31:0/3164759741 conn(0x7f31700fa010 msgr2=0x7f31700fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 -- 172.21.15.31:0/3164759741 shutdown_connections 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.429+0000 7f31773a1640 1 -- 172.21.15.31:0/3164759741 wait complete. 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f31773a1640 1 Processor -- start 2024-09-16T08:58:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f31773a1640 1 -- start start 2024-09-16T08:58:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f31773a1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170100430 0x7f3170195480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f31773a1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3170102d60 0x7f31701959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:56.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f31773a1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3170195f90 con 0x7f3170100430 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f31773a1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3170196100 con 0x7f3170102d60 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f317639f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170100430 0x7f3170195480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f317639f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170100430 0x7f3170195480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60100/0 (socket says 172.21.15.31:60100) 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f317639f640 1 -- 172.21.15.31:0/503677207 learned_addr learned my addr 172.21.15.31:0/503677207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f3175b9e640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3170102d60 0x7f31701959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f317639f640 1 -- 172.21.15.31:0/503677207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3170102d60 msgr2=0x7f31701959c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f317639f640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3170102d60 0x7f31701959c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f317639f640 1 -- 172.21.15.31:0/503677207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3160009560 con 0x7f3170100430 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.430+0000 7f3175b9e640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3170102d60 0x7f31701959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.431+0000 7f317639f640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170100430 0x7f3170195480 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c840 tx=0x7f316c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.431+0000 7f31677fe640 1 -- 172.21.15.31:0/503677207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f316c0041f0 con 0x7f3170100430 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.431+0000 7f31677fe640 1 -- 172.21.15.31:0/503677207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f316c007e70 con 0x7f3170100430 2024-09-16T08:58:56.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.431+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317019aba0 con 0x7f3170100430 2024-09-16T08:58:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.431+0000 7f31677fe640 1 -- 172.21.15.31:0/503677207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f316c0049d0 con 0x7f3170100430 2024-09-16T08:58:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.431+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317019b0f0 con 0x7f3170100430 2024-09-16T08:58:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.432+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3170108590 con 0x7f3170100430 2024-09-16T08:58:56.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.433+0000 7f31677fe640 1 -- 172.21.15.31:0/503677207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 31) v1 ==== 103129+0+0 (secure 0 0 0) 0x7f316c0079b0 con 0x7f3170100430 2024-09-16T08:58:56.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.433+0000 7f31677fe640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f315c07ace0 0x7f315c07d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:58:56.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.433+0000 7f31677fe640 1 -- 172.21.15.31:0/503677207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f316c09da90 con 0x7f3170100430 2024-09-16T08:58:56.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.434+0000 7f3175b9e640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f315c07ace0 0x7f315c07d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:58:56.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.434+0000 7f3175b9e640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f315c07ace0 0x7f315c07d1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f316002a9f0 tx=0x7f3160005de0 comp rx=0 tx=0).ready entity=mgr.14772 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:58:56.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.438+0000 7f31677fe640 1 -- 172.21.15.31:0/503677207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+194376 (secure 0 0 0) 0x7f316c060f00 con 0x7f3170100430 2024-09-16T08:58:56.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.697+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f317019b6b0 con 0x7f3170100430 2024-09-16T08:58:56.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.698+0000 7f31677fe640 1 -- 172.21.15.31:0/503677207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f316c064020 con 0x7f3170100430 2024-09-16T08:58:56.699 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f315c07ace0 msgr2=0x7f315c07d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f315c07ace0 0x7f315c07d1a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f316002a9f0 tx=0x7f3160005de0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170100430 msgr2=0x7f3170195480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170100430 0x7f3170195480 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f316c00c840 tx=0x7f316c00cd00 comp rx=0 tx=0).stop 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 shutdown_connections 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:6800/2307887580,v1:172.21.15.31:6801/2307887580] conn(0x7f315c07ace0 0x7f315c07d1a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3170102d60 0x7f31701959c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 --2- 172.21.15.31:0/503677207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3170100430 0x7f3170195480 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 >> 172.21.15.31:0/503677207 conn(0x7f31700fa010 msgr2=0x7f31700fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:58:56.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 shutdown_connections 2024-09-16T08:58:56.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:58:56.700+0000 7f31773a1640 1 -- 172.21.15.31:0/503677207 wait complete. 2024-09-16T08:58:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:57 smithi031 ceph-mon[27844]: from='client.? 172.21.15.31:0/503677207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:58:57.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:57 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/503677207' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-16T08:58:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:58 smithi031 ceph-mon[27844]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:58:58 smithi031 ceph-mon[27844]: pgmap v17: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.2 MiB/s wr, 721 op/s 2024-09-16T08:58:58.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:58 smithi097 ceph-mon[35478]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:58:58.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:58:58 smithi097 ceph-mon[35478]: pgmap v17: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 2.2 MiB/s wr, 721 op/s 2024-09-16T08:59:00.248 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:00 smithi097 ceph-mon[35478]: pgmap v18: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 422 op/s 2024-09-16T08:59:00.248 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:00 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:00.248 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:00 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:00.249 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:00 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:59:00.249 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:00 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:59:00.249 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:00 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:59:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:00 smithi031 ceph-mon[27844]: pgmap v18: 65 pgs: 65 active+clean; 222 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 1.5 MiB/s wr, 422 op/s 2024-09-16T08:59:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:00 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:00 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:00 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:59:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:00 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:59:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:00 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:59:01.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:01 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:59:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:01 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:59:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:01 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:59:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:01 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:59:01.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:01 smithi031 ceph-mon[27844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T08:59:01.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:01 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T08:59:01.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:01 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T08:59:01.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:01 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:59:02.368 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:02 smithi097 ceph-mon[35478]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 498 op/s 2024-09-16T08:59:02.369 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:02 smithi097 ceph-mon[35478]: Upgrade: Updating mgr.smithi097.mpashf 2024-09-16T08:59:02.369 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:02 smithi097 ceph-mon[35478]: Deploying daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:59:02.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:02 smithi031 ceph-mon[27844]: pgmap v19: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 36 KiB/s rd, 1.7 MiB/s wr, 498 op/s 2024-09-16T08:59:02.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:02 smithi031 ceph-mon[27844]: Upgrade: Updating mgr.smithi097.mpashf 2024-09-16T08:59:02.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:02 smithi031 ceph-mon[27844]: Deploying daemon mgr.smithi097.mpashf on smithi097 2024-09-16T08:59:04.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:04 smithi097 ceph-mon[35478]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 968 KiB/s wr, 278 op/s 2024-09-16T08:59:04.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:04 smithi031 ceph-mon[27844]: pgmap v20: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 968 KiB/s wr, 278 op/s 2024-09-16T08:59:05.849 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:05 smithi031 ceph-mon[27844]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 968 KiB/s wr, 279 op/s 2024-09-16T08:59:05.849 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:05 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:05.849 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:05 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:05.849 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:05 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:59:05.946 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:05 smithi097 ceph-mon[35478]: pgmap v21: 65 pgs: 65 active+clean; 205 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 968 KiB/s wr, 279 op/s 2024-09-16T08:59:05.946 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:05 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:05.946 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:05 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:05.946 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:05 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:59:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:07 smithi031 ceph-mon[27844]: pgmap v22: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 698 KiB/s wr, 239 op/s 2024-09-16T08:59:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:07 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:07 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:08.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:07 smithi097 ceph-mon[35478]: pgmap v22: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 698 KiB/s wr, 239 op/s 2024-09-16T08:59:08.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:07 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:08.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:07 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:09 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:09 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:09.409 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:09 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:09.409 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:09 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: pgmap v23: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8.2 KiB/s rd, 219 KiB/s wr, 81 op/s 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf restarted 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:59:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:10 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:59:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: pgmap v23: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8.2 KiB/s rd, 219 KiB/s wr, 81 op/s 2024-09-16T08:59:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: Standby manager daemon smithi097.mpashf restarted 2024-09-16T08:59:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:59:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:59:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:59:10.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:59:10.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:10 smithi031 ceph-mon[27844]: from='mgr.? 172.21.15.97:0/920479976' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:59:11.810 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: pgmap v24: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8.2 KiB/s rd, 219 KiB/s wr, 81 op/s 2024-09-16T08:59:11.810 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: mgrmap e32: smithi031.dcmqpv(active, since 42s), standbys: smithi097.mpashf 2024-09-16T08:59:11.810 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:11.810 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.dcmqpv"}]: dispatch 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.dcmqpv"}]': finished 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi097.mpashf"}]: dispatch 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi097.mpashf"}]': finished 2024-09-16T08:59:11.811 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:11 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T08:59:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: pgmap v24: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8.2 KiB/s rd, 219 KiB/s wr, 81 op/s 2024-09-16T08:59:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: mgrmap e32: smithi031.dcmqpv(active, since 42s), standbys: smithi097.mpashf 2024-09-16T08:59:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:59:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T08:59:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "versions"}]: dispatch 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.dcmqpv"}]: dispatch 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi031.dcmqpv"}]': finished 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi097.mpashf"}]: dispatch 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi097.mpashf"}]': finished 2024-09-16T08:59:12.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:11 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-16T08:59:12.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:12 smithi031 ceph-mon[27844]: Upgrade: Setting container_image for all mgr 2024-09-16T08:59:13.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:12 smithi097 ceph-mon[35478]: Upgrade: Setting container_image for all mgr 2024-09-16T08:59:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:13 smithi097 ceph-mon[35478]: pgmap v25: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-16T08:59:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:13 smithi097 ceph-mon[35478]: Upgrade: Updating mon.smithi031 2024-09-16T08:59:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:13 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:13 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:59:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:13 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:59:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:13 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:59:14.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:13 smithi097 ceph-mon[35478]: Deploying daemon mon.smithi031 on smithi031 2024-09-16T08:59:14.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:13 smithi031 ceph-mon[27844]: pgmap v25: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-16T08:59:14.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:13 smithi031 ceph-mon[27844]: Upgrade: Updating mon.smithi031 2024-09-16T08:59:14.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:13 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:14.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:13 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T08:59:14.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:13 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T08:59:14.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:13 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T08:59:14.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:13 smithi031 ceph-mon[27844]: Deploying daemon mon.smithi031 on smithi031 2024-09-16T08:59:14.714 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:14 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:59:14.992 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:14 smithi031 ceph-mon[27844]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:59:14.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:14 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[27840]: 2024-09-16T08:59:14.723+0000 7f11f43e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:59:14.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:14 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[27840]: 2024-09-16T08:59:14.723+0000 7f11f43e0640 -1 mon.smithi031@0(leader) e2 *** Got Signal Terminated *** 2024-09-16T08:59:14.993 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:14 smithi031 podman[91710]: 2024-09-16 08:59:14.868329706 +0000 UTC m=+0.258464434 container died 31533db037fd5c9ce47a7df4f0d299b33538bf4d078523f2824b3c5d8a917e52 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T08:59:15.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:14 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-16T08:59:15.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:15 smithi031 podman[91710]: 2024-09-16 08:59:15.333138003 +0000 UTC m=+0.723272716 container cleanup 31533db037fd5c9ce47a7df4f0d299b33538bf4d078523f2824b3c5d8a917e52 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T08:59:15.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:15 smithi031 bash[91710]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031 2024-09-16T08:59:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:15 smithi031 podman[91723]: 2024-09-16 08:59:15.593046515 +0000 UTC m=+0.722405006 container remove 31533db037fd5c9ce47a7df4f0d299b33538bf4d078523f2824b3c5d8a917e52 (image=quay.ceph.io/ceph-ci/ceph:reef, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T08:59:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:15 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service: Deactivated successfully. 2024-09-16T08:59:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:15 smithi031 systemd[1]: Stopped Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:59:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:15 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service: Consumed 12.878s CPU time. 2024-09-16T08:59:16.244 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:15 smithi031 systemd[1]: Starting Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:59:16.245 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 podman[91865]: 2024-09-16 08:59:16.101986585 +0000 UTC m=+0.016463227 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:59:16.245 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 podman[91865]: 2024-09-16 08:59:16.244213485 +0000 UTC m=+0.158690190 container create 13ee87bf1609afec3a2588ebde503d047eac9fd7d2a3f8d233d1b5b44066f924 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 podman[91865]: 2024-09-16 08:59:16.521717105 +0000 UTC m=+0.436193736 container init 13ee87bf1609afec3a2588ebde503d047eac9fd7d2a3f8d233d1b5b44066f924 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 podman[91865]: 2024-09-16 08:59:16.525312576 +0000 UTC m=+0.439789206 container start 13ee87bf1609afec3a2588ebde503d047eac9fd7d2a3f8d233d1b5b44066f924 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 2 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: pidfile_write: ignore empty --pid-file 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: load: jerasure load: lrc 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Git sha 0 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: DB SUMMARY 2024-09-16T08:59:16.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: DB Session ID: DFYQCGD0ABSFY8CXPD6T 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1518 Bytes 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi031/store.db dir, Total Num: 1, files: 000032.sst 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi031/store.db: 000030.log size: 618926 ; 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:59:16.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.env: 0x55ea5f9b1120 2024-09-16T08:59:16.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:59:16.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.info_log: 0x55ea60e578e0 2024-09-16T08:59:16.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:59:16.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.statistics: (nil) 2024-09-16T08:59:16.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.use_fsync: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:59:16.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.db_log_dir: 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.wal_dir: 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:59:16.879 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.write_buffer_manager: 0x55ea60e5bb80 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.unordered_write: 0 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:59:16.880 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.row_cache: None 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.wal_filter: None 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.wal_compression: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:59:16.881 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:59:16.882 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_open_files: -1 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Compression algorithms supported: 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kZSTD supported: 0 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:59:16.883 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000015 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.merge_operator: 2024-09-16T08:59:16.884 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_filter: None 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ea60e57540) 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_type: 0 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_index_type: 0 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_shortening: 1 2024-09-16T08:59:16.885 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: checksum: 4 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: no_block_cache: 0 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache: 0x55ea60e4b610 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_options: 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: capacity : 536870912 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_shard_bits : 4 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: strict_capacity_limit : 0 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_cache_compressed: (nil) 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: persistent_cache: (nil) 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size: 4096 2024-09-16T08:59:16.886 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_size_deviation: 10 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_restart_interval: 16 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: index_block_restart_interval: 1 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_block_size: 4096 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: partition_filters: 0 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: use_delta_encoding: 1 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: filter_policy: bloomfilter 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: whole_key_filtering: 1 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: verify_compression: 0 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: format_version: 5 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_index_compression: 1 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: block_align: 0 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_auto_readahead_size: 262144 2024-09-16T08:59:16.887 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: prepopulate_block_cache: 0 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression: NoCompression 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.num_levels: 7 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:59:16.888 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:59:16.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:59:16.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:59:16.891 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:59:16.892 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.ttl: 2592000 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:59:16.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi031/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 11996, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3312a79e-384d-4a0c-9aaf-cd5b2e0c684c 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726477156553060, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726477156556115, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 579467, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11997, "largest_seqno": 12315, "table_properties": {"data_size": 576884, "index_size": 1095, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 453, "raw_key_size": 4180, "raw_average_key_size": 25, "raw_value_size": 573036, "raw_average_value_size": 3515, "num_data_blocks": 48, "num_entries": 163, "num_filter_entries": 163, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726477156, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3312a79e-384d-4a0c-9aaf-cd5b2e0c684c", "db_session_id": "DFYQCGD0ABSFY8CXPD6T", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:59:16.894 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726477156556226, "job": 1, "event": "recovery_finished"} 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi031/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ea60e7ae00 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: DB pointer 0x55ea60f7c000 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** DB Stats ** 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:59:16.895 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L0 1/0 565.89 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 286.5 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: L6 1/0 6.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Sum 2/0 7.35 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 286.5 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 286.5 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.896 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** Compaction Stats [default] ** 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 286.5 0.00 0.00 1 0.002 0 0 0.0 0.0 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Flush(GB): cumulative 0.001, interval 0.001 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Cumulative compaction: 0.00 GB write, 94.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:59:16.897 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Interval compaction: 0.00 GB write, 94.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache BinnedLRUCache@0x55ea60e4b610#2 capacity: 512.00 MB usage: 1.70 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.50 KB,9.53674e-05%) IndexBlock(1,1.20 KB,0.000229478%) Misc(1,0.00 KB,0%) 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: starting mon.smithi031 rank 0 at public addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] at bind addrs [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi031 fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???) e2 preinit fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???).mds e17 new map 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???).mds e17 print_map 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: e17 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:59:16.898 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: legacy client fscid: 1 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Filesystem 'cephfs' (1) 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: fs_name cephfs 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: epoch 17 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: tableserver 0 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: root 0 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: session_timeout 60 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: session_autoclose 300 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_file_size 1099511627776 2024-09-16T08:59:16.899 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_xattr_size 65536 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: required_client_features {} 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: last_failure 0 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: last_failure_osd_epoch 47 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: max_mds 2 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: in 0,1 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: up {0=14548,1=24325} 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: failed 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: damaged 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: stopped 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: data_pools [3] 2024-09-16T08:59:16.900 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: metadata_pool 2 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: inline_data disabled 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: balancer 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: bal_rank_mask -1 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: standby_count_wanted 1 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: qdb_cluster leader: 0 members: 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Standby daemons: 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:16.901 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: [mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:16.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-16T08:59:16.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T08:59:16.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T08:59:16.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T08:59:16.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 ceph-mon[91884]: mon.smithi031@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-09-16T08:59:16.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 bash[91865]: 13ee87bf1609afec3a2588ebde503d047eac9fd7d2a3f8d233d1b5b44066f924 2024-09-16T08:59:16.902 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:16 smithi031 systemd[1]: Started Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:59:18.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:59:18.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:59:18.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: pgmap v27: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 4.7 KiB/s wr, 5 op/s 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: mon.smithi031 calling monitor election 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: mon.smithi031 is new leader, mons smithi031,smithi097 in quorum (ranks 0,1) 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: monmap epoch 2 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: last_changed 2024-09-16T08:49:34.544973+0000 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: created 2024-09-16T08:47:00.018794+0000 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: min_mon_release 18 (reef) 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: election_strategy: 1 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.smithi097 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:59:18.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T08:59:18.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: mgrmap e32: smithi031.dcmqpv(active, since 49s), standbys: smithi097.mpashf 2024-09-16T08:59:18.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: overall HEALTH_OK 2024-09-16T08:59:18.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: from='mgr.14772 ' entity='' 2024-09-16T08:59:18.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:18 smithi031 ceph-mon[91884]: mgrmap e33: smithi031.dcmqpv(active, since 49s), standbys: smithi097.mpashf 2024-09-16T08:59:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi031"}]: dispatch 2024-09-16T08:59:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "mon metadata", "id": "smithi097"}]: dispatch 2024-09-16T08:59:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: from='mgr.14772 172.21.15.31:0/2085046106' entity='mgr.smithi031.dcmqpv' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:59:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:59:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: pgmap v27: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 682 B/s rd, 4.7 KiB/s wr, 5 op/s 2024-09-16T08:59:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: mon.smithi031 calling monitor election 2024-09-16T08:59:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: mon.smithi031 is new leader, mons smithi031,smithi097 in quorum (ranks 0,1) 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: monmap epoch 2 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: last_changed 2024-09-16T08:49:34.544973+0000 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: created 2024-09-16T08:47:00.018794+0000 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: min_mon_release 18 (reef) 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: election_strategy: 1 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.smithi097 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: osdmap e49: 6 total, 6 up, 6 in 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: mgrmap e32: smithi031.dcmqpv(active, since 49s), standbys: smithi097.mpashf 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: overall HEALTH_OK 2024-09-16T08:59:18.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: from='mgr.14772 ' entity='' 2024-09-16T08:59:18.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:18 smithi097 ceph-mon[35478]: mgrmap e33: smithi031.dcmqpv(active, since 49s), standbys: smithi097.mpashf 2024-09-16T08:59:23.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:23 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf restarted 2024-09-16T08:59:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:23 smithi097 ceph-mon[35478]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:59:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:23 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:59:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:23 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:59:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:23 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:59:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:23 smithi097 ceph-mon[35478]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:59:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:23 smithi031 ceph-mon[91884]: Standby manager daemon smithi097.mpashf restarted 2024-09-16T08:59:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:23 smithi031 ceph-mon[91884]: Standby manager daemon smithi097.mpashf started 2024-09-16T08:59:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:23 smithi031 ceph-mon[91884]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/crt"}]: dispatch 2024-09-16T08:59:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:23 smithi031 ceph-mon[91884]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T08:59:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:23 smithi031 ceph-mon[91884]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi097.mpashf/key"}]: dispatch 2024-09-16T08:59:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:23 smithi031 ceph-mon[91884]: from='mgr.? 172.21.15.97:0/3349932383' entity='mgr.smithi097.mpashf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T08:59:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: mgrmap e34: smithi031.dcmqpv(active, since 54s), standbys: smithi097.mpashf 2024-09-16T08:59:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: Active manager daemon smithi031.dcmqpv restarted 2024-09-16T08:59:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:59:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: osdmap e50: 6 total, 6 up, 6 in 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: mgrmap e35: smithi031.dcmqpv(active, starting, since 0.00613503s), standbys: smithi097.mpashf 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi031"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi097"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:59:24.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:59:24.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:24 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"} : dispatch 2024-09-16T08:59:24.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: mgrmap e34: smithi031.dcmqpv(active, since 54s), standbys: smithi097.mpashf 2024-09-16T08:59:24.369 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: Active manager daemon smithi031.dcmqpv restarted 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: Activating manager daemon smithi031.dcmqpv 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: osdmap e50: 6 total, 6 up, 6 in 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: mgrmap e35: smithi031.dcmqpv(active, starting, since 0.00613503s), standbys: smithi097.mpashf 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi031"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi097"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr metadata", "who": "smithi031.dcmqpv", "id": "smithi031.dcmqpv"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr metadata", "who": "smithi097.mpashf", "id": "smithi097.mpashf"} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T08:59:24.370 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: Manager daemon smithi031.dcmqpv is now available 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/mirror_snapshot_schedule"} : dispatch 2024-09-16T08:59:24.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi031.dcmqpv/trash_purge_schedule"} : dispatch 2024-09-16T08:59:25.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:25 smithi097 ceph-mon[35478]: mgrmap e36: smithi031.dcmqpv(active, since 1.01159s), standbys: smithi097.mpashf 2024-09-16T08:59:25.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:25 smithi097 ceph-mon[35478]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:59:25.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:25 smithi097 ceph-mon[35478]: pgmap v3: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:25.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:25 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:25.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:25 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:25.598 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:25 smithi031 ceph-mon[91884]: mgrmap e36: smithi031.dcmqpv(active, since 1.01159s), standbys: smithi097.mpashf 2024-09-16T08:59:25.598 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:25 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T08:59:25.598 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:25 smithi031 ceph-mon[91884]: pgmap v3: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:25.598 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:25.598 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:26.591 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:26 smithi031 ceph-mon[91884]: mgrmap e37: smithi031.dcmqpv(active, since 2s), standbys: smithi097.mpashf 2024-09-16T08:59:26.591 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:26 smithi031 ceph-mon[91884]: [16/Sep/2024:08:59:25] ENGINE Bus STARTING 2024-09-16T08:59:26.591 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:26 smithi031 ceph-mon[91884]: [16/Sep/2024:08:59:25] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:59:26.591 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:26 smithi031 ceph-mon[91884]: [16/Sep/2024:08:59:25] ENGINE Error in HTTPServer.serve 2024-09-16T08:59:26.591 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: Traceback (most recent call last): 2024-09-16T08:59:26.591 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:59:26.591 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._run(expiration_interval) 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: s = self.context.wrap_socket( 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: return self.sslsocket_class._create( 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:59:26.592 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self.do_handshake() 2024-09-16T08:59:26.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:59:26.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: self._sslobj.do_handshake() 2024-09-16T08:59:26.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:59:26.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:26 smithi031 ceph-mon[91884]: [16/Sep/2024:08:59:25] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:59:26.593 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:26 smithi031 ceph-mon[91884]: [16/Sep/2024:08:59:25] ENGINE Bus STARTED 2024-09-16T08:59:26.738 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:26 smithi097 ceph-mon[35478]: mgrmap e37: smithi031.dcmqpv(active, since 2s), standbys: smithi097.mpashf 2024-09-16T08:59:26.738 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:26 smithi097 ceph-mon[35478]: [16/Sep/2024:08:59:25] ENGINE Bus STARTING 2024-09-16T08:59:26.738 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:26 smithi097 ceph-mon[35478]: [16/Sep/2024:08:59:25] ENGINE Serving on https://172.21.15.31:7150 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:26 smithi097 ceph-mon[35478]: [16/Sep/2024:08:59:25] ENGINE Error in HTTPServer.serve 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Traceback (most recent call last): 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._connections.run(self.expiration_interval) 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._run(expiration_interval) 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: s = self.context.wrap_socket( 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: return self.sslsocket_class._create( 2024-09-16T08:59:26.739 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T08:59:26.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self.do_handshake() 2024-09-16T08:59:26.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T08:59:26.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: self._sslobj.do_handshake() 2024-09-16T08:59:26.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T08:59:26.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:26 smithi097 ceph-mon[35478]: [16/Sep/2024:08:59:25] ENGINE Serving on http://172.21.15.31:8765 2024-09-16T08:59:26.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:26 smithi097 ceph-mon[35478]: [16/Sep/2024:08:59:25] ENGINE Bus STARTED 2024-09-16T08:59:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1133559585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 msgr2=0x7f7b18100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:26.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1133559585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b18100170 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009880 tx=0x7f7b0402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:59:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1133559585 shutdown_connections 2024-09-16T08:59:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1133559585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b18100170 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1133559585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b180feaf0 0x7f7b180feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1133559585 >> 172.21.15.31:0/1133559585 conn(0x7f7b180fa280 msgr2=0x7f7b180fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1133559585 shutdown_connections 2024-09-16T08:59:26.792 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.790+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1133559585 wait complete. 2024-09-16T08:59:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b1ee15640 1 Processor -- start 2024-09-16T08:59:26.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b1ee15640 1 -- start start 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b1ee15640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b180feaf0 0x7f7b18102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b1ee15640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b181010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b1ee15640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18101610 con 0x7f7b180ffcf0 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b1ee15640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b18101780 con 0x7f7b180feaf0 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b181010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b17fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b181010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42296/0 (socket says 172.21.15.31:42296) 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b1cbb1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b180feaf0 0x7f7b18102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.791+0000 7f7b17fff640 1 -- 172.21.15.31:0/1024350870 learned_addr learned my addr 172.21.15.31:0/1024350870 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b17fff640 1 -- 172.21.15.31:0/1024350870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b180feaf0 msgr2=0x7f7b18102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b17fff640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b180feaf0 0x7f7b18102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:26.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b17fff640 1 -- 172.21.15.31:0/1024350870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04009560 con 0x7f7b180ffcf0 2024-09-16T08:59:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b17fff640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b181010d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b04006010 tx=0x7f7b040055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b15ffb640 1 -- 172.21.15.31:0/1024350870 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b04004aa0 con 0x7f7b180ffcf0 2024-09-16T08:59:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b15ffb640 1 -- 172.21.15.31:0/1024350870 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b04005d50 con 0x7f7b180ffcf0 2024-09-16T08:59:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b18101a00 con 0x7f7b180ffcf0 2024-09-16T08:59:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b15ffb640 1 -- 172.21.15.31:0/1024350870 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b040039e0 con 0x7f7b180ffcf0 2024-09-16T08:59:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.792+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b18101ef0 con 0x7f7b180ffcf0 2024-09-16T08:59:26.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.794+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b18107700 con 0x7f7b180ffcf0 2024-09-16T08:59:26.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.795+0000 7f7b15ffb640 1 -- 172.21.15.31:0/1024350870 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 103212+0+0 (secure 0 0 0) 0x7f7b04003be0 con 0x7f7b180ffcf0 2024-09-16T08:59:26.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.795+0000 7f7b15ffb640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7b0807afd0 0x7f7b0807d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:26.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.795+0000 7f7b15ffb640 1 -- 172.21.15.31:0/1024350870 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7b040bf0f0 con 0x7f7b180ffcf0 2024-09-16T08:59:26.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.795+0000 7f7b1cbb1640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7b0807afd0 0x7f7b0807d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:26.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.796+0000 7f7b1cbb1640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7b0807afd0 0x7f7b0807d490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7b180ffb50 tx=0x7f7b0c005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:26.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:26.801+0000 7f7b15ffb640 1 -- 172.21.15.31:0/1024350870 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f7b04082210 con 0x7f7b180ffcf0 2024-09-16T08:59:27.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.012+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b180683d0 con 0x7f7b0807afd0 2024-09-16T08:59:27.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.014+0000 7f7b15ffb640 1 -- 172.21.15.31:0/1024350870 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f7b180683d0 con 0x7f7b0807afd0 2024-09-16T08:59:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7b0807afd0 msgr2=0x7f7b0807d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7b0807afd0 0x7f7b0807d490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7b180ffb50 tx=0x7f7b0c005f90 comp rx=0 tx=0).stop 2024-09-16T08:59:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 msgr2=0x7f7b181010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b181010d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7b04006010 tx=0x7f7b040055a0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 shutdown_connections 2024-09-16T08:59:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7b0807afd0 0x7f7b0807d490 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b180ffcf0 0x7f7b181010d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 --2- 172.21.15.31:0/1024350870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b180feaf0 0x7f7b18102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 >> 172.21.15.31:0/1024350870 conn(0x7f7b180fa280 msgr2=0x7f7b18067cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 shutdown_connections 2024-09-16T08:59:27.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.016+0000 7f7b1ee15640 1 -- 172.21.15.31:0/1024350870 wait complete. 2024-09-16T08:59:27.026 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:59:27.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2484397213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 msgr2=0x7f0f58102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2484397213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f58102820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f480098b0 tx=0x7f0f4802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2484397213 shutdown_connections 2024-09-16T08:59:27.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2484397213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58102d60 0x7f0f58105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2484397213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f58102820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2484397213 >> 172.21.15.31:0/2484397213 conn(0x7f0f580fa010 msgr2=0x7f0f580fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:27.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2484397213 shutdown_connections 2024-09-16T08:59:27.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.101+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2484397213 wait complete. 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f5d39d640 1 Processor -- start 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f5d39d640 1 -- start start 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f5d39d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f581954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f5d39d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58102d60 0x7f0f581959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f5d39d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58195fb0 con 0x7f0f58100430 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f5d39d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58196120 con 0x7f0f58102d60 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f581954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f57fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f581954a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42316/0 (socket says 172.21.15.31:42316) 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f57fff640 1 -- 172.21.15.31:0/2315195575 learned_addr learned my addr 172.21.15.31:0/2315195575 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f577fe640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58102d60 0x7f0f581959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f57fff640 1 -- 172.21.15.31:0/2315195575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58102d60 msgr2=0x7f0f581959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f57fff640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58102d60 0x7f0f581959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.102+0000 7f0f57fff640 1 -- 172.21.15.31:0/2315195575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48009560 con 0x7f0f58100430 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.103+0000 7f0f57fff640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f581954a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f480054f0 tx=0x7f0f48005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.103+0000 7f0f557fa640 1 -- 172.21.15.31:0/2315195575 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f48004aa0 con 0x7f0f58100430 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.103+0000 7f0f557fa640 1 -- 172.21.15.31:0/2315195575 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f48004c00 con 0x7f0f58100430 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.103+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5819ab60 con 0x7f0f58100430 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.103+0000 7f0f557fa640 1 -- 172.21.15.31:0/2315195575 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f48003ac0 con 0x7f0f58100430 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.103+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5819b050 con 0x7f0f58100430 2024-09-16T08:59:27.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.104+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f581085e0 con 0x7f0f58100430 2024-09-16T08:59:27.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.104+0000 7f0f557fa640 1 -- 172.21.15.31:0/2315195575 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 103212+0+0 (secure 0 0 0) 0x7f0f48003d30 con 0x7f0f58100430 2024-09-16T08:59:27.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.105+0000 7f0f557fa640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0f4007afd0 0x7f0f4007d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.105+0000 7f0f557fa640 1 -- 172.21.15.31:0/2315195575 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0f480bf260 con 0x7f0f58100430 2024-09-16T08:59:27.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.105+0000 7f0f577fe640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0f4007afd0 0x7f0f4007d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.106+0000 7f0f577fe640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0f4007afd0 0x7f0f4007d490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f581969c0 tx=0x7f0f4c00b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:27.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.109+0000 7f0f557fa640 1 -- 172.21.15.31:0/2315195575 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f0f48082400 con 0x7f0f58100430 2024-09-16T08:59:27.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.321+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f5819b2c0 con 0x7f0f4007afd0 2024-09-16T08:59:27.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.322+0000 7f0f557fa640 1 -- 172.21.15.31:0/2315195575 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f0f5819b2c0 con 0x7f0f4007afd0 2024-09-16T08:59:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0f4007afd0 msgr2=0x7f0f4007d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0f4007afd0 0x7f0f4007d490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0f581969c0 tx=0x7f0f4c00b390 comp rx=0 tx=0).stop 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 msgr2=0x7f0f581954a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f581954a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f480054f0 tx=0x7f0f48005520 comp rx=0 tx=0).stop 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 shutdown_connections 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0f4007afd0 0x7f0f4007d490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0f58102d60 0x7f0f581959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 --2- 172.21.15.31:0/2315195575 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f58100430 0x7f0f581954a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 >> 172.21.15.31:0/2315195575 conn(0x7f0f580fa010 msgr2=0x7f0f580fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 shutdown_connections 2024-09-16T08:59:27.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.324+0000 7f0f5d39d640 1 -- 172.21.15.31:0/2315195575 wait complete. 2024-09-16T08:59:27.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- 172.21.15.31:0/3092918638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 msgr2=0x7fcf60100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 --2- 172.21.15.31:0/3092918638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60100120 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c009880 tx=0x7fcf4c02a4a0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- 172.21.15.31:0/3092918638 shutdown_connections 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 --2- 172.21.15.31:0/3092918638 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60100120 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 --2- 172.21.15.31:0/3092918638 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf600feaa0 0x7fcf600feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- 172.21.15.31:0/3092918638 >> 172.21.15.31:0/3092918638 conn(0x7fcf600fa250 msgr2=0x7fcf600fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- 172.21.15.31:0/3092918638 shutdown_connections 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- 172.21.15.31:0/3092918638 wait complete. 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 Processor -- start 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- start start 2024-09-16T08:59:27.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf600feaa0 0x7fcf60102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf601015c0 con 0x7fcf600ffca0 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.412+0000 7fcf67064640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf60101730 con 0x7fcf600feaa0 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42336/0 (socket says 172.21.15.31:42336) 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5ffff640 1 -- 172.21.15.31:0/379675378 learned_addr learned my addr 172.21.15.31:0/379675378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf64e00640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf600feaa0 0x7fcf60102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5ffff640 1 -- 172.21.15.31:0/379675378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf600feaa0 msgr2=0x7fcf60102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5ffff640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf600feaa0 0x7fcf60102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5ffff640 1 -- 172.21.15.31:0/379675378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c009560 con 0x7fcf600ffca0 2024-09-16T08:59:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5ffff640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60101080 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0054f0 tx=0x7fcf4c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5dffb640 1 -- 172.21.15.31:0/379675378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c004aa0 con 0x7fcf600ffca0 2024-09-16T08:59:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5dffb640 1 -- 172.21.15.31:0/379675378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf4c004c00 con 0x7fcf600ffca0 2024-09-16T08:59:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf601019b0 con 0x7fcf600ffca0 2024-09-16T08:59:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.413+0000 7fcf5dffb640 1 -- 172.21.15.31:0/379675378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4c003ac0 con 0x7fcf600ffca0 2024-09-16T08:59:27.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.414+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6019d660 con 0x7fcf600ffca0 2024-09-16T08:59:27.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.414+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf600feea0 con 0x7fcf600ffca0 2024-09-16T08:59:27.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.415+0000 7fcf5dffb640 1 -- 172.21.15.31:0/379675378 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 103212+0+0 (secure 0 0 0) 0x7fcf4c003d30 con 0x7fcf600ffca0 2024-09-16T08:59:27.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.416+0000 7fcf5dffb640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcf5007afd0 0x7fcf5007d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.416+0000 7fcf5dffb640 1 -- 172.21.15.31:0/379675378 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcf4c0bf1e0 con 0x7fcf600ffca0 2024-09-16T08:59:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.416+0000 7fcf64e00640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcf5007afd0 0x7fcf5007d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.416+0000 7fcf64e00640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcf5007afd0 0x7fcf5007d490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf600ffb00 tx=0x7fcf5400b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:27.421 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.420+0000 7fcf5dffb640 1 -- 172.21.15.31:0/379675378 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fcf4c082380 con 0x7fcf600ffca0 2024-09-16T08:59:27.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.615+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf60068c10 con 0x7fcf5007afd0 2024-09-16T08:59:27.617 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:27 smithi031 ceph-mon[91884]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:27.617 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:27 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:27.617 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:27 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:27.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.626+0000 7fcf5dffb640 1 -- 172.21.15.31:0/379675378 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fcf60068c10 con 0x7fcf5007afd0 2024-09-16T08:59:27.626 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (8m) 18s ago 10m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (111s) 18s ago 11m 7629k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (100s) 0s ago 10m 7512k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (11m) 18s ago 11m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (10m) 0s ago 10m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (8m) 18s ago 10m 87.2M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (4m) 18s ago 4m 14.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (4m) 18s ago 4m 156M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (4m) 0s ago 4m 26.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (4m) 0s ago 4m 16.7M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (94s) 18s ago 12m 509M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (23s) 0s ago 9m 451M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T08:59:27.627 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (12m) 18s ago 12m 57.3M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 31533db037fd 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (9m) 0s ago 9m 47.6M 2048M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a26ddb2c0523 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (11m) 18s ago 11m 15.7M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (10m) 0s ago 10m 16.3M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 18s ago 8m 247M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (7m) 18s ago 7m 263M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (7m) 18s ago 7m 222M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (7m) 0s ago 7m 278M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (6m) 0s ago 6m 248M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (6m) 0s ago 6m 232M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:59:27.628 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (28s) 18s ago 10m 39.5M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T08:59:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcf5007afd0 msgr2=0x7fcf5007d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcf5007afd0 0x7fcf5007d490 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf600ffb00 tx=0x7fcf5400b390 comp rx=0 tx=0).stop 2024-09-16T08:59:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 msgr2=0x7fcf60101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60101080 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0054f0 tx=0x7fcf4c005520 comp rx=0 tx=0).stop 2024-09-16T08:59:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 shutdown_connections 2024-09-16T08:59:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcf5007afd0 0x7fcf5007d490 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf600ffca0 0x7fcf60101080 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 --2- 172.21.15.31:0/379675378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf600feaa0 0x7fcf60102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 >> 172.21.15.31:0/379675378 conn(0x7fcf600fa250 msgr2=0x7fcf60068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:27.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 shutdown_connections 2024-09-16T08:59:27.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.628+0000 7fcf67064640 1 -- 172.21.15.31:0/379675378 wait complete. 2024-09-16T08:59:27.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 -- 172.21.15.31:0/3348951728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54100da0 msgr2=0x7f6c54101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 --2- 172.21.15.31:0/3348951728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54100da0 0x7f6c54101220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009880 tx=0x7f6c4402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 -- 172.21.15.31:0/3348951728 shutdown_connections 2024-09-16T08:59:27.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 --2- 172.21.15.31:0/3348951728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c54100da0 0x7f6c54101220 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 --2- 172.21.15.31:0/3348951728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c540ffba0 0x7f6c540fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 -- 172.21.15.31:0/3348951728 >> 172.21.15.31:0/3348951728 conn(0x7f6c540fb330 msgr2=0x7f6c540fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:27.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 -- 172.21.15.31:0/3348951728 shutdown_connections 2024-09-16T08:59:27.734 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.732+0000 7f6c5b375640 1 -- 172.21.15.31:0/3348951728 wait complete. 2024-09-16T08:59:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c5b375640 1 Processor -- start 2024-09-16T08:59:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c5b375640 1 -- start start 2024-09-16T08:59:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c5b375640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540ffba0 0x7f6c541955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.735 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c5b375640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c54100da0 0x7f6c54195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c5b375640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541960d0 con 0x7f6c540ffba0 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c5b375640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c54196240 con 0x7f6c54100da0 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c59111640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540ffba0 0x7f6c541955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c59111640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540ffba0 0x7f6c541955c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42358/0 (socket says 172.21.15.31:42358) 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c59111640 1 -- 172.21.15.31:0/2543130266 learned_addr learned my addr 172.21.15.31:0/2543130266 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c58910640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c54100da0 0x7f6c54195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c59111640 1 -- 172.21.15.31:0/2543130266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c54100da0 msgr2=0x7f6c54195b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c59111640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c54100da0 0x7f6c54195b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.733+0000 7f6c59111640 1 -- 172.21.15.31:0/2543130266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44009560 con 0x7f6c540ffba0 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.734+0000 7f6c59111640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540ffba0 0x7f6c541955c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e970 tx=0x7f6c5000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.734+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/2543130266 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c50009700 con 0x7f6c540ffba0 2024-09-16T08:59:27.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.734+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5419ace0 con 0x7f6c540ffba0 2024-09-16T08:59:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.734+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/2543130266 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c5000cc90 con 0x7f6c540ffba0 2024-09-16T08:59:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.734+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/2543130266 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c50010430 con 0x7f6c540ffba0 2024-09-16T08:59:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.734+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5419b230 con 0x7f6c540ffba0 2024-09-16T08:59:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.735+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c54108790 con 0x7f6c540ffba0 2024-09-16T08:59:27.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.737+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/2543130266 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 37) v1 ==== 103212+0+0 (secure 0 0 0) 0x7f6c50003f90 con 0x7f6c540ffba0 2024-09-16T08:59:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.737+0000 7f6c4a7fc640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6c4007afd0 0x7f6c4007d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.737+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/2543130266 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f6c50014070 con 0x7f6c540ffba0 2024-09-16T08:59:27.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.737+0000 7f6c58910640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6c4007afd0 0x7f6c4007d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:27.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.738+0000 7f6c58910640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6c4007afd0 0x7f6c4007d490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c440314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:27.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:27.742+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/2543130266 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6c50061610 con 0x7f6c540ffba0 2024-09-16T08:59:27.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:27 smithi097 ceph-mon[35478]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:27.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:27 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:27.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:27 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:28.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.005+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6c5419b530 con 0x7f6c540ffba0 2024-09-16T08:59:28.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.005+0000 7f6c4a7fc640 1 -- 172.21.15.31:0/2543130266 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+835 (secure 0 0 0) 0x7f6c50064730 con 0x7f6c540ffba0 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 1, 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T08:59:28.007 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T08:59:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T08:59:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T08:59:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 11, 2024-09-16T08:59:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 3 2024-09-16T08:59:28.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T08:59:28.008 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6c4007afd0 msgr2=0x7f6c4007d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6c4007afd0 0x7f6c4007d490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c44000c00 tx=0x7f6c440314e0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540ffba0 msgr2=0x7f6c541955c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540ffba0 0x7f6c541955c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000e970 tx=0x7f6c5000ee30 comp rx=0 tx=0).stop 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 shutdown_connections 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6c4007afd0 0x7f6c4007d490 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c54100da0 0x7f6c54195b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.009 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 --2- 172.21.15.31:0/2543130266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c540ffba0 0x7f6c541955c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 >> 172.21.15.31:0/2543130266 conn(0x7f6c540fb330 msgr2=0x7f6c540fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 shutdown_connections 2024-09-16T08:59:28.010 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.008+0000 7f6c5b375640 1 -- 172.21.15.31:0/2543130266 wait complete. 2024-09-16T08:59:28.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 -- 172.21.15.31:0/1561787251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 msgr2=0x7f9f68100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/1561787251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f68100170 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9f50009880 tx=0x7f9f5002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 -- 172.21.15.31:0/1561787251 shutdown_connections 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/1561787251 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f68100170 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/1561787251 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f680feaf0 0x7f9f680feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 -- 172.21.15.31:0/1561787251 >> 172.21.15.31:0/1561787251 conn(0x7f9f680fa280 msgr2=0x7f9f680fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 -- 172.21.15.31:0/1561787251 shutdown_connections 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.108+0000 7f9f6df2b640 1 -- 172.21.15.31:0/1561787251 wait complete. 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f6df2b640 1 Processor -- start 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f6df2b640 1 -- start start 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f6df2b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f680feaf0 0x7f9f68102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f6df2b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f681010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f6df2b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68101610 con 0x7f9f680ffcf0 2024-09-16T08:59:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f6df2b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f68101780 con 0x7f9f680feaf0 2024-09-16T08:59:28.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f681010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f5ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f681010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42366/0 (socket says 172.21.15.31:42366) 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f677fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f680feaf0 0x7f9f68102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f5ffff640 1 -- 172.21.15.31:0/984231058 learned_addr learned my addr 172.21.15.31:0/984231058 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f677fe640 1 -- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 msgr2=0x7f9f681010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f677fe640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f681010d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f677fe640 1 -- 172.21.15.31:0/984231058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50009560 con 0x7f9f680feaf0 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.109+0000 7f9f5ffff640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f681010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.110+0000 7f9f677fe640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f680feaf0 0x7f9f68102a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f680ffb50 tx=0x7f9f5800db30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.110+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f58009990 con 0x7f9f680feaf0 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.110+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f58010460 con 0x7f9f680feaf0 2024-09-16T08:59:28.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.110+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5800f5f0 con 0x7f9f680feaf0 2024-09-16T08:59:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.110+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f68101a60 con 0x7f9f680feaf0 2024-09-16T08:59:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.110+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6819d730 con 0x7f9f680feaf0 2024-09-16T08:59:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.111+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f680feef0 con 0x7f9f680feaf0 2024-09-16T08:59:28.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.112+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 37) v1 ==== 103212+0+0 (secure 0 0 0) 0x7f9f580105d0 con 0x7f9f680feaf0 2024-09-16T08:59:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.113+0000 7f9f657fa640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f9f5407afd0 0x7f9f5407d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.113+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9f5809dda0 con 0x7f9f680feaf0 2024-09-16T08:59:28.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.113+0000 7f9f5ffff640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f9f5407afd0 0x7f9f5407d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.114+0000 7f9f5ffff640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f9f5407afd0 0x7f9f5407d490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f68102800 tx=0x7f9f500314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:28.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.118+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f9f58060e80 con 0x7f9f680feaf0 2024-09-16T08:59:28.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.202+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f9f58060aa0 con 0x7f9f680feaf0 2024-09-16T08:59:28.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.356+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9f68107970 con 0x7f9f680feaf0 2024-09-16T08:59:28.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.357+0000 7f9f657fa640 1 -- 172.21.15.31:0/984231058 <== mon.1 v2:172.21.15.97:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1848 (secure 0 0 0) 0x7f9f5807f430 con 0x7f9f680feaf0 2024-09-16T08:59:28.358 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T08:59:28.358 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T08:59:28.359 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T08:59:28.360 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:28.361 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:28.362 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T08:59:28.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f9f5407afd0 msgr2=0x7f9f5407d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f9f5407afd0 0x7f9f5407d490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9f68102800 tx=0x7f9f500314e0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f680feaf0 msgr2=0x7f9f68102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f680feaf0 0x7f9f68102a80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9f680ffb50 tx=0x7f9f5800db30 comp rx=0 tx=0).stop 2024-09-16T08:59:28.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 shutdown_connections 2024-09-16T08:59:28.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f9f5407afd0 0x7f9f5407d490 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f680ffcf0 0x7f9f681010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 --2- 172.21.15.31:0/984231058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9f680feaf0 0x7f9f68102a80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 >> 172.21.15.31:0/984231058 conn(0x7f9f680fa280 msgr2=0x7f9f68067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 shutdown_connections 2024-09-16T08:59:28.363 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.359+0000 7f9f6df2b640 1 -- 172.21.15.31:0/984231058 wait complete. 2024-09-16T08:59:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.445+0000 7ffa507eb640 1 -- 172.21.15.31:0/4222875008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480ffb90 msgr2=0x7ffa480fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.445+0000 7ffa507eb640 1 --2- 172.21.15.31:0/4222875008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480ffb90 0x7ffa480fff90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009880 tx=0x7ffa4402a4a0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.445+0000 7ffa507eb640 1 -- 172.21.15.31:0/4222875008 shutdown_connections 2024-09-16T08:59:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.445+0000 7ffa507eb640 1 --2- 172.21.15.31:0/4222875008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa48100d90 0x7ffa48101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.445+0000 7ffa507eb640 1 --2- 172.21.15.31:0/4222875008 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480ffb90 0x7ffa480fff90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.445+0000 7ffa507eb640 1 -- 172.21.15.31:0/4222875008 >> 172.21.15.31:0/4222875008 conn(0x7ffa480fb320 msgr2=0x7ffa480fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:28.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.445+0000 7ffa507eb640 1 -- 172.21.15.31:0/4222875008 shutdown_connections 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa507eb640 1 -- 172.21.15.31:0/4222875008 wait complete. 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa507eb640 1 Processor -- start 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa507eb640 1 -- start start 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa507eb640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa480ffb90 0x7ffa48074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa507eb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa48100d90 0x7ffa480728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa507eb640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48072de0 con 0x7ffa48100d90 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa507eb640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48072f50 con 0x7ffa480ffb90 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa4dd86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa48100d90 0x7ffa480728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa4dd86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa48100d90 0x7ffa480728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42378/0 (socket says 172.21.15.31:42378) 2024-09-16T08:59:28.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa4dd86640 1 -- 172.21.15.31:0/753230509 learned_addr learned my addr 172.21.15.31:0/753230509 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.446+0000 7ffa4e587640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa480ffb90 0x7ffa48074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa4dd86640 1 -- 172.21.15.31:0/753230509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa480ffb90 msgr2=0x7ffa48074200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa4dd86640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa480ffb90 0x7ffa48074200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa4dd86640 1 -- 172.21.15.31:0/753230509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa44009560 con 0x7ffa48100d90 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa4dd86640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa48100d90 0x7ffa480728a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d6f0 tx=0x7ffa3800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa3f7fe640 1 -- 172.21.15.31:0/753230509 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa38009a30 con 0x7ffa48100d90 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa3f7fe640 1 -- 172.21.15.31:0/753230509 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa380054e0 con 0x7ffa48100d90 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa48073230 con 0x7ffa48100d90 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa3f7fe640 1 -- 172.21.15.31:0/753230509 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa38010460 con 0x7ffa48100d90 2024-09-16T08:59:28.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.447+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4819d6f0 con 0x7ffa48100d90 2024-09-16T08:59:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.448+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa480fff90 con 0x7ffa48100d90 2024-09-16T08:59:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.449+0000 7ffa3f7fe640 1 -- 172.21.15.31:0/753230509 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7ffa38009b90 con 0x7ffa48100d90 2024-09-16T08:59:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.450+0000 7ffa3f7fe640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ffa3407b060 0x7ffa3407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.450+0000 7ffa3f7fe640 1 -- 172.21.15.31:0/753230509 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ffa3809dd50 con 0x7ffa48100d90 2024-09-16T08:59:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.450+0000 7ffa4e587640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ffa3407b060 0x7ffa3407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.450+0000 7ffa4e587640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ffa3407b060 0x7ffa3407d520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009230 tx=0x7ffa440314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:28.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.454+0000 7ffa3f7fe640 1 -- 172.21.15.31:0/753230509 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ffa38060cf0 con 0x7ffa48100d90 2024-09-16T08:59:28.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.671+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa480608f0 con 0x7ffa3407b060 2024-09-16T08:59:28.673 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.673+0000 7ffa3f7fe640 1 -- 172.21.15.31:0/753230509 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7ffa480608f0 con 0x7ffa3407b060 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "2/23 daemons upgraded", 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T08:59:28.674 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ffa3407b060 msgr2=0x7ffa3407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ffa3407b060 0x7ffa3407d520 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009230 tx=0x7ffa440314e0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa48100d90 msgr2=0x7ffa480728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa48100d90 0x7ffa480728a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d6f0 tx=0x7ffa3800dbb0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 shutdown_connections 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ffa3407b060 0x7ffa3407d520 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa48100d90 0x7ffa480728a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 --2- 172.21.15.31:0/753230509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa480ffb90 0x7ffa48074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.676 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 >> 172.21.15.31:0/753230509 conn(0x7ffa480fb320 msgr2=0x7ffa481044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 shutdown_connections 2024-09-16T08:59:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.675+0000 7ffa507eb640 1 -- 172.21.15.31:0/753230509 wait complete. 2024-09-16T08:59:28.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 -- 172.21.15.31:0/2076824980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900feaf0 msgr2=0x7fb8900feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 --2- 172.21.15.31:0/2076824980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900feaf0 0x7fb8900feef0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb880009880 tx=0x7fb88002a4a0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 -- 172.21.15.31:0/2076824980 shutdown_connections 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 --2- 172.21.15.31:0/2076824980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900ffcf0 0x7fb890100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 --2- 172.21.15.31:0/2076824980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900feaf0 0x7fb8900feef0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 -- 172.21.15.31:0/2076824980 >> 172.21.15.31:0/2076824980 conn(0x7fb8900fa280 msgr2=0x7fb8900fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 -- 172.21.15.31:0/2076824980 shutdown_connections 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 -- 172.21.15.31:0/2076824980 wait complete. 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.762+0000 7fb895aef640 1 Processor -- start 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb895aef640 1 -- start start 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb895aef640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900feaf0 0x7fb8901956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb895aef640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900ffcf0 0x7fb890195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb895aef640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8901961f0 con 0x7fb8900ffcf0 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb895aef640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb890196360 con 0x7fb8900feaf0 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb887fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900ffcf0 0x7fb890195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb887fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900ffcf0 0x7fb890195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42396/0 (socket says 172.21.15.31:42396) 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb887fff640 1 -- 172.21.15.31:0/2400133207 learned_addr learned my addr 172.21.15.31:0/2400133207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb88f7fe640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900feaf0 0x7fb8901956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb887fff640 1 -- 172.21.15.31:0/2400133207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900feaf0 msgr2=0x7fb8901956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb887fff640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900feaf0 0x7fb8901956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb887fff640 1 -- 172.21.15.31:0/2400133207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb880009560 con 0x7fb8900ffcf0 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.763+0000 7fb887fff640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900ffcf0 0x7fb890195c20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb87800c920 tx=0x7fb87800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:28.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.764+0000 7fb88d7fa640 1 -- 172.21.15.31:0/2400133207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb878007b80 con 0x7fb8900ffcf0 2024-09-16T08:59:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.764+0000 7fb88d7fa640 1 -- 172.21.15.31:0/2400133207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8780054e0 con 0x7fb8900ffcf0 2024-09-16T08:59:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.764+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89019ae00 con 0x7fb8900ffcf0 2024-09-16T08:59:28.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.764+0000 7fb88d7fa640 1 -- 172.21.15.31:0/2400133207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb87800f450 con 0x7fb8900ffcf0 2024-09-16T08:59:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.764+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89019b320 con 0x7fb8900ffcf0 2024-09-16T08:59:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.765+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8900feef0 con 0x7fb8900ffcf0 2024-09-16T08:59:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.765+0000 7fb88d7fa640 1 -- 172.21.15.31:0/2400133207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7fb878004f60 con 0x7fb8900ffcf0 2024-09-16T08:59:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.766+0000 7fb88d7fa640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87c07b050 0x7fb87c07d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.766+0000 7fb88d7fa640 1 -- 172.21.15.31:0/2400133207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fb87809df20 con 0x7fb8900ffcf0 2024-09-16T08:59:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.766+0000 7fb88f7fe640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87c07b050 0x7fb87c07d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:28.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.766+0000 7fb88f7fe640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87c07b050 0x7fb87c07d510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb8800314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:28.771 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:28.770+0000 7fb88d7fa640 1 -- 172.21.15.31:0/2400133207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fb8780d5190 con 0x7fb8900ffcf0 2024-09-16T08:59:28.987 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: pgmap v5: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:28.987 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:28.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:28.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:28.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:28.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2543130266' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T08:59:28.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: mgrmap e38: smithi031.dcmqpv(active, since 4s), standbys: smithi097.mpashf 2024-09-16T08:59:28.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:28 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/984231058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:59:29.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: pgmap v5: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2543130266' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T08:59:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: mgrmap e38: smithi031.dcmqpv(active, since 4s), standbys: smithi097.mpashf 2024-09-16T08:59:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:28 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/984231058' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T08:59:29.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.035+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb890107700 con 0x7fb8900ffcf0 2024-09-16T08:59:29.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.036+0000 7fb88d7fa640 1 -- 172.21.15.31:0/2400133207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb878060f60 con 0x7fb8900ffcf0 2024-09-16T08:59:29.037 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T08:59:29.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87c07b050 msgr2=0x7fb87c07d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87c07b050 0x7fb87c07d510 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb880006010 tx=0x7fb8800314e0 comp rx=0 tx=0).stop 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900ffcf0 msgr2=0x7fb890195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900ffcf0 0x7fb890195c20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb87800c920 tx=0x7fb87800cde0 comp rx=0 tx=0).stop 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 shutdown_connections 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87c07b050 0x7fb87c07d510 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8900ffcf0 0x7fb890195c20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.038+0000 7fb895aef640 1 --2- 172.21.15.31:0/2400133207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8900feaf0 0x7fb8901956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.039+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 >> 172.21.15.31:0/2400133207 conn(0x7fb8900fa280 msgr2=0x7fb890067ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.039+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 shutdown_connections 2024-09-16T08:59:29.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:29.039+0000 7fb895aef640 1 -- 172.21.15.31:0/2400133207 wait complete. 2024-09-16T08:59:30.097 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:29 smithi097 ceph-mon[35478]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:30.097 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:29 smithi097 ceph-mon[35478]: from='client.? 172.21.15.31:0/2400133207' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T08:59:30.097 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:29 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:30.097 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:29 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:30.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:29 smithi031 ceph-mon[91884]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T08:59:30.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:29 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2400133207' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T08:59:30.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:29 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:30.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:29 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:30 smithi097 ceph-mon[35478]: pgmap v6: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:30 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:30 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:30 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:30 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:30 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:30 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:59:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:30 smithi031 ceph-mon[91884]: pgmap v6: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:59:31.976 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:31.977 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:31.978 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T08:59:31.978 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:31 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "quorum_status"} : dispatch 2024-09-16T08:59:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: Updating smithi031:/etc/ceph/ceph.conf 2024-09-16T08:59:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: Updating smithi097:/etc/ceph/ceph.conf 2024-09-16T08:59:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:59:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T08:59:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: Updating smithi031:/etc/ceph/ceph.client.admin.keyring 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T08:59:32.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "quorum_status"} : dispatch 2024-09-16T08:59:32.961 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:32 smithi097 ceph-mon[35478]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:59:32.961 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:32 smithi097 ceph-mon[35478]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:59:32.961 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:32 smithi097 ceph-mon[35478]: pgmap v7: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:59:32.961 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:32 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:32.961 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:32 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:59:32.962 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:32 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:59:32.962 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:32 smithi097 ceph-mon[35478]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:32 smithi031 ceph-mon[91884]: Updating smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:59:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:32 smithi031 ceph-mon[91884]: Updating smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T08:59:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:32 smithi031 ceph-mon[91884]: pgmap v7: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T08:59:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:59:33.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:59:33.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:33 smithi097 ceph-mon[35478]: Upgrade: Updating mon.smithi097 2024-09-16T08:59:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:33 smithi097 ceph-mon[35478]: Deploying daemon mon.smithi097 on smithi097 2024-09-16T08:59:34.112 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:33 smithi031 ceph-mon[91884]: Upgrade: Updating mon.smithi097 2024-09-16T08:59:34.112 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:33 smithi031 ceph-mon[91884]: Deploying daemon mon.smithi097 on smithi097 2024-09-16T08:59:34.347 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:34 smithi097 systemd[1]: Stopping Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:59:34.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:34 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097[35460]: 2024-09-16T08:59:34.345+0000 7f53fa917640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi097 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T08:59:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:34 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097[35460]: 2024-09-16T08:59:34.345+0000 7f53fa917640 -1 mon.smithi097@1(peon) e2 *** Got Signal Terminated *** 2024-09-16T08:59:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:34 smithi097 podman[79733]: 2024-09-16 08:59:34.50969143 +0000 UTC m=+0.287784286 container died a26ddb2c052373921d65bcd9579d2820a30a2e29adb28b5547809bd0c96ab79d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T08:59:35.173 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:34 smithi097 podman[79733]: 2024-09-16 08:59:34.904268942 +0000 UTC m=+0.682361797 container cleanup a26ddb2c052373921d65bcd9579d2820a30a2e29adb28b5547809bd0c96ab79d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T08:59:35.173 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:34 smithi097 bash[79733]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097 2024-09-16T08:59:35.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 podman[79748]: 2024-09-16 08:59:35.172542134 +0000 UTC m=+0.657447707 container remove a26ddb2c052373921d65bcd9579d2820a30a2e29adb28b5547809bd0c96ab79d (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T08:59:35.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi097.service: Deactivated successfully. 2024-09-16T08:59:35.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 systemd[1]: Stopped Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:59:35.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi097.service: Consumed 6.498s CPU time. 2024-09-16T08:59:35.822 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 systemd[1]: Starting Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T08:59:35.822 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 podman[79873]: 2024-09-16 08:59:35.732399185 +0000 UTC m=+0.106586831 container create ab1d49446e3799e5a12343174d4256f6bbed954bc5f8b86e7538b33d90049c51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T08:59:35.822 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 podman[79873]: 2024-09-16 08:59:35.641581487 +0000 UTC m=+0.015769132 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T08:59:36.184 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 podman[79873]: 2024-09-16 08:59:35.943321592 +0000 UTC m=+0.317509237 container init ab1d49446e3799e5a12343174d4256f6bbed954bc5f8b86e7538b33d90049c51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T08:59:36.184 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 podman[79873]: 2024-09-16 08:59:35.946435455 +0000 UTC m=+0.320623100 container start ab1d49446e3799e5a12343174d4256f6bbed954bc5f8b86e7538b33d90049c51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T08:59:36.184 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 2 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: pidfile_write: ignore empty --pid-file 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: load: jerasure load: lrc 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: RocksDB version: 7.9.2 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Git sha 0 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: DB SUMMARY 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: DB Session ID: SVZO3Z7TWTD6PXKQHYBU 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: CURRENT file: CURRENT 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T08:59:36.185 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1411 Bytes 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi097/store.db dir, Total Num: 1, files: 000027.sst 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi097/store.db: 000025.log size: 4091881 ; 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.error_if_exists: 0 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.create_if_missing: 0 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.paranoid_checks: 1 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.env: 0x55652ab15120 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.info_log: 0x55652c35b8e0 2024-09-16T08:59:36.186 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.statistics: (nil) 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.use_fsync: 0 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_log_file_size: 0 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.allow_fallocate: 1 2024-09-16T08:59:36.188 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.use_direct_reads: 0 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.db_log_dir: 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.wal_dir: 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T08:59:36.189 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.write_buffer_manager: 0x55652c35fb80 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T08:59:36.190 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.unordered_write: 0 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.row_cache: None 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.wal_filter: None 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.two_write_queues: 0 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.wal_compression: 0 2024-09-16T08:59:36.191 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.atomic_flush: 0 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.log_readahead_size: 0 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_background_jobs: 2 2024-09-16T08:59:36.192 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_background_compactions: -1 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_subcompactions: 1 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_open_files: -1 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T08:59:36.193 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_background_flushes: -1 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Compression algorithms supported: 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kZSTD supported: 0 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kXpressCompression supported: 0 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kBZip2Compression supported: 0 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kLZ4Compression supported: 1 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kZlibCompression supported: 1 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: kSnappyCompression supported: 1 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T08:59:36.194 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi097/store.db/MANIFEST-000010 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.merge_operator: 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_filter: None 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55652c35b540) 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: cache_index_and_filter_blocks: 1 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: pin_top_level_index_and_filter: 1 2024-09-16T08:59:36.195 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: index_type: 0 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: data_block_index_type: 0 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: index_shortening: 1 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: checksum: 4 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: no_block_cache: 0 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache: 0x55652c34f610 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache_name: BinnedLRUCache 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache_options: 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: capacity : 536870912 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: num_shard_bits : 4 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: strict_capacity_limit : 0 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: high_pri_pool_ratio: 0.000 2024-09-16T08:59:36.196 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_cache_compressed: (nil) 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: persistent_cache: (nil) 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_size: 4096 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_size_deviation: 10 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_restart_interval: 16 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: index_block_restart_interval: 1 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: metadata_block_size: 4096 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: partition_filters: 0 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: use_delta_encoding: 1 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: filter_policy: bloomfilter 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: whole_key_filtering: 1 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: verify_compression: 0 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: read_amp_bytes_per_bit: 0 2024-09-16T08:59:36.197 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: format_version: 5 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: enable_index_compression: 1 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: block_align: 0 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: max_auto_readahead_size: 262144 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: prepopulate_block_cache: 0 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: initial_auto_readahead_size: 8192 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression: NoCompression 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.num_levels: 7 2024-09-16T08:59:36.198 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T08:59:36.199 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T08:59:36.200 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T08:59:36.201 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.inplace_update_support: 0 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T08:59:36.202 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.bloom_locality: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.max_successive_merges: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.ttl: 2592000 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T08:59:36.203 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.enable_blob_files: false 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.min_blob_size: 0 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi097/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12170, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-09-16T08:59:36.204 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 450393fc-fda0-48b4-8d44-55dd7896794d 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726477175972454, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726477175981517, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 2571172, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12175, "largest_seqno": 13011, "table_properties": {"data_size": 2566566, "index_size": 2602, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 965, "raw_key_size": 9348, "raw_average_key_size": 24, "raw_value_size": 2558039, "raw_average_value_size": 6696, "num_data_blocks": 119, "num_entries": 382, "num_filter_entries": 382, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726477175, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "450393fc-fda0-48b4-8d44-55dd7896794d", "db_session_id": "SVZO3Z7TWTD6PXKQHYBU", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726477175981687, "job": 1, "event": "recovery_finished"} 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi097/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55652c37ee00 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: DB pointer 0x55652c480000 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T08:59:36.205 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** DB Stats ** 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** Compaction Stats [default] ** 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: L0 1/0 2.45 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 678.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-16T08:59:36.206 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: L6 1/0 6.79 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Sum 2/0 9.25 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 678.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 678.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** Compaction Stats [default] ** 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 678.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T08:59:36.207 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Cumulative compaction: 0.00 GB write, 185.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Interval compaction: 0.00 GB write, 185.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Block cache BinnedLRUCache@0x55652c34f610#2 capacity: 512.00 MB usage: 3.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.03 KB,0.000196695%) IndexBlock(1,2.83 KB,0.000539422%) Misc(1,0.00 KB,0%) 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: starting mon.smithi097 rank 1 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi097 fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:36.208 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???) e2 preinit fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???).mds e17 new map 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???).mds e17 print_map 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: e17 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: legacy client fscid: 1 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Filesystem 'cephfs' (1) 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: fs_name cephfs 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: epoch 17 2024-09-16T08:59:36.209 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: created 2024-09-16T08:54:37.289736+0000 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: modified 2024-09-16T08:54:56.899545+0000 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: tableserver 0 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: root 0 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: session_timeout 60 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: session_autoclose 300 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: max_file_size 1099511627776 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: max_xattr_size 65536 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: required_client_features {} 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: last_failure 0 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: last_failure_osd_epoch 47 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: max_mds 2 2024-09-16T08:59:36.210 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: in 0,1 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: up {0=14548,1=24325} 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: failed 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: damaged 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: stopped 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: data_pools [3] 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: metadata_pool 2 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: inline_data disabled 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: balancer 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: bal_rank_mask -1 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: standby_count_wanted 1 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: qdb_cluster leader: 0 members: 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: [mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:36.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: [mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: Standby daemons: 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: [mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout: [mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:35 smithi097 ceph-mon[79892]: mon.smithi097@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:36 smithi097 bash[79873]: ab1d49446e3799e5a12343174d4256f6bbed954bc5f8b86e7538b33d90049c51 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:36 smithi097 systemd[1]: Started Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T08:59:36.212 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:36 smithi097 ceph-mon[79892]: pgmap v8: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:59:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:36 smithi031 ceph-mon[91884]: pgmap v8: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-16T08:59:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi031"} : dispatch 2024-09-16T08:59:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi097"} : dispatch 2024-09-16T08:59:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: mon.smithi031 calling monitor election 2024-09-16T08:59:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: mon.smithi097 calling monitor election 2024-09-16T08:59:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: mon.smithi031 is new leader, mons smithi031,smithi097 in quorum (ranks 0,1) 2024-09-16T08:59:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: monmap epoch 3 2024-09-16T08:59:37.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: last_changed 2024-09-16T08:59:36.186369+0000 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: created 2024-09-16T08:47:00.018794+0000 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: min_mon_release 19 (squid) 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: election_strategy: 1 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.smithi097 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: osdmap e50: 6 total, 6 up, 6 in 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: mgrmap e38: smithi031.dcmqpv(active, since 12s), standbys: smithi097.mpashf 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: overall HEALTH_OK 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:37.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:37 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi031"} : dispatch 2024-09-16T08:59:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mon metadata", "id": "smithi097"} : dispatch 2024-09-16T08:59:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: mon.smithi031 calling monitor election 2024-09-16T08:59:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: mon.smithi097 calling monitor election 2024-09-16T08:59:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: mon.smithi031 is new leader, mons smithi031,smithi097 in quorum (ranks 0,1) 2024-09-16T08:59:37.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: monmap epoch 3 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: fsid 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: last_changed 2024-09-16T08:59:36.186369+0000 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: created 2024-09-16T08:47:00.018794+0000 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: min_mon_release 19 (squid) 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: election_strategy: 1 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: 0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.smithi031 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: 1: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.smithi097 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: osdmap e50: 6 total, 6 up, 6 in 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: mgrmap e38: smithi031.dcmqpv(active, since 12s), standbys: smithi097.mpashf 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: overall HEALTH_OK 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:37.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:37.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:37 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T08:59:38.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:38 smithi097 ceph-mon[79892]: pgmap v10: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:59:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:38 smithi031 ceph-mon[91884]: pgmap v10: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:59:39.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:39.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:39.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:39.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:40.994 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:40 smithi031 ceph-mon[91884]: pgmap v11: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:59:40.994 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:40 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:40.995 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:40 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:41.002 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:40 smithi097 ceph-mon[79892]: pgmap v11: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:59:41.002 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:40 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:41.002 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:40 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: pgmap v12: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:59:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:43.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:59:43.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:43.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:59:43.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:59:43.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: pgmap v12: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T08:59:43.362 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T08:59:43.363 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:44.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:44 smithi097 ceph-mon[79892]: Reconfiguring mon.smithi031 (monmap changed)... 2024-09-16T08:59:44.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:44 smithi097 ceph-mon[79892]: Reconfiguring daemon mon.smithi031 on smithi031 2024-09-16T08:59:44.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:44 smithi031 ceph-mon[91884]: Reconfiguring mon.smithi031 (monmap changed)... 2024-09-16T08:59:44.275 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:44 smithi031 ceph-mon[91884]: Reconfiguring daemon mon.smithi031 on smithi031 2024-09-16T08:59:45.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:45 smithi031 ceph-mon[91884]: pgmap v13: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:45.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:45 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:45.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:45 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:45.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:45 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:59:45.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:45 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:59:45.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:45 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:45 smithi097 ceph-mon[79892]: pgmap v13: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:45 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:45 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:45 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi031.dcmqpv", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T08:59:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:45 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T08:59:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:45 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: Reconfiguring mgr.smithi031.dcmqpv (monmap changed)... 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: Reconfiguring daemon mgr.smithi031.dcmqpv on smithi031 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi031"} : dispatch 2024-09-16T08:59:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:46.338 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: Reconfiguring mgr.smithi031.dcmqpv (monmap changed)... 2024-09-16T08:59:46.338 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: Reconfiguring daemon mgr.smithi031.dcmqpv on smithi031 2024-09-16T08:59:46.338 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:46.338 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:46.338 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:46.338 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:46.339 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T08:59:46.339 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi031"} : dispatch 2024-09-16T08:59:46.339 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:47.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:47 smithi097 ceph-mon[79892]: pgmap v14: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:47 smithi097 ceph-mon[79892]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-09-16T08:59:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:47 smithi097 ceph-mon[79892]: Unable to update caps for client.ceph-exporter.smithi031 2024-09-16T08:59:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:47 smithi097 ceph-mon[79892]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-09-16T08:59:47.353 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:47 smithi031 ceph-mon[91884]: pgmap v14: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:47.354 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:47 smithi031 ceph-mon[91884]: Reconfiguring ceph-exporter.smithi031 (monmap changed)... 2024-09-16T08:59:47.354 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:47 smithi031 ceph-mon[91884]: Unable to update caps for client.ceph-exporter.smithi031 2024-09-16T08:59:47.354 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:47 smithi031 ceph-mon[91884]: Reconfiguring daemon ceph-exporter.smithi031 on smithi031 2024-09-16T08:59:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:48 smithi097 ceph-mon[79892]: pgmap v15: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:48 smithi097 ceph-mon[79892]: Reconfiguring crash.smithi031 (monmap changed)... 2024-09-16T08:59:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T08:59:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:48 smithi097 ceph-mon[79892]: Reconfiguring daemon crash.smithi031 on smithi031 2024-09-16T08:59:48.844 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:48 smithi031 ceph-mon[91884]: pgmap v15: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:48.844 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:48.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:48.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:48 smithi031 ceph-mon[91884]: Reconfiguring crash.smithi031 (monmap changed)... 2024-09-16T08:59:48.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T08:59:48.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:48.845 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:48 smithi031 ceph-mon[91884]: Reconfiguring daemon crash.smithi031 on smithi031 2024-09-16T08:59:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:49 smithi097 ceph-mon[79892]: Reconfiguring osd.0 (monmap changed)... 2024-09-16T08:59:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T08:59:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:49 smithi097 ceph-mon[79892]: Reconfiguring daemon osd.0 on smithi031 2024-09-16T08:59:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:49 smithi031 ceph-mon[91884]: Reconfiguring osd.0 (monmap changed)... 2024-09-16T08:59:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T08:59:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:49 smithi031 ceph-mon[91884]: Reconfiguring daemon osd.0 on smithi031 2024-09-16T08:59:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:50 smithi031 ceph-mon[91884]: pgmap v16: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:50 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:50 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:50 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T08:59:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:50 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:50 smithi097 ceph-mon[79892]: pgmap v16: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:50 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:50 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:50 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T08:59:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:50 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:52.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:51 smithi031 ceph-mon[91884]: Reconfiguring osd.1 (monmap changed)... 2024-09-16T08:59:52.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:51 smithi031 ceph-mon[91884]: Reconfiguring daemon osd.1 on smithi031 2024-09-16T08:59:52.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:51 smithi097 ceph-mon[79892]: Reconfiguring osd.1 (monmap changed)... 2024-09-16T08:59:52.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:51 smithi097 ceph-mon[79892]: Reconfiguring daemon osd.1 on smithi031 2024-09-16T08:59:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:52 smithi097 ceph-mon[79892]: pgmap v17: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:52 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:52 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:52 smithi097 ceph-mon[79892]: Reconfiguring osd.2 (monmap changed)... 2024-09-16T08:59:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:52 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T08:59:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:52 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:52 smithi097 ceph-mon[79892]: Reconfiguring daemon osd.2 on smithi031 2024-09-16T08:59:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:52 smithi031 ceph-mon[91884]: pgmap v17: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:52 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:52 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:53.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:52 smithi031 ceph-mon[91884]: Reconfiguring osd.2 (monmap changed)... 2024-09-16T08:59:53.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:52 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T08:59:53.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:52 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:53.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:52 smithi031 ceph-mon[91884]: Reconfiguring daemon osd.2 on smithi031 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: pgmap v18: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: Reconfiguring mds.cephfs.smithi031.lcfutm (monmap changed)... 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: Reconfiguring daemon mds.cephfs.smithi031.lcfutm on smithi031 2024-09-16T08:59:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: pgmap v18: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: Reconfiguring mds.cephfs.smithi031.lcfutm (monmap changed)... 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: Reconfiguring daemon mds.cephfs.smithi031.lcfutm on smithi031 2024-09-16T08:59:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T08:59:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:55 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:55 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:55 smithi097 ceph-mon[79892]: Reconfiguring mds.cephfs.smithi031.mjhxgi (monmap changed)... 2024-09-16T08:59:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:55 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T08:59:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:55 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:55 smithi097 ceph-mon[79892]: Reconfiguring daemon mds.cephfs.smithi031.mjhxgi on smithi031 2024-09-16T08:59:56.340 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:55 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:56.340 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:55 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:56.340 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:55 smithi031 ceph-mon[91884]: Reconfiguring mds.cephfs.smithi031.mjhxgi (monmap changed)... 2024-09-16T08:59:56.340 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:55 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T08:59:56.340 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:55 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:56.340 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:55 smithi031 ceph-mon[91884]: Reconfiguring daemon mds.cephfs.smithi031.mjhxgi on smithi031 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: pgmap v19: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi097"} : dispatch 2024-09-16T08:59:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:56 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: pgmap v19: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth caps", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.ceph-exporter.smithi097"} : dispatch 2024-09-16T08:59:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:56 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:58.235 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:57 smithi097 ceph-mon[79892]: Reconfiguring ceph-exporter.smithi097 (monmap changed)... 2024-09-16T08:59:58.235 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:57 smithi097 ceph-mon[79892]: Unable to update caps for client.ceph-exporter.smithi097 2024-09-16T08:59:58.235 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:57 smithi097 ceph-mon[79892]: Reconfiguring daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:59:58.235 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:58.235 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:58.236 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T08:59:58.236 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:57 smithi031 ceph-mon[91884]: Reconfiguring ceph-exporter.smithi097 (monmap changed)... 2024-09-16T08:59:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:57 smithi031 ceph-mon[91884]: Unable to update caps for client.ceph-exporter.smithi097 2024-09-16T08:59:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:57 smithi031 ceph-mon[91884]: Reconfiguring daemon ceph-exporter.smithi097 on smithi097 2024-09-16T08:59:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T08:59:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T08:59:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T08:59:58.966 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:58 smithi097 ceph-mon[79892]: pgmap v20: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:59.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.136+0000 7efe3c9a8640 1 -- 172.21.15.31:0/3595738995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 msgr2=0x7efe381029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.136+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/3595738995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe381029a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efe280098b0 tx=0x7efe2802a4d0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.137+0000 7efe3c9a8640 1 -- 172.21.15.31:0/3595738995 shutdown_connections 2024-09-16T08:59:59.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.137+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/3595738995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe38102ee0 0x7efe381052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.137+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/3595738995 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe381029a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.137+0000 7efe3c9a8640 1 -- 172.21.15.31:0/3595738995 >> 172.21.15.31:0/3595738995 conn(0x7efe380fa1f0 msgr2=0x7efe380fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.137+0000 7efe3c9a8640 1 -- 172.21.15.31:0/3595738995 shutdown_connections 2024-09-16T08:59:59.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.137+0000 7efe3c9a8640 1 -- 172.21.15.31:0/3595738995 wait complete. 2024-09-16T08:59:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.137+0000 7efe3c9a8640 1 Processor -- start 2024-09-16T08:59:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3c9a8640 1 -- start start 2024-09-16T08:59:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3c9a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe380ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3c9a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe38102ee0 0x7efe380fe560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3c9a8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380feaa0 con 0x7efe381005b0 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3c9a8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe380fec10 con 0x7efe38102ee0 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe380ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe380ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43846/0 (socket says 172.21.15.31:43846) 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3659c640 1 -- 172.21.15.31:0/1244260633 learned_addr learned my addr 172.21.15.31:0/1244260633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe2ffff640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe38102ee0 0x7efe380fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3659c640 1 -- 172.21.15.31:0/1244260633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe38102ee0 msgr2=0x7efe380fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3659c640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe38102ee0 0x7efe380fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3659c640 1 -- 172.21.15.31:0/1244260633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28009560 con 0x7efe381005b0 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe2ffff640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe38102ee0 0x7efe380fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.138+0000 7efe3659c640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe380ffec0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efe28006010 tx=0x7efe280055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.139+0000 7efe2f7fe640 1 -- 172.21.15.31:0/1244260633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe28004aa0 con 0x7efe381005b0 2024-09-16T08:59:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.139+0000 7efe2f7fe640 1 -- 172.21.15.31:0/1244260633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe28005d50 con 0x7efe381005b0 2024-09-16T08:59:59.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.139+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe380fee90 con 0x7efe381005b0 2024-09-16T08:59:59.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.139+0000 7efe2f7fe640 1 -- 172.21.15.31:0/1244260633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe28003a20 con 0x7efe381005b0 2024-09-16T08:59:59.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.139+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3819d6b0 con 0x7efe381005b0 2024-09-16T08:59:59.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.140+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe38100990 con 0x7efe381005b0 2024-09-16T08:59:59.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.141+0000 7efe2f7fe640 1 -- 172.21.15.31:0/1244260633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7efe28031c00 con 0x7efe381005b0 2024-09-16T08:59:59.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.141+0000 7efe2f7fe640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efe2407b060 0x7efe2407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.141+0000 7efe2f7fe640 1 -- 172.21.15.31:0/1244260633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efe280bf1a0 con 0x7efe381005b0 2024-09-16T08:59:59.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.141+0000 7efe2ffff640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efe2407b060 0x7efe2407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.142+0000 7efe2ffff640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efe2407b060 0x7efe2407d520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe380ffc40 tx=0x7efe20005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:59.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.146+0000 7efe2f7fe640 1 -- 172.21.15.31:0/1244260633 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7efe28082260 con 0x7efe381005b0 2024-09-16T08:59:59.237 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:58 smithi097 ceph-mon[79892]: Reconfiguring crash.smithi097 (monmap changed)... 2024-09-16T08:59:59.237 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 08:59:58 smithi097 ceph-mon[79892]: Reconfiguring daemon crash.smithi097 on smithi097 2024-09-16T08:59:59.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.353+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe38103940 con 0x7efe2407b060 2024-09-16T08:59:59.355 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:58 smithi031 ceph-mon[91884]: pgmap v20: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T08:59:59.355 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:58 smithi031 ceph-mon[91884]: Reconfiguring crash.smithi097 (monmap changed)... 2024-09-16T08:59:59.355 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 08:59:58 smithi031 ceph-mon[91884]: Reconfiguring daemon crash.smithi097 on smithi097 2024-09-16T08:59:59.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.354+0000 7efe2f7fe640 1 -- 172.21.15.31:0/1244260633 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7efe38103940 con 0x7efe2407b060 2024-09-16T08:59:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efe2407b060 msgr2=0x7efe2407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efe2407b060 0x7efe2407d520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efe380ffc40 tx=0x7efe20005f90 comp rx=0 tx=0).stop 2024-09-16T08:59:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 msgr2=0x7efe380ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe380ffec0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efe28006010 tx=0x7efe280055a0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 shutdown_connections 2024-09-16T08:59:59.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efe2407b060 0x7efe2407d520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe38102ee0 0x7efe380fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 --2- 172.21.15.31:0/1244260633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe381005b0 0x7efe380ffec0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 >> 172.21.15.31:0/1244260633 conn(0x7efe380fa1f0 msgr2=0x7efe380fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 shutdown_connections 2024-09-16T08:59:59.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.357+0000 7efe3c9a8640 1 -- 172.21.15.31:0/1244260633 wait complete. 2024-09-16T08:59:59.367 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T08:59:59.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 -- 172.21.15.31:0/888176873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 msgr2=0x7ff644102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 --2- 172.21.15.31:0/888176873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644102a20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff6340098b0 tx=0x7ff63402a4d0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 -- 172.21.15.31:0/888176873 shutdown_connections 2024-09-16T08:59:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 --2- 172.21.15.31:0/888176873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff644102f60 0x7ff644105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 --2- 172.21.15.31:0/888176873 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644102a20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 -- 172.21.15.31:0/888176873 >> 172.21.15.31:0/888176873 conn(0x7ff6440fa250 msgr2=0x7ff6440fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 -- 172.21.15.31:0/888176873 shutdown_connections 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.441+0000 7ff64a439640 1 -- 172.21.15.31:0/888176873 wait complete. 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.442+0000 7ff64a439640 1 Processor -- start 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.442+0000 7ff64a439640 1 -- start start 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.442+0000 7ff64a439640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.442+0000 7ff64a439640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff644102f60 0x7ff644195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.442+0000 7ff64a439640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff644196170 con 0x7ff644100630 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.442+0000 7ff64a439640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6441962e0 con 0x7ff644102f60 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff643fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff643fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43878/0 (socket says 172.21.15.31:43878) 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff643fff640 1 -- 172.21.15.31:0/1001742969 learned_addr learned my addr 172.21.15.31:0/1001742969 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff6437fe640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff644102f60 0x7ff644195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff643fff640 1 -- 172.21.15.31:0/1001742969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff644102f60 msgr2=0x7ff644195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff643fff640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff644102f60 0x7ff644195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff643fff640 1 -- 172.21.15.31:0/1001742969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff634009560 con 0x7ff644100630 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff643fff640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644195660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff634004650 tx=0x7ff6340055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff6417fa640 1 -- 172.21.15.31:0/1001742969 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff634004aa0 con 0x7ff644100630 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff6417fa640 1 -- 172.21.15.31:0/1001742969 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff634005d50 con 0x7ff644100630 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64419ad20 con 0x7ff644100630 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff6417fa640 1 -- 172.21.15.31:0/1001742969 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6340039e0 con 0x7ff644100630 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.443+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64419b240 con 0x7ff644100630 2024-09-16T08:59:59.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.444+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6441087e0 con 0x7ff644100630 2024-09-16T08:59:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.445+0000 7ff6417fa640 1 -- 172.21.15.31:0/1001742969 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7ff634031c00 con 0x7ff644100630 2024-09-16T08:59:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.445+0000 7ff6417fa640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff62c07b050 0x7ff62c07d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.446+0000 7ff6417fa640 1 -- 172.21.15.31:0/1001742969 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff6340bf280 con 0x7ff644100630 2024-09-16T08:59:59.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.446+0000 7ff6437fe640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff62c07b050 0x7ff62c07d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.446+0000 7ff6437fe640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff62c07b050 0x7ff62c07d510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff644196b80 tx=0x7ff638005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:59.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.450+0000 7ff6417fa640 1 -- 172.21.15.31:0/1001742969 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ff6340822c0 con 0x7ff644100630 2024-09-16T08:59:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.657+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff64419b430 con 0x7ff62c07b050 2024-09-16T08:59:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.658+0000 7ff6417fa640 1 -- 172.21.15.31:0/1001742969 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff64419b430 con 0x7ff62c07b050 2024-09-16T08:59:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff62c07b050 msgr2=0x7ff62c07d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff62c07b050 0x7ff62c07d510 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff644196b80 tx=0x7ff638005f90 comp rx=0 tx=0).stop 2024-09-16T08:59:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 msgr2=0x7ff644195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644195660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff634004650 tx=0x7ff6340055a0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 shutdown_connections 2024-09-16T08:59:59.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff62c07b050 0x7ff62c07d510 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff644102f60 0x7ff644195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 --2- 172.21.15.31:0/1001742969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff644100630 0x7ff644195660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.660+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 >> 172.21.15.31:0/1001742969 conn(0x7ff6440fa250 msgr2=0x7ff6440fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:59.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.661+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 shutdown_connections 2024-09-16T08:59:59.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.661+0000 7ff64a439640 1 -- 172.21.15.31:0/1001742969 wait complete. 2024-09-16T08:59:59.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 -- 172.21.15.31:0/3030853671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af4100d90 msgr2=0x7f0af4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 --2- 172.21.15.31:0/3030853671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af4100d90 0x7f0af4101210 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8009880 tx=0x7f0ae802a4a0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 -- 172.21.15.31:0/3030853671 shutdown_connections 2024-09-16T08:59:59.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 --2- 172.21.15.31:0/3030853671 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af4100d90 0x7f0af4101210 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 --2- 172.21.15.31:0/3030853671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af40ffb90 0x7f0af40fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 -- 172.21.15.31:0/3030853671 >> 172.21.15.31:0/3030853671 conn(0x7f0af40fb320 msgr2=0x7f0af40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 -- 172.21.15.31:0/3030853671 shutdown_connections 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.744+0000 7f0afa241640 1 -- 172.21.15.31:0/3030853671 wait complete. 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0afa241640 1 Processor -- start 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0afa241640 1 -- start start 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0afa241640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af40ffb90 0x7f0af4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0afa241640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af4100d90 0x7f0af40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0afa241640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4072de0 con 0x7f0af40ffb90 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0afa241640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4072f50 con 0x7f0af4100d90 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0af37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af40ffb90 0x7f0af4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0af37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af40ffb90 0x7f0af4074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43904/0 (socket says 172.21.15.31:43904) 2024-09-16T08:59:59.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0af37fe640 1 -- 172.21.15.31:0/2867187600 learned_addr learned my addr 172.21.15.31:0/2867187600 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T08:59:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0af2ffd640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af4100d90 0x7f0af40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0af37fe640 1 -- 172.21.15.31:0/2867187600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af4100d90 msgr2=0x7f0af40728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.745+0000 7f0af37fe640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af4100d90 0x7f0af40728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0af37fe640 1 -- 172.21.15.31:0/2867187600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae8009560 con 0x7f0af40ffb90 2024-09-16T08:59:59.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0af2ffd640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af4100d90 0x7f0af40728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T08:59:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0af37fe640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af40ffb90 0x7f0af4074250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b3d0 tx=0x7f0ae400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0af0ff9640 1 -- 172.21.15.31:0/2867187600 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae4011800 con 0x7f0af40ffb90 2024-09-16T08:59:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0af0ff9640 1 -- 172.21.15.31:0/2867187600 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ae4011e20 con 0x7f0af40ffb90 2024-09-16T08:59:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af4073230 con 0x7f0af40ffb90 2024-09-16T08:59:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0af0ff9640 1 -- 172.21.15.31:0/2867187600 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ae400f510 con 0x7f0af40ffb90 2024-09-16T08:59:59.749 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.746+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af419d790 con 0x7f0af40ffb90 2024-09-16T08:59:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.747+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af4100010 con 0x7f0af40ffb90 2024-09-16T08:59:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.748+0000 7f0af0ff9640 1 -- 172.21.15.31:0/2867187600 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f0ae4010380 con 0x7f0af40ffb90 2024-09-16T08:59:59.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.748+0000 7f0af0ff9640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0adc07b060 0x7f0adc07d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T08:59:59.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.749+0000 7f0af0ff9640 1 -- 172.21.15.31:0/2867187600 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f0ae409e170 con 0x7f0af40ffb90 2024-09-16T08:59:59.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.749+0000 7f0af2ffd640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0adc07b060 0x7f0adc07d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T08:59:59.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.749+0000 7f0af2ffd640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0adc07b060 0x7f0adc07d520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8000c00 tx=0x7f0ae80314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T08:59:59.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.752+0000 7f0af0ff9640 1 -- 172.21.15.31:0/2867187600 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f0ae4061180 con 0x7f0af40ffb90 2024-09-16T08:59:59.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.946+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0af41056e0 con 0x7f0adc07b060 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.956+0000 7f0af0ff9640 1 -- 172.21.15.31:0/2867187600 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0af41056e0 con 0x7f0adc07b060 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (9m) 30s ago 11m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (2m) 30s ago 11m 7654k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (2m) 20s ago 10m 7524k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (11m) 30s ago 11m 6895k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e6dfb1b773c3 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (10m) 20s ago 10m 6891k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 81521a1b0825 2024-09-16T08:59:59.957 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (9m) 30s ago 11m 87.2M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (5m) 30s ago 5m 14.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (5m) 30s ago 5m 155M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (5m) 20s ago 5m 26.7M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (5m) 20s ago 5m 17.1M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (2m) 30s ago 12m 487M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (55s) 20s ago 10m 452M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (43s) 30s ago 12m 32.2M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (24s) 20s ago 10m 22.5M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (11m) 30s ago 11m 15.6M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (10m) 20s ago 10m 16.4M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 30s ago 8m 248M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (8m) 30s ago 8m 264M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T08:59:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (8m) 30s ago 8m 222M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T08:59:59.959 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (7m) 20s ago 7m 278M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T08:59:59.959 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (7m) 20s ago 7m 249M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T08:59:59.959 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (6m) 20s ago 6m 233M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T08:59:59.959 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (61s) 30s ago 10m 40.9M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T08:59:59.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0adc07b060 msgr2=0x7f0adc07d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0adc07b060 0x7f0adc07d520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8000c00 tx=0x7f0ae80314e0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af40ffb90 msgr2=0x7f0af4074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af40ffb90 0x7f0af4074250 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ae400b3d0 tx=0x7f0ae400b890 comp rx=0 tx=0).stop 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 shutdown_connections 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0adc07b060 0x7f0adc07d520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0af4100d90 0x7f0af40728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 --2- 172.21.15.31:0/2867187600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0af40ffb90 0x7f0af4074250 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 >> 172.21.15.31:0/2867187600 conn(0x7f0af40fb320 msgr2=0x7f0af4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 shutdown_connections 2024-09-16T08:59:59.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T08:59:59.958+0000 7f0afa241640 1 -- 172.21.15.31:0/2867187600 wait complete. 2024-09-16T09:00:00.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 -- 172.21.15.31:0/700201597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 msgr2=0x7fe0101053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 --2- 172.21.15.31:0/700201597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe0101053a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0040098b0 tx=0x7fe00402a4d0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 -- 172.21.15.31:0/700201597 shutdown_connections 2024-09-16T09:00:00.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 --2- 172.21.15.31:0/700201597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe0101053a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 --2- 172.21.15.31:0/700201597 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe010100680 0x7fe010102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 -- 172.21.15.31:0/700201597 >> 172.21.15.31:0/700201597 conn(0x7fe0100fa280 msgr2=0x7fe0100fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:00.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 -- 172.21.15.31:0/700201597 shutdown_connections 2024-09-16T09:00:00.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.040+0000 7fe016b3d640 1 -- 172.21.15.31:0/700201597 wait complete. 2024-09-16T09:00:00.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.041+0000 7fe016b3d640 1 Processor -- start 2024-09-16T09:00:00.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.041+0000 7fe016b3d640 1 -- start start 2024-09-16T09:00:00.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.041+0000 7fe016b3d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe010100680 0x7fe0101956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.041+0000 7fe016b3d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe010195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.041+0000 7fe016b3d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010196200 con 0x7fe010102fb0 2024-09-16T09:00:00.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.041+0000 7fe016b3d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010196370 con 0x7fe010100680 2024-09-16T09:00:00.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe010195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe010195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43928/0 (socket says 172.21.15.31:43928) 2024-09-16T09:00:00.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00ffff640 1 -- 172.21.15.31:0/28829020 learned_addr learned my addr 172.21.15.31:0/28829020 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:00.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe0148d9640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe010100680 0x7fe0101956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00ffff640 1 -- 172.21.15.31:0/28829020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe010100680 msgr2=0x7fe0101956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00ffff640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe010100680 0x7fe0101956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00ffff640 1 -- 172.21.15.31:0/28829020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe004009560 con 0x7fe010102fb0 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00ffff640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe010195c30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0040055a0 tx=0x7fe0040055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00dffb640 1 -- 172.21.15.31:0/28829020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe004004aa0 con 0x7fe010102fb0 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00dffb640 1 -- 172.21.15.31:0/28829020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe004004c00 con 0x7fe010102fb0 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01019adb0 con 0x7fe010102fb0 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.042+0000 7fe00dffb640 1 -- 172.21.15.31:0/28829020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe004003b50 con 0x7fe010102fb0 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.043+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01019b2a0 con 0x7fe010102fb0 2024-09-16T09:00:00.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.044+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0101087e0 con 0x7fe010102fb0 2024-09-16T09:00:00.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.044+0000 7fe00dffb640 1 -- 172.21.15.31:0/28829020 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7fe004003cb0 con 0x7fe010102fb0 2024-09-16T09:00:00.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.045+0000 7fe00dffb640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdff807b060 0x7fdff807d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.045+0000 7fe00dffb640 1 -- 172.21.15.31:0/28829020 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe0040bf290 con 0x7fe010102fb0 2024-09-16T09:00:00.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.045+0000 7fe0148d9640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdff807b060 0x7fdff807d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.045+0000 7fe0148d9640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdff807b060 0x7fdff807d520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe000005d00 tx=0x7fe00000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:00.053 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.049+0000 7fe00dffb640 1 -- 172.21.15.31:0/28829020 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fe004082350 con 0x7fe010102fb0 2024-09-16T09:00:00.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.311+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe01019b4e0 con 0x7fe010102fb0 2024-09-16T09:00:00.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.311+0000 7fe00dffb640 1 -- 172.21.15.31:0/28829020 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7fe01019b4e0 con 0x7fe010102fb0 2024-09-16T09:00:00.313 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:00:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:00:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:00:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:00.313 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:00:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:00:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:00:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T09:00:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:00.314 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:00:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:00:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:00:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 10, 2024-09-16T09:00:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:00:00.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:00:00.315 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:00:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdff807b060 msgr2=0x7fdff807d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdff807b060 0x7fdff807d520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe000005d00 tx=0x7fe00000b410 comp rx=0 tx=0).stop 2024-09-16T09:00:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 msgr2=0x7fe010195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe010195c30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe0040055a0 tx=0x7fe0040055d0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 shutdown_connections 2024-09-16T09:00:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdff807b060 0x7fdff807d520 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe010102fb0 0x7fe010195c30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 --2- 172.21.15.31:0/28829020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe010100680 0x7fe0101956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 >> 172.21.15.31:0/28829020 conn(0x7fe0100fa280 msgr2=0x7fe0100fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 shutdown_connections 2024-09-16T09:00:00.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.314+0000 7fe016b3d640 1 -- 172.21.15.31:0/28829020 wait complete. 2024-09-16T09:00:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 -- 172.21.15.31:0/3025155078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 msgr2=0x7fa9cc073370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/3025155078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc073370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009880 tx=0x7fa9c002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 -- 172.21.15.31:0/3025155078 shutdown_connections 2024-09-16T09:00:00.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/3025155078 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc073370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/3025155078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9cc0744e0 0x7fa9cc072940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 -- 172.21.15.31:0/3025155078 >> 172.21.15.31:0/3025155078 conn(0x7fa9cc0fb320 msgr2=0x7fa9cc0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 -- 172.21.15.31:0/3025155078 shutdown_connections 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 -- 172.21.15.31:0/3025155078 wait complete. 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 Processor -- start 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.398+0000 7fa9d35f8640 1 -- start start 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d35f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc1977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d35f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9cc0744e0 0x7fa9cc197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d35f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1982e0 con 0x7fa9cc072f10 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d35f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc198450 con 0x7fa9cc0744e0 2024-09-16T09:00:00.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d1394640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc1977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d1394640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc1977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43944/0 (socket says 172.21.15.31:43944) 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d1394640 1 -- 172.21.15.31:0/719121424 learned_addr learned my addr 172.21.15.31:0/719121424 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d0b93640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9cc0744e0 0x7fa9cc197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d1394640 1 -- 172.21.15.31:0/719121424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9cc0744e0 msgr2=0x7fa9cc197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d1394640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9cc0744e0 0x7fa9cc197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d1394640 1 -- 172.21.15.31:0/719121424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0009560 con 0x7fa9cc072f10 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d0b93640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9cc0744e0 0x7fa9cc197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d1394640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc1977d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00eb70 tx=0x7fa9bc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9ca7fc640 1 -- 172.21.15.31:0/719121424 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc004b20 con 0x7fa9cc072f10 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9ca7fc640 1 -- 172.21.15.31:0/719121424 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9bc004c80 con 0x7fa9cc072f10 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc19cef0 con 0x7fa9cc072f10 2024-09-16T09:00:00.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9ca7fc640 1 -- 172.21.15.31:0/719121424 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc010700 con 0x7fa9cc072f10 2024-09-16T09:00:00.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.399+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc19d430 con 0x7fa9cc072f10 2024-09-16T09:00:00.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.400+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc10a9b0 con 0x7fa9cc072f10 2024-09-16T09:00:00.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.401+0000 7fa9ca7fc640 1 -- 172.21.15.31:0/719121424 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7fa9bc005020 con 0x7fa9cc072f10 2024-09-16T09:00:00.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.402+0000 7fa9ca7fc640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b407b060 0x7fa9b407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.402+0000 7fa9ca7fc640 1 -- 172.21.15.31:0/719121424 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fa9bc014070 con 0x7fa9cc072f10 2024-09-16T09:00:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.402+0000 7fa9d0b93640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b407b060 0x7fa9b407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.402+0000 7fa9d0b93640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b407b060 0x7fa9b407d520 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000c00 tx=0x7fa9c00314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:00.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.406+0000 7fa9ca7fc640 1 -- 172.21.15.31:0/719121424 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fa9bc061a60 con 0x7fa9cc072f10 2024-09-16T09:00:00.498 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: Reconfiguring mgr.smithi097.mpashf (monmap changed)... 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: Reconfiguring daemon mgr.smithi097.mpashf on smithi097 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: pgmap v21: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:00.499 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:00.500 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:00.500 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:00.500 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:00 smithi097 ceph-mon[79892]: overall HEALTH_OK 2024-09-16T09:00:00.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: Reconfiguring mgr.smithi097.mpashf (monmap changed)... 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mgr.smithi097.mpashf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: Reconfiguring daemon mgr.smithi097.mpashf on smithi097 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: pgmap v21: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:00.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:00 smithi031 ceph-mon[91884]: overall HEALTH_OK 2024-09-16T09:00:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.629+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa9cc19d680 con 0x7fa9cc072f10 2024-09-16T09:00:00.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.629+0000 7fa9ca7fc640 1 -- 172.21.15.31:0/719121424 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7fa9bc064b80 con 0x7fa9cc072f10 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:00:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:00:00.632 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:00.633 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:00.634 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:00.634 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:00:00.634 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:00.634 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:00.634 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:00.634 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:00:00.634 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b407b060 msgr2=0x7fa9b407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b407b060 0x7fa9b407d520 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0000c00 tx=0x7fa9c00314e0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 msgr2=0x7fa9cc1977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc1977d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00eb70 tx=0x7fa9bc00c6a0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 shutdown_connections 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b407b060 0x7fa9b407d520 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9cc0744e0 0x7fa9cc197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 --2- 172.21.15.31:0/719121424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9cc072f10 0x7fa9cc1977d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.631+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 >> 172.21.15.31:0/719121424 conn(0x7fa9cc0fb320 msgr2=0x7fa9cc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.632+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 shutdown_connections 2024-09-16T09:00:00.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.632+0000 7fa9d35f8640 1 -- 172.21.15.31:0/719121424 wait complete. 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 -- 172.21.15.31:0/2798223513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 msgr2=0x7f868c105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 --2- 172.21.15.31:0/2798223513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c105350 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86800098b0 tx=0x7f868002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 -- 172.21.15.31:0/2798223513 shutdown_connections 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 --2- 172.21.15.31:0/2798223513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c105350 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 --2- 172.21.15.31:0/2798223513 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f868c100630 0x7f868c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 -- 172.21.15.31:0/2798223513 >> 172.21.15.31:0/2798223513 conn(0x7f868c0fa250 msgr2=0x7f868c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 -- 172.21.15.31:0/2798223513 shutdown_connections 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.716+0000 7f8692840640 1 -- 172.21.15.31:0/2798223513 wait complete. 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f8692840640 1 Processor -- start 2024-09-16T09:00:00.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f8692840640 1 -- start start 2024-09-16T09:00:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f8692840640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f868c100630 0x7f868c195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f8692840640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f8692840640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c196160 con 0x7f868c102f60 2024-09-16T09:00:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f8692840640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868c1962d0 con 0x7f868c100630 2024-09-16T09:00:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f868b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c195b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43962/0 (socket says 172.21.15.31:43962) 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f868b7fe640 1 -- 172.21.15.31:0/1294657999 learned_addr learned my addr 172.21.15.31:0/1294657999 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.717+0000 7f868bfff640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f868c100630 0x7f868c195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f868b7fe640 1 -- 172.21.15.31:0/1294657999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f868c100630 msgr2=0x7f868c195650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f868b7fe640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f868c100630 0x7f868c195650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f868b7fe640 1 -- 172.21.15.31:0/1294657999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680009560 con 0x7f868c102f60 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f868b7fe640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c195b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86800055a0 tx=0x7f86800055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f86897fa640 1 -- 172.21.15.31:0/1294657999 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8680004aa0 con 0x7f868c102f60 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868c19ad10 con 0x7f868c102f60 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f86897fa640 1 -- 172.21.15.31:0/1294657999 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8680004c00 con 0x7f868c102f60 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f86897fa640 1 -- 172.21.15.31:0/1294657999 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8680003b50 con 0x7f868c102f60 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.718+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868c19b200 con 0x7f868c102f60 2024-09-16T09:00:00.721 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.719+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868c108790 con 0x7f868c102f60 2024-09-16T09:00:00.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.720+0000 7f86897fa640 1 -- 172.21.15.31:0/1294657999 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f8680003cb0 con 0x7f868c102f60 2024-09-16T09:00:00.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.720+0000 7f86897fa640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f867407b060 0x7f867407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:00.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.720+0000 7f86897fa640 1 -- 172.21.15.31:0/1294657999 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f86800bf380 con 0x7f868c102f60 2024-09-16T09:00:00.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.720+0000 7f868bfff640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f867407b060 0x7f867407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:00.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.721+0000 7f868bfff640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f867407b060 0x7f867407d520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f867c005d00 tx=0x7f867c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.725+0000 7f86897fa640 1 -- 172.21.15.31:0/1294657999 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f86800823c0 con 0x7f868c102f60 2024-09-16T09:00:00.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.929+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f868c1039c0 con 0x7f867407b060 2024-09-16T09:00:00.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.930+0000 7f86897fa640 1 -- 172.21.15.31:0/1294657999 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f868c1039c0 con 0x7f867407b060 2024-09-16T09:00:00.931 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:00:00.931 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "4/23 daemons upgraded", 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mon daemons", 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:00:00.932 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f867407b060 msgr2=0x7f867407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f867407b060 0x7f867407d520 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f867c005d00 tx=0x7f867c00b410 comp rx=0 tx=0).stop 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 msgr2=0x7f868c195b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c195b90 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86800055a0 tx=0x7f86800055d0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 shutdown_connections 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f867407b060 0x7f867407d520 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f868c102f60 0x7f868c195b90 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 --2- 172.21.15.31:0/1294657999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f868c100630 0x7f868c195650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 >> 172.21.15.31:0/1294657999 conn(0x7f868c0fa250 msgr2=0x7f868c0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 shutdown_connections 2024-09-16T09:00:00.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:00.933+0000 7f8692840640 1 -- 172.21.15.31:0/1294657999 wait complete. 2024-09-16T09:00:01.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 -- 172.21.15.31:0/3089029288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 msgr2=0x7f3e6c0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:01.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 --2- 172.21.15.31:0/3089029288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c0feea0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009880 tx=0x7f3e6802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 -- 172.21.15.31:0/3089029288 shutdown_connections 2024-09-16T09:00:01.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 --2- 172.21.15.31:0/3089029288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0ffca0 0x7f3e6c100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 --2- 172.21.15.31:0/3089029288 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c0feea0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 -- 172.21.15.31:0/3089029288 >> 172.21.15.31:0/3089029288 conn(0x7f3e6c0fa250 msgr2=0x7f3e6c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 -- 172.21.15.31:0/3089029288 shutdown_connections 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.016+0000 7f3e746df640 1 -- 172.21.15.31:0/3089029288 wait complete. 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e746df640 1 Processor -- start 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e746df640 1 -- start start 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e746df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c195660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e746df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0ffca0 0x7f3e6c195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e746df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c196170 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e746df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c1962e0 con 0x7f3e6c0ffca0 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e7247b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c195660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:01.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e7247b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c195660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43982/0 (socket says 172.21.15.31:43982) 2024-09-16T09:00:01.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e7247b640 1 -- 172.21.15.31:0/204127467 learned_addr learned my addr 172.21.15.31:0/204127467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:01.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e71c7a640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0ffca0 0x7f3e6c195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:01.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e7247b640 1 -- 172.21.15.31:0/204127467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0ffca0 msgr2=0x7f3e6c195ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:01.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e7247b640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0ffca0 0x7f3e6c195ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.017+0000 7f3e7247b640 1 -- 172.21.15.31:0/204127467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68009560 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.018+0000 7f3e7247b640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c195660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e68004650 tx=0x7f3e680055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:01.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.018+0000 7f3e637fe640 1 -- 172.21.15.31:0/204127467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e68004aa0 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.018+0000 7f3e637fe640 1 -- 172.21.15.31:0/204127467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e68005d50 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.018+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c19ad20 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.021 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.018+0000 7f3e637fe640 1 -- 172.21.15.31:0/204127467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e680039e0 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.018+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c19b240 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.019+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c107690 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.020+0000 7f3e637fe640 1 -- 172.21.15.31:0/204127467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f3e68031c00 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.020+0000 7f3e637fe640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3e5807b050 0x7f3e5807d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:01.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.020+0000 7f3e637fe640 1 -- 172.21.15.31:0/204127467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3e680bf260 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.020+0000 7f3e71c7a640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3e5807b050 0x7f3e5807d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:01.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.021+0000 7f3e71c7a640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3e5807b050 0x7f3e5807d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c196b80 tx=0x7f3e5c005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:01.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.025+0000 7f3e637fe640 1 -- 172.21.15.31:0/204127467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f3e680822a0 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.284+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3e6c19b680 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.285+0000 7f3e637fe640 1 -- 172.21.15.31:0/204127467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3e680853c0 con 0x7f3e6c0feaa0 2024-09-16T09:00:01.286 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:00:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3e5807b050 msgr2=0x7f3e5807d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3e5807b050 0x7f3e5807d510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c196b80 tx=0x7f3e5c005f90 comp rx=0 tx=0).stop 2024-09-16T09:00:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 msgr2=0x7f3e6c195660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c195660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e68004650 tx=0x7f3e680055a0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 shutdown_connections 2024-09-16T09:00:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3e5807b050 0x7f3e5807d510 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3e6c0ffca0 0x7f3e6c195ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 --2- 172.21.15.31:0/204127467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6c0feaa0 0x7f3e6c195660 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 >> 172.21.15.31:0/204127467 conn(0x7f3e6c0fa250 msgr2=0x7f3e6c0684a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 shutdown_connections 2024-09-16T09:00:01.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:01.287+0000 7f3e746df640 1 -- 172.21.15.31:0/204127467 wait complete. 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/28829020' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: Reconfiguring mon.smithi097 (monmap changed)... 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: Reconfiguring daemon mon.smithi097 on smithi097 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/719121424' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:00:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:01 smithi097 ceph-mon[79892]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/28829020' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: Reconfiguring mon.smithi097 (monmap changed)... 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:01.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: Reconfiguring daemon mon.smithi097 on smithi097 2024-09-16T09:00:01.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/719121424' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:00:01.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:01 smithi031 ceph-mon[91884]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:02.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: pgmap v22: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:02.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/204127467' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:00:02.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:02.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:02.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: Reconfiguring osd.3 (monmap changed)... 2024-09-16T09:00:02.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T09:00:02.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:02.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:02 smithi097 ceph-mon[79892]: Reconfiguring daemon osd.3 on smithi097 2024-09-16T09:00:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: pgmap v22: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/204127467' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:00:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: Reconfiguring osd.3 (monmap changed)... 2024-09-16T09:00:02.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T09:00:02.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:02.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:02 smithi031 ceph-mon[91884]: Reconfiguring daemon osd.3 on smithi097 2024-09-16T09:00:04.596 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:04 smithi031 ceph-mon[91884]: pgmap v23: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:04.597 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:04.597 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:04.597 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:04 smithi031 ceph-mon[91884]: Reconfiguring osd.4 (monmap changed)... 2024-09-16T09:00:04.597 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T09:00:04.597 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:04.597 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:04 smithi031 ceph-mon[91884]: Reconfiguring daemon osd.4 on smithi097 2024-09-16T09:00:04.677 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:04 smithi097 ceph-mon[79892]: pgmap v23: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:04.677 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:04.677 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:04.677 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:04 smithi097 ceph-mon[79892]: Reconfiguring osd.4 (monmap changed)... 2024-09-16T09:00:04.677 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T09:00:04.677 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:04.678 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:04 smithi097 ceph-mon[79892]: Reconfiguring daemon osd.4 on smithi097 2024-09-16T09:00:06.050 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:05 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:06.051 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:05 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:06.051 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:05 smithi097 ceph-mon[79892]: Reconfiguring osd.5 (monmap changed)... 2024-09-16T09:00:06.051 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:05 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T09:00:06.051 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:05 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:06.051 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:05 smithi097 ceph-mon[79892]: Reconfiguring daemon osd.5 on smithi097 2024-09-16T09:00:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:05 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:05 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:05 smithi031 ceph-mon[91884]: Reconfiguring osd.5 (monmap changed)... 2024-09-16T09:00:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:05 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T09:00:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:05 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:06.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:05 smithi031 ceph-mon[91884]: Reconfiguring daemon osd.5 on smithi097 2024-09-16T09:00:07.105 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:06 smithi097 ceph-mon[79892]: pgmap v24: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:07.106 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:07.106 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:07.106 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:00:07.106 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:06 smithi031 ceph-mon[91884]: pgmap v24: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:00:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:07.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:07 smithi097 ceph-mon[79892]: Reconfiguring mds.cephfs.smithi097.cagmus (monmap changed)... 2024-09-16T09:00:07.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:07 smithi097 ceph-mon[79892]: Reconfiguring daemon mds.cephfs.smithi097.cagmus on smithi097 2024-09-16T09:00:07.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:07.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:07.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:00:07.993 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:07 smithi031 ceph-mon[91884]: Reconfiguring mds.cephfs.smithi097.cagmus (monmap changed)... 2024-09-16T09:00:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:07 smithi031 ceph-mon[91884]: Reconfiguring daemon mds.cephfs.smithi097.cagmus on smithi097 2024-09-16T09:00:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:08.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:08.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:00:08.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:08 smithi097 ceph-mon[79892]: pgmap v25: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:08 smithi097 ceph-mon[79892]: Reconfiguring mds.cephfs.smithi097.sbwmfq (monmap changed)... 2024-09-16T09:00:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:08 smithi097 ceph-mon[79892]: Reconfiguring daemon mds.cephfs.smithi097.sbwmfq on smithi097 2024-09-16T09:00:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:08 smithi031 ceph-mon[91884]: pgmap v25: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:08 smithi031 ceph-mon[91884]: Reconfiguring mds.cephfs.smithi097.sbwmfq (monmap changed)... 2024-09-16T09:00:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:08 smithi031 ceph-mon[91884]: Reconfiguring daemon mds.cephfs.smithi097.sbwmfq on smithi097 2024-09-16T09:00:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:10.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:10.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all mon 2024-09-16T09:00:10.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:10.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"} : dispatch 2024-09-16T09:00:10.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]': finished 2024-09-16T09:00:10.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi097"} : dispatch 2024-09-16T09:00:10.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi097"}]': finished 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all mon 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"} : dispatch 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi031"}]': finished 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon.smithi097"} : dispatch 2024-09-16T09:00:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi097"}]': finished 2024-09-16T09:00:11.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:10 smithi031 ceph-mon[91884]: pgmap v26: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:11.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:11.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T09:00:11.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:11.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:10 smithi097 ceph-mon[79892]: pgmap v26: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:11.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:11.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi031", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T09:00:11.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:12.109 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:11 smithi031 ceph-mon[91884]: Upgrade: Updating crash.smithi031 (1/2) 2024-09-16T09:00:12.109 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:11 smithi031 ceph-mon[91884]: Deploying daemon crash.smithi031 on smithi031 2024-09-16T09:00:12.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:11 smithi097 ceph-mon[79892]: Upgrade: Updating crash.smithi031 (1/2) 2024-09-16T09:00:12.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:11 smithi097 ceph-mon[79892]: Deploying daemon crash.smithi031 on smithi031 2024-09-16T09:00:13.042 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:12 smithi031 ceph-mon[91884]: pgmap v27: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:13.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:12 smithi097 ceph-mon[79892]: pgmap v27: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:14 smithi031 ceph-mon[91884]: pgmap v28: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:15.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:14 smithi097 ceph-mon[79892]: pgmap v28: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:16.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:16 smithi031 ceph-mon[91884]: Upgrade: Updating crash.smithi097 (2/2) 2024-09-16T09:00:16.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:16.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T09:00:16.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:16.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:16 smithi031 ceph-mon[91884]: Deploying daemon crash.smithi097 on smithi097 2024-09-16T09:00:16.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:16 smithi031 ceph-mon[91884]: pgmap v29: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:16 smithi097 ceph-mon[79892]: Upgrade: Updating crash.smithi097 (2/2) 2024-09-16T09:00:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.crash.smithi097", "caps": ["mon", "profile crash", "mgr", "profile crash"]} : dispatch 2024-09-16T09:00:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:16 smithi097 ceph-mon[79892]: Deploying daemon crash.smithi097 on smithi097 2024-09-16T09:00:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:16 smithi097 ceph-mon[79892]: pgmap v29: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:18.566 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:18 smithi097 ceph-mon[79892]: pgmap v30: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:18.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:18 smithi031 ceph-mon[91884]: pgmap v30: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:19.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:19.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:19.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:20.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:20.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:20.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:21.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:20 smithi097 ceph-mon[79892]: pgmap v31: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:21.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:20 smithi031 ceph-mon[91884]: pgmap v31: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:22.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:22 smithi097 ceph-mon[79892]: pgmap v32: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:22.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:22.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:22.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:22.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:22 smithi031 ceph-mon[91884]: pgmap v32: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:24 smithi097 ceph-mon[79892]: pgmap v33: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.847 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:24 smithi031 ceph-mon[91884]: pgmap v33: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:24.847 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.847 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.847 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:24.847 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:24.847 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:26.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:26.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:26.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:26 smithi031 ceph-mon[91884]: pgmap v34: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:26.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:26.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:26.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:26 smithi097 ceph-mon[79892]: pgmap v34: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: pgmap v35: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all crash 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"} : dispatch 2024-09-16T09:00:29.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]': finished 2024-09-16T09:00:29.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi097"} : dispatch 2024-09-16T09:00:29.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi097"}]': finished 2024-09-16T09:00:29.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-16T09:00:29.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T09:00:29.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:28 smithi097 ceph-mon[79892]: Upgrade: osd.0 is safe to restart 2024-09-16T09:00:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: pgmap v35: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all crash 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi031"}]': finished 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi097"} : dispatch 2024-09-16T09:00:29.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi097"}]': finished 2024-09-16T09:00:29.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16} : dispatch 2024-09-16T09:00:29.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-16T09:00:29.125 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:28 smithi031 ceph-mon[91884]: Upgrade: osd.0 is safe to restart 2024-09-16T09:00:30.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:30 smithi097 ceph-mon[79892]: Upgrade: Updating osd.0 2024-09-16T09:00:30.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:30.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T09:00:30.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:30.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:30 smithi097 ceph-mon[79892]: Deploying daemon osd.0 on smithi031 2024-09-16T09:00:30.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:30 smithi097 ceph-mon[79892]: pgmap v36: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:30 smithi031 ceph-mon[91884]: Upgrade: Updating osd.0 2024-09-16T09:00:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T09:00:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:30 smithi031 ceph-mon[91884]: Deploying daemon osd.0 on smithi031 2024-09-16T09:00:30.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:30 smithi031 ceph-mon[91884]: pgmap v36: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 -- 172.21.15.31:0/2617113777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 msgr2=0x7f40ac0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 --2- 172.21.15.31:0/2617113777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac0fff90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f40a8009c20 tx=0x7f40a802a970 comp rx=0 tx=0).stop 2024-09-16T09:00:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 -- 172.21.15.31:0/2617113777 shutdown_connections 2024-09-16T09:00:31.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 --2- 172.21.15.31:0/2617113777 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac100d90 0x7f40ac101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 --2- 172.21.15.31:0/2617113777 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac0fff90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 -- 172.21.15.31:0/2617113777 >> 172.21.15.31:0/2617113777 conn(0x7f40ac0fb320 msgr2=0x7f40ac0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 -- 172.21.15.31:0/2617113777 shutdown_connections 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.377+0000 7f40b4446640 1 -- 172.21.15.31:0/2617113777 wait complete. 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b4446640 1 Processor -- start 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b4446640 1 -- start start 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b4446640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b4446640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac100d90 0x7f40ac0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b4446640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac072de0 con 0x7f40ac0ffb90 2024-09-16T09:00:31.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b4446640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac072f50 con 0x7f40ac100d90 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b21e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b21e2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35774/0 (socket says 172.21.15.31:35774) 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b21e2640 1 -- 172.21.15.31:0/2437591768 learned_addr learned my addr 172.21.15.31:0/2437591768 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b19e1640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac100d90 0x7f40ac0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b21e2640 1 -- 172.21.15.31:0/2437591768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac100d90 msgr2=0x7f40ac0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b21e2640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac100d90 0x7f40ac0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.378+0000 7f40b21e2640 1 -- 172.21.15.31:0/2437591768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8009900 con 0x7f40ac0ffb90 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.379+0000 7f40b21e2640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac074250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40a802fd20 tx=0x7f40a802f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.379+0000 7f40a37fe640 1 -- 172.21.15.31:0/2437591768 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a8030840 con 0x7f40ac0ffb90 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.379+0000 7f40a37fe640 1 -- 172.21.15.31:0/2437591768 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40a802f9f0 con 0x7f40ac0ffb90 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.379+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac0731d0 con 0x7f40ac0ffb90 2024-09-16T09:00:31.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.379+0000 7f40a37fe640 1 -- 172.21.15.31:0/2437591768 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a8039680 con 0x7f40ac0ffb90 2024-09-16T09:00:31.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.379+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac0736c0 con 0x7f40ac0ffb90 2024-09-16T09:00:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.380+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac18f710 con 0x7f40ac0ffb90 2024-09-16T09:00:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.381+0000 7f40a37fe640 1 -- 172.21.15.31:0/2437591768 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f40a800a040 con 0x7f40ac0ffb90 2024-09-16T09:00:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.381+0000 7f40a37fe640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f409807b050 0x7f409807d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.381+0000 7f40a37fe640 1 -- 172.21.15.31:0/2437591768 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f40a802c040 con 0x7f40ac0ffb90 2024-09-16T09:00:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.381+0000 7f40b19e1640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f409807b050 0x7f409807d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.382+0000 7f40b19e1640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f409807b050 0x7f409807d510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f409c005fd0 tx=0x7f409c005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:31.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.386+0000 7f40a37fe640 1 -- 172.21.15.31:0/2437591768 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f40a80825a0 con 0x7f40ac0ffb90 2024-09-16T09:00:31.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.589+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40ac0608f0 con 0x7f409807b050 2024-09-16T09:00:31.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.590+0000 7f40a37fe640 1 -- 172.21.15.31:0/2437591768 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f40ac0608f0 con 0x7f409807b050 2024-09-16T09:00:31.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f409807b050 msgr2=0x7f409807d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f409807b050 0x7f409807d510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f409c005fd0 tx=0x7f409c005c70 comp rx=0 tx=0).stop 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 msgr2=0x7f40ac074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac074250 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f40a802fd20 tx=0x7f40a802f5f0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 shutdown_connections 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f409807b050 0x7f409807d510 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40ac100d90 0x7f40ac0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 --2- 172.21.15.31:0/2437591768 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40ac0ffb90 0x7f40ac074250 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 >> 172.21.15.31:0/2437591768 conn(0x7f40ac0fb320 msgr2=0x7f40ac104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.592+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 shutdown_connections 2024-09-16T09:00:31.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.593+0000 7f40b4446640 1 -- 172.21.15.31:0/2437591768 wait complete. 2024-09-16T09:00:31.602 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 -- 172.21.15.31:0/605417586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 msgr2=0x7f22cc0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 --2- 172.21.15.31:0/605417586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc0feea0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009880 tx=0x7f22bc02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 -- 172.21.15.31:0/605417586 shutdown_connections 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 --2- 172.21.15.31:0/605417586 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22cc0ffca0 0x7f22cc100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 --2- 172.21.15.31:0/605417586 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc0feea0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 -- 172.21.15.31:0/605417586 >> 172.21.15.31:0/605417586 conn(0x7f22cc0fa250 msgr2=0x7f22cc0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 -- 172.21.15.31:0/605417586 shutdown_connections 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.677+0000 7f22d133b640 1 -- 172.21.15.31:0/605417586 wait complete. 2024-09-16T09:00:31.679 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22d133b640 1 Processor -- start 2024-09-16T09:00:31.680 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22d133b640 1 -- start start 2024-09-16T09:00:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22d133b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22d133b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22cc0ffca0 0x7f22cc195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22d133b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc1961a0 con 0x7f22cc0feaa0 2024-09-16T09:00:31.681 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22d133b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc196310 con 0x7f22cc0ffca0 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22caffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22caffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc195690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35796/0 (socket says 172.21.15.31:35796) 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22caffd640 1 -- 172.21.15.31:0/2907784409 learned_addr learned my addr 172.21.15.31:0/2907784409 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22ca7fc640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22cc0ffca0 0x7f22cc195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22caffd640 1 -- 172.21.15.31:0/2907784409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22cc0ffca0 msgr2=0x7f22cc195bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22caffd640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22cc0ffca0 0x7f22cc195bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22caffd640 1 -- 172.21.15.31:0/2907784409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc009560 con 0x7f22cc0feaa0 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.678+0000 7f22ca7fc640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22cc0ffca0 0x7f22cc195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.679+0000 7f22caffd640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc195690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22bc02a9c0 tx=0x7f22bc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.679+0000 7f22b3fff640 1 -- 172.21.15.31:0/2907784409 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22bc004aa0 con 0x7f22cc0feaa0 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.679+0000 7f22b3fff640 1 -- 172.21.15.31:0/2907784409 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22bc005d50 con 0x7f22cc0feaa0 2024-09-16T09:00:31.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.679+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc19ad50 con 0x7f22cc0feaa0 2024-09-16T09:00:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.679+0000 7f22b3fff640 1 -- 172.21.15.31:0/2907784409 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22bc0039e0 con 0x7f22cc0feaa0 2024-09-16T09:00:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.679+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc19b260 con 0x7f22cc0feaa0 2024-09-16T09:00:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.680+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc1076b0 con 0x7f22cc0feaa0 2024-09-16T09:00:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.681+0000 7f22b3fff640 1 -- 172.21.15.31:0/2907784409 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f22bc031c00 con 0x7f22cc0feaa0 2024-09-16T09:00:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.681+0000 7f22b3fff640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f22b407b060 0x7f22b407d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.681+0000 7f22b3fff640 1 -- 172.21.15.31:0/2907784409 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f22bc0bf250 con 0x7f22cc0feaa0 2024-09-16T09:00:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.681+0000 7f22ca7fc640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f22b407b060 0x7f22b407d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.684 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.682+0000 7f22ca7fc640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f22b407b060 0x7f22b407d520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22cc196bb0 tx=0x7f22c0005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:31.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.686+0000 7f22b3fff640 1 -- 172.21.15.31:0/2907784409 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f22bc0821e0 con 0x7f22cc0feaa0 2024-09-16T09:00:31.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.890+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22cc068bf0 con 0x7f22b407b060 2024-09-16T09:00:31.892 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.891+0000 7f22b3fff640 1 -- 172.21.15.31:0/2907784409 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f22cc068bf0 con 0x7f22b407b060 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f22b407b060 msgr2=0x7f22b407d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f22b407b060 0x7f22b407d520 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f22cc196bb0 tx=0x7f22c0005f90 comp rx=0 tx=0).stop 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 msgr2=0x7f22cc195690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc195690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f22bc02a9c0 tx=0x7f22bc0055a0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 shutdown_connections 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f22b407b060 0x7f22b407d520 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22cc0ffca0 0x7f22cc195bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 --2- 172.21.15.31:0/2907784409 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22cc0feaa0 0x7f22cc195690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.895 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 >> 172.21.15.31:0/2907784409 conn(0x7f22cc0fa250 msgr2=0x7f22cc0684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:31.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 shutdown_connections 2024-09-16T09:00:31.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.894+0000 7f22d133b640 1 -- 172.21.15.31:0/2907784409 wait complete. 2024-09-16T09:00:31.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 -- 172.21.15.31:0/2523012383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 msgr2=0x7f78dc105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 --2- 172.21.15.31:0/2523012383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc105350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78c80098b0 tx=0x7f78c802a4d0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 -- 172.21.15.31:0/2523012383 shutdown_connections 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 --2- 172.21.15.31:0/2523012383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc105350 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 --2- 172.21.15.31:0/2523012383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78dc100630 0x7f78dc102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 -- 172.21.15.31:0/2523012383 >> 172.21.15.31:0/2523012383 conn(0x7f78dc0fa250 msgr2=0x7f78dc0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 -- 172.21.15.31:0/2523012383 shutdown_connections 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.979+0000 7f78e2cee640 1 -- 172.21.15.31:0/2523012383 wait complete. 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78e2cee640 1 Processor -- start 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78e2cee640 1 -- start start 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78e2cee640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78dc100630 0x7f78dc195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78e2cee640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78e2cee640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc196260 con 0x7f78dc102f60 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78e2cee640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78dc1963d0 con 0x7f78dc100630 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78dbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc195c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35818/0 (socket says 172.21.15.31:35818) 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78dbfff640 1 -- 172.21.15.31:0/626974192 learned_addr learned my addr 172.21.15.31:0/626974192 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78e0a8a640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78dc100630 0x7f78dc195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.980+0000 7f78dbfff640 1 -- 172.21.15.31:0/626974192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78dc100630 msgr2=0x7f78dc195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78dbfff640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78dc100630 0x7f78dc195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78dbfff640 1 -- 172.21.15.31:0/626974192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8009560 con 0x7f78dc102f60 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78dbfff640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc195c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009230 tx=0x7f78c8005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:31.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78d9ffb640 1 -- 172.21.15.31:0/626974192 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c8004b20 con 0x7f78dc102f60 2024-09-16T09:00:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78d9ffb640 1 -- 172.21.15.31:0/626974192 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78c8004c80 con 0x7f78dc102f60 2024-09-16T09:00:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78dc19ae10 con 0x7f78dc102f60 2024-09-16T09:00:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78d9ffb640 1 -- 172.21.15.31:0/626974192 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c8003bc0 con 0x7f78dc102f60 2024-09-16T09:00:31.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.981+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78dc19b350 con 0x7f78dc102f60 2024-09-16T09:00:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.982+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78dc108810 con 0x7f78dc102f60 2024-09-16T09:00:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.983+0000 7f78d9ffb640 1 -- 172.21.15.31:0/626974192 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f78c8031640 con 0x7f78dc102f60 2024-09-16T09:00:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.983+0000 7f78d9ffb640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f78cc07b050 0x7f78cc07d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.983+0000 7f78d9ffb640 1 -- 172.21.15.31:0/626974192 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f78c80bf450 con 0x7f78dc102f60 2024-09-16T09:00:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.983+0000 7f78e0a8a640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f78cc07b050 0x7f78cc07d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:31.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.984+0000 7f78e0a8a640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f78cc07b050 0x7f78cc07d510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009890 tx=0x7f78d0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:31.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:31.988+0000 7f78d9ffb640 1 -- 172.21.15.31:0/626974192 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f78c80823e0 con 0x7f78dc102f60 2024-09-16T09:00:32.185 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.183+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f78dc1039a0 con 0x7f78cc07b050 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.194+0000 7f78d9ffb640 1 -- 172.21.15.31:0/626974192 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f78dc1039a0 con 0x7f78cc07b050 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (10m) 8s ago 11m 25.4M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (2m) 8s ago 12m 7726k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (2m) 10s ago 11m 7578k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (18s) 8s ago 12m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (13s) 10s ago 11m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (9m) 8s ago 11m 84.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (5m) 8s ago 5m 15.6M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (5m) 8s ago 5m 152M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (5m) 10s ago 5m 26.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (5m) 10s ago 5m 17.4M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:00:32.195 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (2m) 8s ago 13m 507M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (88s) 10s ago 11m 452M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (76s) 8s ago 13m 43.5M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (56s) 10s ago 10m 33.6M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (12m) 8s ago 12m 15.5M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (11m) 10s ago 11m 16.4M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (9m) 8s ago 9m 249M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 cefa539e811b 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (9m) 8s ago 9m 265M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (8m) 8s ago 8m 223M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (8m) 10s ago 8m 278M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (7m) 10s ago 7m 250M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (7m) 10s ago 7m 233M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T09:00:32.196 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (93s) 8s ago 11m 51.0M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:00:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f78cc07b050 msgr2=0x7f78cc07d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f78cc07b050 0x7f78cc07d510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009890 tx=0x7f78d0008040 comp rx=0 tx=0).stop 2024-09-16T09:00:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 msgr2=0x7f78dc195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc195c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f78c8009230 tx=0x7f78c8005610 comp rx=0 tx=0).stop 2024-09-16T09:00:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 shutdown_connections 2024-09-16T09:00:32.200 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f78cc07b050 0x7f78cc07d510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78dc102f60 0x7f78dc195c90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 --2- 172.21.15.31:0/626974192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78dc100630 0x7f78dc195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 >> 172.21.15.31:0/626974192 conn(0x7f78dc0fa250 msgr2=0x7f78dc0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 shutdown_connections 2024-09-16T09:00:32.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.196+0000 7f78e2cee640 1 -- 172.21.15.31:0/626974192 wait complete. 2024-09-16T09:00:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 -- 172.21.15.31:0/3916067868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954100d90 msgr2=0x7fc954101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 --2- 172.21.15.31:0/3916067868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954100d90 0x7fc954101210 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc944009880 tx=0x7fc94402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 -- 172.21.15.31:0/3916067868 shutdown_connections 2024-09-16T09:00:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 --2- 172.21.15.31:0/3916067868 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc954100d90 0x7fc954101210 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 --2- 172.21.15.31:0/3916067868 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9540ffb90 0x7fc9540fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 -- 172.21.15.31:0/3916067868 >> 172.21.15.31:0/3916067868 conn(0x7fc9540fb360 msgr2=0x7fc9540fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 -- 172.21.15.31:0/3916067868 shutdown_connections 2024-09-16T09:00:32.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.290+0000 7fc95be6f640 1 -- 172.21.15.31:0/3916067868 wait complete. 2024-09-16T09:00:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc95be6f640 1 Processor -- start 2024-09-16T09:00:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc95be6f640 1 -- start start 2024-09-16T09:00:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc95be6f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9540ffb90 0x7fc954074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc95be6f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc954100d90 0x7fc9540728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc95be6f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc954072de0 con 0x7fc9540ffb90 2024-09-16T09:00:32.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc95be6f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc954072f50 con 0x7fc954100d90 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc959c0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9540ffb90 0x7fc954074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc959c0b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9540ffb90 0x7fc954074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35850/0 (socket says 172.21.15.31:35850) 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc959c0b640 1 -- 172.21.15.31:0/2395164893 learned_addr learned my addr 172.21.15.31:0/2395164893 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc95940a640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc954100d90 0x7fc9540728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc959c0b640 1 -- 172.21.15.31:0/2395164893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc954100d90 msgr2=0x7fc9540728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc959c0b640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc954100d90 0x7fc9540728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc959c0b640 1 -- 172.21.15.31:0/2395164893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944009560 con 0x7fc9540ffb90 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.291+0000 7fc959c0b640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9540ffb90 0x7fc954074250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc95000e940 tx=0x7fc95000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.292+0000 7fc94affd640 1 -- 172.21.15.31:0/2395164893 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9500096e0 con 0x7fc9540ffb90 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.292+0000 7fc94affd640 1 -- 172.21.15.31:0/2395164893 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9500044f0 con 0x7fc9540ffb90 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.292+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc954073230 con 0x7fc9540ffb90 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.292+0000 7fc94affd640 1 -- 172.21.15.31:0/2395164893 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc950010430 con 0x7fc9540ffb90 2024-09-16T09:00:32.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.292+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95419d670 con 0x7fc9540ffb90 2024-09-16T09:00:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.293+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9540fff90 con 0x7fc9540ffb90 2024-09-16T09:00:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.293+0000 7fc94affd640 1 -- 172.21.15.31:0/2395164893 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7fc95000cc90 con 0x7fc9540ffb90 2024-09-16T09:00:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.294+0000 7fc94affd640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007b060 0x7fc94007d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.294+0000 7fc95940a640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007b060 0x7fc94007d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.294+0000 7fc94affd640 1 -- 172.21.15.31:0/2395164893 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fc950014070 con 0x7fc9540ffb90 2024-09-16T09:00:32.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.295+0000 7fc95940a640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007b060 0x7fc94007d520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc944000c00 tx=0x7fc9440314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:32.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.299+0000 7fc94affd640 1 -- 172.21.15.31:0/2395164893 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fc950061600 con 0x7fc9540ffb90 2024-09-16T09:00:32.475 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:32 smithi031 ceph-mon[91884]: pgmap v37: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:32.475 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:32 smithi031 ceph-mon[91884]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:32.475 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:32 smithi031 ceph-mon[91884]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:32.570 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.569+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc9541087a0 con 0x7fc9540ffb90 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.569+0000 7fc94affd640 1 -- 172.21.15.31:0/2395164893 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+730 (secure 0 0 0) 0x7fc950064720 con 0x7fc9540ffb90 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6 2024-09-16T09:00:32.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 10, 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:00:32.572 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:00:32.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007b060 msgr2=0x7fc94007d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007b060 0x7fc94007d520 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc944000c00 tx=0x7fc9440314e0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9540ffb90 msgr2=0x7fc954074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9540ffb90 0x7fc954074250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc95000e940 tx=0x7fc95000ee00 comp rx=0 tx=0).stop 2024-09-16T09:00:32.573 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 shutdown_connections 2024-09-16T09:00:32.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007b060 0x7fc94007d520 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc954100d90 0x7fc9540728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 --2- 172.21.15.31:0/2395164893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9540ffb90 0x7fc954074250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 >> 172.21.15.31:0/2395164893 conn(0x7fc9540fb360 msgr2=0x7fc954104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:32.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 shutdown_connections 2024-09-16T09:00:32.574 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.572+0000 7fc95be6f640 1 -- 172.21.15.31:0/2395164893 wait complete. 2024-09-16T09:00:32.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- 172.21.15.31:0/3715691727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 msgr2=0x7f58a8102b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 --2- 172.21.15.31:0/3715691727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8102b10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f589c009880 tx=0x7f589c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- 172.21.15.31:0/3715691727 shutdown_connections 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 --2- 172.21.15.31:0/3715691727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a8103050 0x7f58a8105440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 --2- 172.21.15.31:0/3715691727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8102b10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- 172.21.15.31:0/3715691727 >> 172.21.15.31:0/3715691727 conn(0x7f58a80fa280 msgr2=0x7f58a80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- 172.21.15.31:0/3715691727 shutdown_connections 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- 172.21.15.31:0/3715691727 wait complete. 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 Processor -- start 2024-09-16T09:00:32.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- start start 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8100080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a8103050 0x7f58a80fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a81005c0 con 0x7f58a8100720 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.663+0000 7f58aef2c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a80fec40 con 0x7f58a8103050 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58accc8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8100080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58accc8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8100080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35880/0 (socket says 172.21.15.31:35880) 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58accc8640 1 -- 172.21.15.31:0/2674896829 learned_addr learned my addr 172.21.15.31:0/2674896829 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58a7fff640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a8103050 0x7f58a80fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58accc8640 1 -- 172.21.15.31:0/2674896829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a8103050 msgr2=0x7f58a80fe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58accc8640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a8103050 0x7f58a80fe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58accc8640 1 -- 172.21.15.31:0/2674896829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c009560 con 0x7f58a8100720 2024-09-16T09:00:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58accc8640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8100080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f589c005610 tx=0x7f589c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58a5ffb640 1 -- 172.21.15.31:0/2674896829 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f589c004ba0 con 0x7f58a8100720 2024-09-16T09:00:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58a5ffb640 1 -- 172.21.15.31:0/2674896829 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f589c004d00 con 0x7f58a8100720 2024-09-16T09:00:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a80feec0 con 0x7f58a8100720 2024-09-16T09:00:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.664+0000 7f58a5ffb640 1 -- 172.21.15.31:0/2674896829 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f589c003b50 con 0x7f58a8100720 2024-09-16T09:00:32.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.665+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a80ff3b0 con 0x7f58a8100720 2024-09-16T09:00:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.666+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a8100b00 con 0x7f58a8100720 2024-09-16T09:00:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.667+0000 7f58a5ffb640 1 -- 172.21.15.31:0/2674896829 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f589c003d40 con 0x7f58a8100720 2024-09-16T09:00:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.667+0000 7f58a5ffb640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f589807b060 0x7f589807d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.667+0000 7f58a5ffb640 1 -- 172.21.15.31:0/2674896829 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f589c0bf790 con 0x7f58a8100720 2024-09-16T09:00:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.667+0000 7f58a7fff640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f589807b060 0x7f589807d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.668+0000 7f58a7fff640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f589807b060 0x7f589807d520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f58a80ffe00 tx=0x7f589400a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:32.672 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.671+0000 7f58a5ffb640 1 -- 172.21.15.31:0/2674896829 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f589c082840 con 0x7f58a8100720 2024-09-16T09:00:32.748 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:32 smithi031 systemd[1]: Stopping Ceph osd.0 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:00:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:32 smithi097 ceph-mon[79892]: pgmap v37: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:32 smithi097 ceph-mon[79892]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:32 smithi097 ceph-mon[79892]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:32.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.897+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f58a819d930 con 0x7f58a8100720 2024-09-16T09:00:32.899 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.898+0000 7f58a5ffb640 1 -- 172.21.15.31:0/2674896829 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7f589c08be60 con 0x7f58a8100720 2024-09-16T09:00:32.899 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:00:32.900 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:00:32.901 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:32.902 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:32.903 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:00:32.903 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:00:32.903 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:32.903 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:00:32.903 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f589807b060 msgr2=0x7f589807d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f589807b060 0x7f589807d520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f58a80ffe00 tx=0x7f589400a2d0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 msgr2=0x7f58a8100080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8100080 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f589c005610 tx=0x7f589c005640 comp rx=0 tx=0).stop 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 shutdown_connections 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f589807b060 0x7f589807d520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a8103050 0x7f58a80fe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 --2- 172.21.15.31:0/2674896829 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58a8100720 0x7f58a8100080 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 >> 172.21.15.31:0/2674896829 conn(0x7f58a80fa280 msgr2=0x7f58a80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 shutdown_connections 2024-09-16T09:00:32.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.900+0000 7f58aef2c640 1 -- 172.21.15.31:0/2674896829 wait complete. 2024-09-16T09:00:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 -- 172.21.15.31:0/3718585421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c100630 msgr2=0x7fec0c102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 --2- 172.21.15.31:0/3718585421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c100630 0x7fec0c102a20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7febfc0098b0 tx=0x7febfc02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 -- 172.21.15.31:0/3718585421 shutdown_connections 2024-09-16T09:00:32.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 --2- 172.21.15.31:0/3718585421 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec0c102f60 0x7fec0c105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 --2- 172.21.15.31:0/3718585421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c100630 0x7fec0c102a20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 -- 172.21.15.31:0/3718585421 >> 172.21.15.31:0/3718585421 conn(0x7fec0c0fa250 msgr2=0x7fec0c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 -- 172.21.15.31:0/3718585421 shutdown_connections 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 -- 172.21.15.31:0/3718585421 wait complete. 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 Processor -- start 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.985+0000 7fec11f54640 1 -- start start 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec11f54640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec0c100630 0x7fec0c0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec11f54640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c102f60 0x7fec0c0fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec11f54640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0feb20 con 0x7fec0c102f60 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec11f54640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0c0fec90 con 0x7fec0c100630 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec03fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c102f60 0x7fec0c0fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec03fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c102f60 0x7fec0c0fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35892/0 (socket says 172.21.15.31:35892) 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec03fff640 1 -- 172.21.15.31:0/1740274467 learned_addr learned my addr 172.21.15.31:0/1740274467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec0b7fe640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec0c100630 0x7fec0c0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec03fff640 1 -- 172.21.15.31:0/1740274467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec0c100630 msgr2=0x7fec0c0fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec03fff640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec0c100630 0x7fec0c0fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec03fff640 1 -- 172.21.15.31:0/1740274467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc009560 con 0x7fec0c102f60 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.986+0000 7fec03fff640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c102f60 0x7fec0c0fe5e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf400dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.987+0000 7fec097fa640 1 -- 172.21.15.31:0/1740274467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf4009c00 con 0x7fec0c102f60 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.987+0000 7fec097fa640 1 -- 172.21.15.31:0/1740274467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febf400de90 con 0x7fec0c102f60 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.987+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0c0fef70 con 0x7fec0c102f60 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.987+0000 7fec097fa640 1 -- 172.21.15.31:0/1740274467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febf4003ea0 con 0x7fec0c102f60 2024-09-16T09:00:32.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.987+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec0c19d6a0 con 0x7fec0c102f60 2024-09-16T09:00:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.988+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0c100a10 con 0x7fec0c102f60 2024-09-16T09:00:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.988+0000 7fec097fa640 1 -- 172.21.15.31:0/1740274467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7febf4004000 con 0x7fec0c102f60 2024-09-16T09:00:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.989+0000 7fec097fa640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7febf807b060 0x7febf807d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.989+0000 7fec0b7fe640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7febf807b060 0x7febf807d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.989+0000 7fec097fa640 1 -- 172.21.15.31:0/1740274467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7febf409e140 con 0x7fec0c102f60 2024-09-16T09:00:32.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.990+0000 7fec0b7fe640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7febf807b060 0x7febf807d520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:32.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:32.994+0000 7fec097fa640 1 -- 172.21.15.31:0/1740274467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7febf4061030 con 0x7fec0c102f60 2024-09-16T09:00:33.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:32 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[49733]: 2024-09-16T09:00:32.746+0000 7f2e6222b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:00:33.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:32 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[49733]: 2024-09-16T09:00:32.746+0000 7f2e6222b640 -1 osd.0 50 *** Got signal Terminated *** 2024-09-16T09:00:33.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:32 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[49733]: 2024-09-16T09:00:32.746+0000 7f2e6222b640 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:00:33.198 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.197+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec0c103980 con 0x7febf807b060 2024-09-16T09:00:33.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.198+0000 7fec097fa640 1 -- 172.21.15.31:0/1740274467 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fec0c103980 con 0x7febf807b060 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "6/23 daemons upgraded", 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:00:33.200 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:00:33.201 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:00:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.200+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7febf807b060 msgr2=0x7febf807d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.200+0000 7fec11f54640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7febf807b060 0x7febf807d520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc0314e0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.200+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c102f60 msgr2=0x7fec0c0fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.200+0000 7fec11f54640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c102f60 0x7fec0c0fe5e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febf4005fd0 tx=0x7febf400dd60 comp rx=0 tx=0).stop 2024-09-16T09:00:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.201+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 shutdown_connections 2024-09-16T09:00:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.201+0000 7fec11f54640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7febf807b060 0x7febf807d520 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.201+0000 7fec11f54640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec0c102f60 0x7fec0c0fe5e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.201+0000 7fec11f54640 1 --2- 172.21.15.31:0/1740274467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fec0c100630 0x7fec0c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.201+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 >> 172.21.15.31:0/1740274467 conn(0x7fec0c0fa250 msgr2=0x7fec0c0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:33.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.201+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 shutdown_connections 2024-09-16T09:00:33.203 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.201+0000 7fec11f54640 1 -- 172.21.15.31:0/1740274467 wait complete. 2024-09-16T09:00:33.285 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 -- 172.21.15.31:0/2585333331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2240100d90 msgr2=0x7f2240101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:33.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 --2- 172.21.15.31:0/2585333331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2240100d90 0x7f2240101210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2230009880 tx=0x7f223002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 -- 172.21.15.31:0/2585333331 shutdown_connections 2024-09-16T09:00:33.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 --2- 172.21.15.31:0/2585333331 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2240100d90 0x7f2240101210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 --2- 172.21.15.31:0/2585333331 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f22400ffb90 0x7f22400fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 -- 172.21.15.31:0/2585333331 >> 172.21.15.31:0/2585333331 conn(0x7f22400fb320 msgr2=0x7f22400fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:33.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 -- 172.21.15.31:0/2585333331 shutdown_connections 2024-09-16T09:00:33.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.284+0000 7f224775c640 1 -- 172.21.15.31:0/2585333331 wait complete. 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f224775c640 1 Processor -- start 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f224775c640 1 -- start start 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f224775c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22400ffb90 0x7f22401955e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f224775c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100d90 0x7f2240195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f224775c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22401960f0 con 0x7f22400ffb90 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f224775c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2240196260 con 0x7f2240100d90 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f22454f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22400ffb90 0x7f22401955e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:33.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f22454f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22400ffb90 0x7f22401955e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:35900/0 (socket says 172.21.15.31:35900) 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f22454f8640 1 -- 172.21.15.31:0/1473241772 learned_addr learned my addr 172.21.15.31:0/1473241772 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.285+0000 7f2244cf7640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100d90 0x7f2240195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f22454f8640 1 -- 172.21.15.31:0/1473241772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100d90 msgr2=0x7f2240195b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f22454f8640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100d90 0x7f2240195b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f22454f8640 1 -- 172.21.15.31:0/1473241772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2230009560 con 0x7f22400ffb90 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f22454f8640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22400ffb90 0x7f22401955e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f223c00d720 tx=0x7f223c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f22367fc640 1 -- 172.21.15.31:0/1473241772 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223c0041f0 con 0x7f22400ffb90 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224019ad00 con 0x7f22400ffb90 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f22367fc640 1 -- 172.21.15.31:0/1473241772 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f223c004350 con 0x7f22400ffb90 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f22367fc640 1 -- 172.21.15.31:0/1473241772 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f223c003df0 con 0x7f22400ffb90 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.286+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224019b240 con 0x7f22400ffb90 2024-09-16T09:00:33.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.287+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2240108780 con 0x7f22400ffb90 2024-09-16T09:00:33.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.288+0000 7f22367fc640 1 -- 172.21.15.31:0/1473241772 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 38) v1 ==== 103268+0+0 (secure 0 0 0) 0x7f223c0035f0 con 0x7f22400ffb90 2024-09-16T09:00:33.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.288+0000 7f22367fc640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f222c07b000 0x7f222c07d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:00:33.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.288+0000 7f22367fc640 1 -- 172.21.15.31:0/1473241772 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f223c09dba0 con 0x7f22400ffb90 2024-09-16T09:00:33.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.288+0000 7f2244cf7640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f222c07b000 0x7f222c07d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:00:33.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.289+0000 7f2244cf7640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f222c07b000 0x7f222c07d4c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2230000c00 tx=0x7f22300314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:00:33.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.293+0000 7f22367fc640 1 -- 172.21.15.31:0/1473241772 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f223c060be0 con 0x7f22400ffb90 2024-09-16T09:00:33.453 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:33 smithi031 ceph-mon[91884]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:33.453 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:33 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2395164893' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:33.454 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:33 smithi031 ceph-mon[91884]: osd.0 marked itself down and dead 2024-09-16T09:00:33.454 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:33 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2674896829' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:00:33.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.559+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f224019b480 con 0x7f22400ffb90 2024-09-16T09:00:33.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.560+0000 7f22367fc640 1 -- 172.21.15.31:0/1473241772 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f223c063d00 con 0x7f22400ffb90 2024-09-16T09:00:33.561 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 osds down 2024-09-16T09:00:33.561 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T09:00:33.561 INFO:teuthology.orchestra.run.smithi031.stdout: osd.0 (root=default,host=smithi031) is down 2024-09-16T09:00:33.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f222c07b000 msgr2=0x7f222c07d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:33.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f222c07b000 0x7f222c07d4c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2230000c00 tx=0x7f22300314e0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22400ffb90 msgr2=0x7f22401955e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:00:33.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22400ffb90 0x7f22401955e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f223c00d720 tx=0x7f223c00dbe0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 shutdown_connections 2024-09-16T09:00:33.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f222c07b000 0x7f222c07d4c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2240100d90 0x7f2240195b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 --2- 172.21.15.31:0/1473241772 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22400ffb90 0x7f22401955e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:00:33.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 >> 172.21.15.31:0/1473241772 conn(0x7f22400fb320 msgr2=0x7f22400fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:00:33.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 shutdown_connections 2024-09-16T09:00:33.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:00:33.562+0000 7f224775c640 1 -- 172.21.15.31:0/1473241772 wait complete. 2024-09-16T09:00:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:33 smithi097 ceph-mon[79892]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:33 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2395164893' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:33 smithi097 ceph-mon[79892]: osd.0 marked itself down and dead 2024-09-16T09:00:33.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:33 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2674896829' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:00:34.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:33 smithi031 podman[99182]: 2024-09-16 09:00:33.866307627 +0000 UTC m=+1.240892006 container died cefa539e811b55ee2e03bea1d90839c21317e7ca2a9c543653f36eb4925da469 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T09:00:34.560 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:34 smithi031 ceph-mon[91884]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:34.561 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:34 smithi031 ceph-mon[91884]: pgmap v38: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:34.561 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:34 smithi031 ceph-mon[91884]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:00:34.561 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:34 smithi031 ceph-mon[91884]: osdmap e51: 6 total, 5 up, 6 in 2024-09-16T09:00:34.561 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:34 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1473241772' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:00:34.561 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:34 smithi031 podman[99182]: 2024-09-16 09:00:34.311919838 +0000 UTC m=+1.686504205 container cleanup cefa539e811b55ee2e03bea1d90839c21317e7ca2a9c543653f36eb4925da469 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-16T09:00:34.561 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:34 smithi031 bash[99182]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0 2024-09-16T09:00:34.620 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:34 smithi097 ceph-mon[79892]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:00:34.620 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:34 smithi097 ceph-mon[79892]: pgmap v38: 65 pgs: 65 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:34.620 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:34 smithi097 ceph-mon[79892]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:00:34.620 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:34 smithi097 ceph-mon[79892]: osdmap e51: 6 total, 5 up, 6 in 2024-09-16T09:00:34.620 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:34 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1473241772' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:00:34.873 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:34 smithi031 podman[99264]: 2024-09-16 09:00:34.560191624 +0000 UTC m=+0.689680964 container remove cefa539e811b55ee2e03bea1d90839c21317e7ca2a9c543653f36eb4925da469 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T09:00:35.267 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:34 smithi031 podman[99354]: 2024-09-16 09:00:34.88742986 +0000 UTC m=+0.016433571 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:00:35.267 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:34 smithi031 podman[99354]: 2024-09-16 09:00:34.997988097 +0000 UTC m=+0.126991805 container create 901af9f2f488079179d9cbef946b1b2c14e82cc53cc0a467100345fd74961684 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T09:00:35.267 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:35 smithi031 podman[99354]: 2024-09-16 09:00:35.266948362 +0000 UTC m=+0.395952080 container init 901af9f2f488079179d9cbef946b1b2c14e82cc53cc0a467100345fd74961684 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T09:00:35.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:35 smithi031 ceph-mon[91884]: osdmap e52: 6 total, 5 up, 6 in 2024-09-16T09:00:35.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:35 smithi031 podman[99354]: 2024-09-16 09:00:35.270511706 +0000 UTC m=+0.399515416 container start 901af9f2f488079179d9cbef946b1b2c14e82cc53cc0a467100345fd74961684 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T09:00:35.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:35 smithi031 podman[99354]: 2024-09-16 09:00:35.367062856 +0000 UTC m=+0.496066580 container attach 901af9f2f488079179d9cbef946b1b2c14e82cc53cc0a467100345fd74961684 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T09:00:35.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:35 smithi031 podman[99354]: 2024-09-16 09:00:35.472905734 +0000 UTC m=+0.601909444 container died 901af9f2f488079179d9cbef946b1b2c14e82cc53cc0a467100345fd74961684 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T09:00:35.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:35 smithi097 ceph-mon[79892]: osdmap e52: 6 total, 5 up, 6 in 2024-09-16T09:00:36.272 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 podman[99378]: 2024-09-16 09:00:36.012039213 +0000 UTC m=+0.533409592 container remove 901af9f2f488079179d9cbef946b1b2c14e82cc53cc0a467100345fd74961684 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:00:36.272 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-16T09:00:36.272 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 systemd[1]: Stopped Ceph osd.0 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:00:36.272 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.0.service: Consumed 1min 1.217s CPU time. 2024-09-16T09:00:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:36 smithi031 ceph-mon[91884]: pgmap v41: 65 pgs: 34 peering, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:36.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:36 smithi031 ceph-mon[91884]: Health check failed: Reduced data availability: 32 pgs peering (PG_AVAILABILITY) 2024-09-16T09:00:36.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 systemd[1]: Starting Ceph osd.0 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:00:36.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 podman[99477]: 2024-09-16 09:00:36.386802479 +0000 UTC m=+0.016399576 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:00:36.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 podman[99477]: 2024-09-16 09:00:36.504840845 +0000 UTC m=+0.134437942 container create e9f0684d393b8846fc5fc4375a75b538ab3aa3bdfc8a2bb3a66f02cc300e119e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T09:00:36.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:36 smithi097 ceph-mon[79892]: pgmap v41: 65 pgs: 34 peering, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:36.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:36 smithi097 ceph-mon[79892]: Health check failed: Reduced data availability: 32 pgs peering (PG_AVAILABILITY) 2024-09-16T09:00:36.922 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 podman[99477]: 2024-09-16 09:00:36.782875652 +0000 UTC m=+0.412472750 container init e9f0684d393b8846fc5fc4375a75b538ab3aa3bdfc8a2bb3a66f02cc300e119e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T09:00:36.922 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 podman[99477]: 2024-09-16 09:00:36.786452441 +0000 UTC m=+0.416049538 container start e9f0684d393b8846fc5fc4375a75b538ab3aa3bdfc8a2bb3a66f02cc300e119e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T09:00:36.923 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 podman[99477]: 2024-09-16 09:00:36.849253119 +0000 UTC m=+0.478850229 container attach e9f0684d393b8846fc5fc4375a75b538ab3aa3bdfc8a2bb3a66f02cc300e119e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T09:00:37.373 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:00:37.373 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 bash[99477]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:00:37.373 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:00:37.373 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:36 smithi031 bash[99477]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:00:38.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:38 smithi031 ceph-mon[91884]: pgmap v42: 65 pgs: 34 peering, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:38.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:38 smithi097 ceph-mon[79892]: pgmap v42: 65 pgs: 34 peering, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:00:39.568 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T09:00:39.569 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 bash[99477]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T09:00:39.569 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T09:00:39.569 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 bash[99477]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T09:00:39.569 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T09:00:39.569 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 bash[99477]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T09:00:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:39.873 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 bash[99477]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 bash[99477]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 bash[99477]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate[99492]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 bash[99477]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-16T09:00:39.874 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:39 smithi031 podman[99477]: 2024-09-16 09:00:39.604776794 +0000 UTC m=+3.234373891 container died e9f0684d393b8846fc5fc4375a75b538ab3aa3bdfc8a2bb3a66f02cc300e119e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:00:40.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:40.530 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 podman[99648]: 2024-09-16 09:00:40.177371237 +0000 UTC m=+0.565640144 container remove e9f0684d393b8846fc5fc4375a75b538ab3aa3bdfc8a2bb3a66f02cc300e119e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:00:40.530 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 podman[99708]: 2024-09-16 09:00:40.29736759 +0000 UTC m=+0.016697878 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:00:40.530 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 podman[99708]: 2024-09-16 09:00:40.419989921 +0000 UTC m=+0.139320265 container create 8060870d2228a307a6817b56f152ce5bb290fbc5c6b330d4849dbb9cb88d75e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T09:00:40.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:40 smithi031 ceph-mon[91884]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:40.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:40 smithi031 ceph-mon[91884]: Health check failed: Degraded data redundancy: 55/282 objects degraded (19.504%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T09:00:40.806 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:40 smithi031 ceph-mon[91884]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 32 pgs peering) 2024-09-16T09:00:40.806 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 podman[99708]: 2024-09-16 09:00:40.704060477 +0000 UTC m=+0.423390756 container init 8060870d2228a307a6817b56f152ce5bb290fbc5c6b330d4849dbb9cb88d75e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-16T09:00:40.806 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 podman[99708]: 2024-09-16 09:00:40.707602458 +0000 UTC m=+0.426932737 container start 8060870d2228a307a6817b56f152ce5bb290fbc5c6b330d4849dbb9cb88d75e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T09:00:40.807 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 ceph-osd[99728]: -- 172.21.15.31:0/125306253 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b936f21e0 con 0x563b928f3c00 2024-09-16T09:00:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:40 smithi097 ceph-mon[79892]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:40 smithi097 ceph-mon[79892]: Health check failed: Degraded data redundancy: 55/282 objects degraded (19.504%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T09:00:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:40 smithi097 ceph-mon[79892]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 32 pgs peering) 2024-09-16T09:00:41.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 bash[99708]: 8060870d2228a307a6817b56f152ce5bb290fbc5c6b330d4849dbb9cb88d75e8 2024-09-16T09:00:41.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:40 smithi031 systemd[1]: Started Ceph osd.0 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:00:42.114 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:42.114 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:42.114 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:42.373 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:42 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[99724]: 2024-09-16T09:00:42.112+0000 7fc9154c7640 -1 Falling back to public interface 2024-09-16T09:00:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:42 smithi031 ceph-mon[91884]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:43.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:42 smithi097 ceph-mon[79892]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:44 smithi031 ceph-mon[91884]: pgmap v45: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:45.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:44 smithi097 ceph-mon[79892]: pgmap v45: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:46.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:46 smithi031 ceph-mon[91884]: pgmap v46: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:46.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:46.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:46.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:46.996 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:47.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:46 smithi097 ceph-mon[79892]: pgmap v46: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:47.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:47.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:47.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:47.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:48.087 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:47 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[99724]: 2024-09-16T09:00:47.908+0000 7fc9154c7640 -1 osd.0 50 log_to_monitors true 2024-09-16T09:00:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:48 smithi097 ceph-mon[79892]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:48 smithi097 ceph-mon[79892]: from='osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T09:00:49.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:48 smithi031 ceph-mon[91884]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:49.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:48 smithi031 ceph-mon[91884]: from='osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T09:00:49.124 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:00:48 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[99724]: 2024-09-16T09:00:48.769+0000 7fc90d271640 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:00:50.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T09:00:50.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: osdmap e53: 6 total, 5 up, 6 in 2024-09-16T09:00:50.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-16T09:00:50.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:50.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:50.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:50.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:00:50.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:50.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:50.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:50.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:50.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:50.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-16T09:00:50.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T09:00:50.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: osdmap e53: 6 total, 5 up, 6 in 2024-09-16T09:00:50.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-16T09:00:50.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:50.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:50.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:00:50.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:00:50.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:50.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:00:50.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:50.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:50.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:00:50.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-16T09:00:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:50 smithi097 ceph-mon[79892]: pgmap v49: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:50 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T09:00:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:50 smithi097 ceph-mon[79892]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-16T09:00:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:50 smithi097 ceph-mon[79892]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:00:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:50 smithi097 ceph-mon[79892]: osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989] boot 2024-09-16T09:00:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:50 smithi097 ceph-mon[79892]: osdmap e54: 6 total, 6 up, 6 in 2024-09-16T09:00:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:50 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:00:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:50 smithi031 ceph-mon[91884]: pgmap v49: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:50 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T09:00:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:50 smithi031 ceph-mon[91884]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-16T09:00:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:50 smithi031 ceph-mon[91884]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:00:51.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:50 smithi031 ceph-mon[91884]: osd.0 [v2:172.21.15.31:6802/198948989,v1:172.21.15.31:6803/198948989] boot 2024-09-16T09:00:51.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:50 smithi031 ceph-mon[91884]: osdmap e54: 6 total, 6 up, 6 in 2024-09-16T09:00:51.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:50 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:00:52.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:51 smithi031 ceph-mon[91884]: osdmap e55: 6 total, 6 up, 6 in 2024-09-16T09:00:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:51 smithi097 ceph-mon[79892]: osdmap e55: 6 total, 6 up, 6 in 2024-09-16T09:00:53.024 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:52 smithi031 ceph-mon[91884]: pgmap v52: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:53.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:52 smithi097 ceph-mon[79892]: pgmap v52: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:55.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:54 smithi097 ceph-mon[79892]: pgmap v53: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:55.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:55.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:54 smithi031 ceph-mon[91884]: pgmap v53: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/282 objects degraded (19.504%) 2024-09-16T09:00:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:00:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:00:56.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:55 smithi031 ceph-mon[91884]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 55/282 objects degraded (19.504%), 19 pgs degraded) 2024-09-16T09:00:56.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:55 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:00:56.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:55 smithi031 ceph-mon[91884]: mgrmap e39: smithi031.dcmqpv(active, since 92s), standbys: smithi097.mpashf 2024-09-16T09:00:56.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:55 smithi097 ceph-mon[79892]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 55/282 objects degraded (19.504%), 19 pgs degraded) 2024-09-16T09:00:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:55 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:00:56.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:55 smithi097 ceph-mon[79892]: mgrmap e39: smithi031.dcmqpv(active, since 92s), standbys: smithi097.mpashf 2024-09-16T09:00:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:56 smithi031 ceph-mon[91884]: pgmap v54: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 628 B/s wr, 0 op/s 2024-09-16T09:00:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:56 smithi097 ceph-mon[79892]: pgmap v54: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 628 B/s wr, 0 op/s 2024-09-16T09:00:59.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:00:58 smithi031 ceph-mon[91884]: pgmap v55: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T09:00:59.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:00:58 smithi097 ceph-mon[79892]: pgmap v55: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-09-16T09:01:01.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:00 smithi031 ceph-mon[91884]: pgmap v56: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 430 B/s wr, 0 op/s 2024-09-16T09:01:01.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:00 smithi097 ceph-mon[79892]: pgmap v56: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 430 B/s wr, 0 op/s 2024-09-16T09:01:03.079 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:02 smithi031 ceph-mon[91884]: pgmap v57: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 389 B/s wr, 0 op/s 2024-09-16T09:01:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:02 smithi097 ceph-mon[79892]: pgmap v57: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 389 B/s wr, 0 op/s 2024-09-16T09:01:03.663 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 -- 172.21.15.31:0/3512106887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 msgr2=0x7f1b54105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 --2- 172.21.15.31:0/3512106887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54105350 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1b480098b0 tx=0x7f1b4802a4d0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 -- 172.21.15.31:0/3512106887 shutdown_connections 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 --2- 172.21.15.31:0/3512106887 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54105350 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 --2- 172.21.15.31:0/3512106887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b54100630 0x7f1b54102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 -- 172.21.15.31:0/3512106887 >> 172.21.15.31:0/3512106887 conn(0x7f1b540fa230 msgr2=0x7f1b540fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 -- 172.21.15.31:0/3512106887 shutdown_connections 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.660+0000 7f1b58870640 1 -- 172.21.15.31:0/3512106887 wait complete. 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.661+0000 7f1b58870640 1 Processor -- start 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.661+0000 7f1b58870640 1 -- start start 2024-09-16T09:01:03.664 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.661+0000 7f1b58870640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b54100630 0x7f1b54195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.661+0000 7f1b58870640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.661+0000 7f1b58870640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b54196180 con 0x7f1b54102f60 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.661+0000 7f1b58870640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541962f0 con 0x7f1b54100630 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b51d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b51d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54195bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53398/0 (socket says 172.21.15.31:53398) 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b51d9b640 1 -- 172.21.15.31:0/4195910149 learned_addr learned my addr 172.21.15.31:0/4195910149 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b5259c640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b54100630 0x7f1b54195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b51d9b640 1 -- 172.21.15.31:0/4195910149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b54100630 msgr2=0x7f1b54195670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b51d9b640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b54100630 0x7f1b54195670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.665 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b51d9b640 1 -- 172.21.15.31:0/4195910149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48009560 con 0x7f1b54102f60 2024-09-16T09:01:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b5259c640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b54100630 0x7f1b54195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:01:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b51d9b640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54195bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009230 tx=0x7f1b480054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b437fe640 1 -- 172.21.15.31:0/4195910149 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b48004aa0 con 0x7f1b54102f60 2024-09-16T09:01:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b437fe640 1 -- 172.21.15.31:0/4195910149 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b48004c00 con 0x7f1b54102f60 2024-09-16T09:01:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5419ad30 con 0x7f1b54102f60 2024-09-16T09:01:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b437fe640 1 -- 172.21.15.31:0/4195910149 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b48003a80 con 0x7f1b54102f60 2024-09-16T09:01:03.666 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.662+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5419b2a0 con 0x7f1b54102f60 2024-09-16T09:01:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.663+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b54108790 con 0x7f1b54102f60 2024-09-16T09:01:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.664+0000 7f1b437fe640 1 -- 172.21.15.31:0/4195910149 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f1b48031c90 con 0x7f1b54102f60 2024-09-16T09:01:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.665+0000 7f1b437fe640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b3c07af30 0x7f1b3c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.665+0000 7f1b437fe640 1 -- 172.21.15.31:0/4195910149 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1b480bf0f0 con 0x7f1b54102f60 2024-09-16T09:01:03.667 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.665+0000 7f1b5259c640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b3c07af30 0x7f1b3c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:03.668 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.666+0000 7f1b5259c640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b3c07af30 0x7f1b3c07d3f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005fd0 tx=0x7f1b44009520 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:03.670 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.669+0000 7f1b437fe640 1 -- 172.21.15.31:0/4195910149 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f1b48082320 con 0x7f1b54102f60 2024-09-16T09:01:03.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.873+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b54103980 con 0x7f1b3c07af30 2024-09-16T09:01:03.875 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.874+0000 7f1b437fe640 1 -- 172.21.15.31:0/4195910149 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1b54103980 con 0x7f1b3c07af30 2024-09-16T09:01:03.877 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b3c07af30 msgr2=0x7f1b3c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b3c07af30 0x7f1b3c07d3f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1b44005fd0 tx=0x7f1b44009520 comp rx=0 tx=0).stop 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 msgr2=0x7f1b54195bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54195bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009230 tx=0x7f1b480054f0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 shutdown_connections 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b3c07af30 0x7f1b3c07d3f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b54102f60 0x7f1b54195bb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 --2- 172.21.15.31:0/4195910149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b54100630 0x7f1b54195670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 >> 172.21.15.31:0/4195910149 conn(0x7f1b540fa230 msgr2=0x7f1b540fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 shutdown_connections 2024-09-16T09:01:03.878 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.876+0000 7f1b58870640 1 -- 172.21.15.31:0/4195910149 wait complete. 2024-09-16T09:01:03.886 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 -- 172.21.15.31:0/1296954332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d4100d90 msgr2=0x7f69d4101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 --2- 172.21.15.31:0/1296954332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d4100d90 0x7f69d4101210 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f69c4009c20 tx=0x7f69c402a970 comp rx=0 tx=0).stop 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 -- 172.21.15.31:0/1296954332 shutdown_connections 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 --2- 172.21.15.31:0/1296954332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d4100d90 0x7f69d4101210 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 --2- 172.21.15.31:0/1296954332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d40ffb90 0x7f69d40fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 -- 172.21.15.31:0/1296954332 >> 172.21.15.31:0/1296954332 conn(0x7f69d40fb320 msgr2=0x7f69d40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 -- 172.21.15.31:0/1296954332 shutdown_connections 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.961+0000 7f69dab06640 1 -- 172.21.15.31:0/1296954332 wait complete. 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69dab06640 1 Processor -- start 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69dab06640 1 -- start start 2024-09-16T09:01:03.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69dab06640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d40ffb90 0x7f69d4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69dab06640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d4100d90 0x7f69d40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69dab06640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4074790 con 0x7f69d40ffb90 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69dab06640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d4072e10 con 0x7f69d4100d90 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69d88a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d40ffb90 0x7f69d4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69d88a2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d40ffb90 0x7f69d4074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53404/0 (socket says 172.21.15.31:53404) 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.962+0000 7f69d88a2640 1 -- 172.21.15.31:0/1252866137 learned_addr learned my addr 172.21.15.31:0/1252866137 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69d3fff640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d4100d90 0x7f69d40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69d88a2640 1 -- 172.21.15.31:0/1252866137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d4100d90 msgr2=0x7f69d40728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69d88a2640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d4100d90 0x7f69d40728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:03.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69d88a2640 1 -- 172.21.15.31:0/1252866137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c4009900 con 0x7f69d40ffb90 2024-09-16T09:01:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69d88a2640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d40ffb90 0x7f69d4074250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c8f0 tx=0x7f69bc00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69d1ffb640 1 -- 172.21.15.31:0/1252866137 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69bc0041f0 con 0x7f69d40ffb90 2024-09-16T09:01:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69d1ffb640 1 -- 172.21.15.31:0/1252866137 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69bc007e10 con 0x7f69d40ffb90 2024-09-16T09:01:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.963+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d40730f0 con 0x7f69d40ffb90 2024-09-16T09:01:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.964+0000 7f69d1ffb640 1 -- 172.21.15.31:0/1252866137 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69bc00f450 con 0x7f69d40ffb90 2024-09-16T09:01:03.966 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.964+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d40736c0 con 0x7f69d40ffb90 2024-09-16T09:01:03.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.965+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d40fff90 con 0x7f69d40ffb90 2024-09-16T09:01:03.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.966+0000 7f69d1ffb640 1 -- 172.21.15.31:0/1252866137 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f69bc007950 con 0x7f69d40ffb90 2024-09-16T09:01:03.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.967+0000 7f69d1ffb640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f69c007ad30 0x7f69c007d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:03.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.967+0000 7f69d1ffb640 1 -- 172.21.15.31:0/1252866137 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f69bc013070 con 0x7f69d40ffb90 2024-09-16T09:01:03.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.967+0000 7f69d3fff640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f69c007ad30 0x7f69c007d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:03.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.967+0000 7f69d3fff640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f69c007ad30 0x7f69c007d1f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69d4073fd0 tx=0x7f69c4003970 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:03.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:03.972+0000 7f69d1ffb640 1 -- 172.21.15.31:0/1252866137 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f69bc061240 con 0x7f69d40ffb90 2024-09-16T09:01:04.172 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.171+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69d4105540 con 0x7f69c007ad30 2024-09-16T09:01:04.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.172+0000 7f69d1ffb640 1 -- 172.21.15.31:0/1252866137 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f69d4105540 con 0x7f69c007ad30 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f69c007ad30 msgr2=0x7f69c007d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f69c007ad30 0x7f69c007d1f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69d4073fd0 tx=0x7f69c4003970 comp rx=0 tx=0).stop 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d40ffb90 msgr2=0x7f69d4074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d40ffb90 0x7f69d4074250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00c8f0 tx=0x7f69bc00cdb0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 shutdown_connections 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f69c007ad30 0x7f69c007d1f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69d4100d90 0x7f69d40728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 --2- 172.21.15.31:0/1252866137 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69d40ffb90 0x7f69d4074250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.174+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 >> 172.21.15.31:0/1252866137 conn(0x7f69d40fb320 msgr2=0x7f69d40fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.175+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 shutdown_connections 2024-09-16T09:01:04.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.175+0000 7f69dab06640 1 -- 172.21.15.31:0/1252866137 wait complete. 2024-09-16T09:01:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.259+0000 7f5a0e482640 1 -- 172.21.15.31:0/2096554445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 msgr2=0x7f5a080fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.259+0000 7f5a0e482640 1 --2- 172.21.15.31:0/2096554445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a080fffa0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009880 tx=0x7f59f802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.259+0000 7f5a0e482640 1 -- 172.21.15.31:0/2096554445 shutdown_connections 2024-09-16T09:01:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.259+0000 7f5a0e482640 1 --2- 172.21.15.31:0/2096554445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a08100da0 0x7f5a08101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.259+0000 7f5a0e482640 1 --2- 172.21.15.31:0/2096554445 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a080fffa0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.260+0000 7f5a0e482640 1 -- 172.21.15.31:0/2096554445 >> 172.21.15.31:0/2096554445 conn(0x7f5a080fb330 msgr2=0x7f5a080fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.260+0000 7f5a0e482640 1 -- 172.21.15.31:0/2096554445 shutdown_connections 2024-09-16T09:01:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.260+0000 7f5a0e482640 1 -- 172.21.15.31:0/2096554445 wait complete. 2024-09-16T09:01:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.260+0000 7f5a0e482640 1 Processor -- start 2024-09-16T09:01:04.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.260+0000 7f5a0e482640 1 -- start start 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a0e482640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a08195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a0e482640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a08100da0 0x7f5a08195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a0e482640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08196150 con 0x7f5a080ffba0 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a0e482640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a081962c0 con 0x7f5a08100da0 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a08195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a07fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a08195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53422/0 (socket says 172.21.15.31:53422) 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a07fff640 1 -- 172.21.15.31:0/864240688 learned_addr learned my addr 172.21.15.31:0/864240688 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:04.264 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a077fe640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a08100da0 0x7f5a08195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a07fff640 1 -- 172.21.15.31:0/864240688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a08100da0 msgr2=0x7f5a08195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a07fff640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a08100da0 0x7f5a08195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.261+0000 7f5a07fff640 1 -- 172.21.15.31:0/864240688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8009560 con 0x7f5a080ffba0 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.262+0000 7f5a07fff640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a08195640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59f8004650 tx=0x7f59f80055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.262+0000 7f5a057fa640 1 -- 172.21.15.31:0/864240688 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59f8004aa0 con 0x7f5a080ffba0 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.262+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0819acb0 con 0x7f5a080ffba0 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.262+0000 7f5a057fa640 1 -- 172.21.15.31:0/864240688 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59f8005d50 con 0x7f5a080ffba0 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.262+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0819b1a0 con 0x7f5a080ffba0 2024-09-16T09:01:04.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.262+0000 7f5a057fa640 1 -- 172.21.15.31:0/864240688 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59f80039d0 con 0x7f5a080ffba0 2024-09-16T09:01:04.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.263+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0818f740 con 0x7f5a080ffba0 2024-09-16T09:01:04.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.264+0000 7f5a057fa640 1 -- 172.21.15.31:0/864240688 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f59f8031c00 con 0x7f5a080ffba0 2024-09-16T09:01:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.264+0000 7f5a057fa640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f59f007af30 0x7f59f007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.265+0000 7f5a057fa640 1 -- 172.21.15.31:0/864240688 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f59f80bf380 con 0x7f5a080ffba0 2024-09-16T09:01:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.265+0000 7f5a077fe640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f59f007af30 0x7f59f007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.267 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.265+0000 7f5a077fe640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f59f007af30 0x7f59f007d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5a08196b10 tx=0x7f59fc005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:04.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.269+0000 7f5a057fa640 1 -- 172.21.15.31:0/864240688 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f59f80825e0 con 0x7f5a080ffba0 2024-09-16T09:01:04.464 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.463+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a0802c4f0 con 0x7f59f007af30 2024-09-16T09:01:04.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.473+0000 7f5a057fa640 1 -- 172.21.15.31:0/864240688 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f5a0802c4f0 con 0x7f59f007af30 2024-09-16T09:01:04.474 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:01:04.474 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (10m) 18s ago 12m 25.3M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:01:04.474 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (3m) 18s ago 12m 7763k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:01:04.474 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (3m) 42s ago 11m 7578k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (50s) 18s ago 12m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (46s) 42s ago 11m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (10m) 18s ago 12m 85.5M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (6m) 18s ago 6m 15.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (6m) 18s ago 6m 153M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (6m) 42s ago 6m 26.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (6m) 42s ago 6m 17.4M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (3m) 18s ago 13m 509M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (2m) 42s ago 11m 452M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (108s) 18s ago 14m 45.0M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (88s) 42s ago 11m 33.6M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (12m) 18s ago 12m 15.5M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:01:04.475 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (11m) 42s ago 11m 16.4M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:01:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (24s) 18s ago 10m 11.1M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:01:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (9m) 18s ago 9m 265M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 bdb1a021ae06 2024-09-16T09:01:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (9m) 18s ago 9m 222M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T09:01:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (8m) 42s ago 8m 278M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T09:01:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (8m) 42s ago 8m 250M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T09:01:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (7m) 42s ago 7m 233M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T09:01:04.476 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (2m) 18s ago 12m 51.4M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f59f007af30 msgr2=0x7f59f007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f59f007af30 0x7f59f007d3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5a08196b10 tx=0x7f59fc005f90 comp rx=0 tx=0).stop 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 msgr2=0x7f5a08195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a08195640 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59f8004650 tx=0x7f59f80055a0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 shutdown_connections 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f59f007af30 0x7f59f007d3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5a08100da0 0x7f5a08195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 --2- 172.21.15.31:0/864240688 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a080ffba0 0x7f5a08195640 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.475+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 >> 172.21.15.31:0/864240688 conn(0x7f5a080fb330 msgr2=0x7f5a08104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:04.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.476+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 shutdown_connections 2024-09-16T09:01:04.478 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.476+0000 7f5a0e482640 1 -- 172.21.15.31:0/864240688 wait complete. 2024-09-16T09:01:04.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 -- 172.21.15.31:0/518246641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 msgr2=0x7f2bec0fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 --2- 172.21.15.31:0/518246641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec0fff90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009880 tx=0x7f2bdc02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.559 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 -- 172.21.15.31:0/518246641 shutdown_connections 2024-09-16T09:01:04.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 --2- 172.21.15.31:0/518246641 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec100d90 0x7f2bec101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 --2- 172.21.15.31:0/518246641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec0fff90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 -- 172.21.15.31:0/518246641 >> 172.21.15.31:0/518246641 conn(0x7f2bec0fb320 msgr2=0x7f2bec0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:04.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 -- 172.21.15.31:0/518246641 shutdown_connections 2024-09-16T09:01:04.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.558+0000 7f2bf0895640 1 -- 172.21.15.31:0/518246641 wait complete. 2024-09-16T09:01:04.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bf0895640 1 Processor -- start 2024-09-16T09:01:04.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bf0895640 1 -- start start 2024-09-16T09:01:04.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bf0895640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bf0895640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec100d90 0x7f2bec197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bf0895640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec198390 con 0x7f2bec0ffb90 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bf0895640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec198500 con 0x7f2bec100d90 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bea59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bea59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec197880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53434/0 (socket says 172.21.15.31:53434) 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2bea59c640 1 -- 172.21.15.31:0/2944784438 learned_addr learned my addr 172.21.15.31:0/2944784438 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.559+0000 7f2be9d9b640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec100d90 0x7f2bec197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bea59c640 1 -- 172.21.15.31:0/2944784438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec100d90 msgr2=0x7f2bec197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bea59c640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec100d90 0x7f2bec197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bea59c640 1 -- 172.21.15.31:0/2944784438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc009560 con 0x7f2bec0ffb90 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2be9d9b640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec100d90 0x7f2bec197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:01:04.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bea59c640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec197880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc02a9c0 tx=0x7f2bdc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:04.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bdb7fe640 1 -- 172.21.15.31:0/2944784438 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bdc004aa0 con 0x7f2bec0ffb90 2024-09-16T09:01:04.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bdb7fe640 1 -- 172.21.15.31:0/2944784438 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bdc004c00 con 0x7f2bec0ffb90 2024-09-16T09:01:04.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bdb7fe640 1 -- 172.21.15.31:0/2944784438 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bdc003a50 con 0x7f2bec0ffb90 2024-09-16T09:01:04.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec19cf40 con 0x7f2bec0ffb90 2024-09-16T09:01:04.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.560+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec19d430 con 0x7f2bec0ffb90 2024-09-16T09:01:04.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.561+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec105260 con 0x7f2bec0ffb90 2024-09-16T09:01:04.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.562+0000 7f2bdb7fe640 1 -- 172.21.15.31:0/2944784438 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f2bdc031c90 con 0x7f2bec0ffb90 2024-09-16T09:01:04.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.563+0000 7f2bdb7fe640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2bd407af30 0x7f2bd407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.563+0000 7f2be9d9b640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2bd407af30 0x7f2bd407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.563+0000 7f2bdb7fe640 1 -- 172.21.15.31:0/2944784438 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f2bdc0bf040 con 0x7f2bec0ffb90 2024-09-16T09:01:04.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.563+0000 7f2be9d9b640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2bd407af30 0x7f2bd407d3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2be00058c0 tx=0x7f2be0005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:04.569 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.568+0000 7f2bdb7fe640 1 -- 172.21.15.31:0/2944784438 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f2bdc0821f0 con 0x7f2bec0ffb90 2024-09-16T09:01:04.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.835+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2bec02c4e0 con 0x7f2bec0ffb90 2024-09-16T09:01:04.837 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:04 smithi031 ceph-mon[91884]: pgmap v58: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T09:01:04.837 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:04.837 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:04 smithi031 ceph-mon[91884]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:04.837 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-16T09:01:04.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.836+0000 7f2bdb7fe640 1 -- 172.21.15.31:0/2944784438 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f2bdc085310 con 0x7f2bec0ffb90 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 5, 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 1 2024-09-16T09:01:04.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 9, 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 5 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:01:04.839 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2bd407af30 msgr2=0x7f2bd407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2bd407af30 0x7f2bd407d3f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2be00058c0 tx=0x7f2be0005850 comp rx=0 tx=0).stop 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 msgr2=0x7f2bec197880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec197880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc02a9c0 tx=0x7f2bdc0055a0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 shutdown_connections 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2bd407af30 0x7f2bd407d3f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2bec100d90 0x7f2bec197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 --2- 172.21.15.31:0/2944784438 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2bec0ffb90 0x7f2bec197880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 >> 172.21.15.31:0/2944784438 conn(0x7f2bec0fb320 msgr2=0x7f2bec102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:04.840 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 shutdown_connections 2024-09-16T09:01:04.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.839+0000 7f2bf0895640 1 -- 172.21.15.31:0/2944784438 wait complete. 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 -- 172.21.15.31:0/1245137370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fee30 msgr2=0x7f3cb40ff290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 --2- 172.21.15.31:0/1245137370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fee30 0x7f3cb40ff290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c009880 tx=0x7f3c9c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 -- 172.21.15.31:0/1245137370 shutdown_connections 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 --2- 172.21.15.31:0/1245137370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fee30 0x7f3cb40ff290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 --2- 172.21.15.31:0/1245137370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb40fe4f0 0x7f3cb40fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 -- 172.21.15.31:0/1245137370 >> 172.21.15.31:0/1245137370 conn(0x7f3cb40fa280 msgr2=0x7f3cb40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 -- 172.21.15.31:0/1245137370 shutdown_connections 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.926+0000 7f3cba214640 1 -- 172.21.15.31:0/1245137370 wait complete. 2024-09-16T09:01:04.928 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cba214640 1 Processor -- start 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cba214640 1 -- start start 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cba214640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fe4f0 0x7f3cb41978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cba214640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb40fee30 0x7f3cb4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cba214640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb41983d0 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cba214640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4198540 con 0x7f3cb40fee30 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cb37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fe4f0 0x7f3cb41978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cb37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fe4f0 0x7f3cb41978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53452/0 (socket says 172.21.15.31:53452) 2024-09-16T09:01:04.929 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cb37fe640 1 -- 172.21.15.31:0/171526130 learned_addr learned my addr 172.21.15.31:0/171526130 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:04.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cabfff640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb40fee30 0x7f3cb4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cb37fe640 1 -- 172.21.15.31:0/171526130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb40fee30 msgr2=0x7f3cb4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:04.930 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cb37fe640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb40fee30 0x7f3cb4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cb37fe640 1 -- 172.21.15.31:0/171526130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c009560 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cabfff640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb40fee30 0x7f3cb4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.927+0000 7f3cb37fe640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fe4f0 0x7f3cb41978c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400d900 tx=0x7f3ca400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.928+0000 7f3cb17fa640 1 -- 172.21.15.31:0/171526130 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca4009c00 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.928+0000 7f3cb17fa640 1 -- 172.21.15.31:0/171526130 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ca400b660 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.928+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb419cfe0 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.928+0000 7f3cb17fa640 1 -- 172.21.15.31:0/171526130 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca400f680 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.928+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb419d4d0 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.929+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4191960 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.931 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.930+0000 7f3cb17fa640 1 -- 172.21.15.31:0/171526130 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f3ca4010ab0 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.930+0000 7f3cb17fa640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3ca007af30 0x7f3ca007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:04.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.930+0000 7f3cb17fa640 1 -- 172.21.15.31:0/171526130 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3ca409dd20 con 0x7f3cb40fe4f0 2024-09-16T09:01:04.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.930+0000 7f3cabfff640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3ca007af30 0x7f3ca007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:04.932 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.931+0000 7f3cabfff640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3ca007af30 0x7f3ca007d3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c031540 tx=0x7f3c9c005de0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:04.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:04.935+0000 7f3cb17fa640 1 -- 172.21.15.31:0/171526130 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f3ca4060ed0 con 0x7f3cb40fe4f0 2024-09-16T09:01:05.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:04 smithi097 ceph-mon[79892]: pgmap v58: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T09:01:05.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:05.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:04 smithi097 ceph-mon[79892]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:05.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16} : dispatch 2024-09-16T09:01:05.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.166+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3cb402c4f0 con 0x7f3cb40fe4f0 2024-09-16T09:01:05.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.167+0000 7f3cb17fa640 1 -- 172.21.15.31:0/171526130 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7f3ca4063ff0 con 0x7f3cb40fe4f0 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:01:05.169 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:01:05.170 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:05.171 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:05.172 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:01:05.172 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:05.172 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:05.172 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:05.172 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:01:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3ca007af30 msgr2=0x7f3ca007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3ca007af30 0x7f3ca007d3f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c031540 tx=0x7f3c9c005de0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fe4f0 msgr2=0x7f3cb41978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.173 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fe4f0 0x7f3cb41978c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3ca400d900 tx=0x7f3ca400ddc0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 shutdown_connections 2024-09-16T09:01:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3ca007af30 0x7f3ca007d3f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3cb40fee30 0x7f3cb4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 --2- 172.21.15.31:0/171526130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3cb40fe4f0 0x7f3cb41978c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 >> 172.21.15.31:0/171526130 conn(0x7f3cb40fa280 msgr2=0x7f3cb4106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 shutdown_connections 2024-09-16T09:01:05.177 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.169+0000 7f3cba214640 1 -- 172.21.15.31:0/171526130 wait complete. 2024-09-16T09:01:05.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 -- 172.21.15.31:0/2731861254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 msgr2=0x7fde081053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 --2- 172.21.15.31:0/2731861254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde081053a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fddf00098b0 tx=0x7fddf002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 -- 172.21.15.31:0/2731861254 shutdown_connections 2024-09-16T09:01:05.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 --2- 172.21.15.31:0/2731861254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde081053a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 --2- 172.21.15.31:0/2731861254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde08100680 0x7fde08102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 -- 172.21.15.31:0/2731861254 >> 172.21.15.31:0/2731861254 conn(0x7fde080fa280 msgr2=0x7fde080fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 -- 172.21.15.31:0/2731861254 shutdown_connections 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.255+0000 7fde0d995640 1 -- 172.21.15.31:0/2731861254 wait complete. 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde0d995640 1 Processor -- start 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde0d995640 1 -- start start 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde0d995640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde08100680 0x7fde08197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde0d995640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde08197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde0d995640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08198420 con 0x7fde08102fb0 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde0d995640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08198590 con 0x7fde08100680 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde067fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde08197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde067fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde08197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53476/0 (socket says 172.21.15.31:53476) 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde067fc640 1 -- 172.21.15.31:0/3766646676 learned_addr learned my addr 172.21.15.31:0/3766646676 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:05.258 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde06ffd640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde08100680 0x7fde08197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde067fc640 1 -- 172.21.15.31:0/3766646676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde08100680 msgr2=0x7fde08197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.256+0000 7fde067fc640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde08100680 0x7fde08197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde067fc640 1 -- 172.21.15.31:0/3766646676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf0009560 con 0x7fde08102fb0 2024-09-16T09:01:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde06ffd640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde08100680 0x7fde08197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:01:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde067fc640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde08197e50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fddf00055a0 tx=0x7fddf00055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde0c993640 1 -- 172.21.15.31:0/3766646676 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf0004aa0 con 0x7fde08102fb0 2024-09-16T09:01:05.260 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde0c993640 1 -- 172.21.15.31:0/3766646676 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddf0004c00 con 0x7fde08102fb0 2024-09-16T09:01:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0819cfd0 con 0x7fde08102fb0 2024-09-16T09:01:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde0c993640 1 -- 172.21.15.31:0/3766646676 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf0003b50 con 0x7fde08102fb0 2024-09-16T09:01:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.257+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0819d490 con 0x7fde08102fb0 2024-09-16T09:01:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.259+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde08100a60 con 0x7fde08102fb0 2024-09-16T09:01:05.261 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.259+0000 7fde0c993640 1 -- 172.21.15.31:0/3766646676 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fddf0003cb0 con 0x7fde08102fb0 2024-09-16T09:01:05.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.259+0000 7fde0c993640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf407af30 0x7fddf407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:05.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.260+0000 7fde0c993640 1 -- 172.21.15.31:0/3766646676 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fddf00bf1a0 con 0x7fde08102fb0 2024-09-16T09:01:05.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.260+0000 7fde06ffd640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf407af30 0x7fddf407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:05.262 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.260+0000 7fde06ffd640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf407af30 0x7fddf407d3f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fddf8005fd0 tx=0x7fddf8009600 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:05.265 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.264+0000 7fde0c993640 1 -- 172.21.15.31:0/3766646676 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fddf0082350 con 0x7fde08102fb0 2024-09-16T09:01:05.473 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.472+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde080ffcb0 con 0x7fddf407af30 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.473+0000 7fde0c993640 1 -- 172.21.15.31:0/3766646676 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fde080ffcb0 con 0x7fddf407af30 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:01:05.474 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:01:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:01:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "7/23 daemons upgraded", 2024-09-16T09:01:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:01:05.475 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:01:05.475 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf407af30 msgr2=0x7fddf407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf407af30 0x7fddf407d3f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fddf8005fd0 tx=0x7fddf8009600 comp rx=0 tx=0).stop 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 msgr2=0x7fde08197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde08197e50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fddf00055a0 tx=0x7fddf00055d0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 shutdown_connections 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf407af30 0x7fddf407d3f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde08102fb0 0x7fde08197e50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 --2- 172.21.15.31:0/3766646676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde08100680 0x7fde08197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.475+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 >> 172.21.15.31:0/3766646676 conn(0x7fde080fa280 msgr2=0x7fde080ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.476+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 shutdown_connections 2024-09-16T09:01:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.476+0000 7fde0d995640 1 -- 172.21.15.31:0/3766646676 wait complete. 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.559+0000 7fe33e3df640 1 -- 172.21.15.31:0/2289567264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3381006e0 msgr2=0x7fe338102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.559+0000 7fe33e3df640 1 --2- 172.21.15.31:0/2289567264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3381006e0 0x7fe338102ad0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe328009880 tx=0x7fe32802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.559+0000 7fe33e3df640 1 -- 172.21.15.31:0/2289567264 shutdown_connections 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.559+0000 7fe33e3df640 1 --2- 172.21.15.31:0/2289567264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe338103010 0x7fe338105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.559+0000 7fe33e3df640 1 --2- 172.21.15.31:0/2289567264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3381006e0 0x7fe338102ad0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.559+0000 7fe33e3df640 1 -- 172.21.15.31:0/2289567264 >> 172.21.15.31:0/2289567264 conn(0x7fe3380fa280 msgr2=0x7fe3380fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.559+0000 7fe33e3df640 1 -- 172.21.15.31:0/2289567264 shutdown_connections 2024-09-16T09:01:05.561 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.560+0000 7fe33e3df640 1 -- 172.21.15.31:0/2289567264 wait complete. 2024-09-16T09:01:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.560+0000 7fe33e3df640 1 Processor -- start 2024-09-16T09:01:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.560+0000 7fe33e3df640 1 -- start start 2024-09-16T09:01:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe33e3df640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3381006e0 0x7fe3380714f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe33e3df640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338103010 0x7fe338071a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:05.562 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe33e3df640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe338072000 con 0x7fe338103010 2024-09-16T09:01:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe33e3df640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe338072170 con 0x7fe3381006e0 2024-09-16T09:01:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe3377fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338103010 0x7fe338071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe337fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3381006e0 0x7fe3380714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe337fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3381006e0 0x7fe3380714f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:34984/0 (socket says 172.21.15.31:34984) 2024-09-16T09:01:05.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.561+0000 7fe337fff640 1 -- 172.21.15.31:0/4011157725 learned_addr learned my addr 172.21.15.31:0/4011157725 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe3377fe640 1 -- 172.21.15.31:0/4011157725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3381006e0 msgr2=0x7fe3380714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe3377fe640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3381006e0 0x7fe3380714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe3377fe640 1 -- 172.21.15.31:0/4011157725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328009560 con 0x7fe338103010 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe337fff640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3381006e0 0x7fe3380714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe3377fe640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338103010 0x7fe338071a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c8c0 tx=0x7fe32c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe3357fa640 1 -- 172.21.15.31:0/4011157725 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe32c007b80 con 0x7fe338103010 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe3357fa640 1 -- 172.21.15.31:0/4011157725 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe32c00ae60 con 0x7fe338103010 2024-09-16T09:01:05.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3381a31f0 con 0x7fe338103010 2024-09-16T09:01:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe3357fa640 1 -- 172.21.15.31:0/4011157725 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe32c004e20 con 0x7fe338103010 2024-09-16T09:01:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.562+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3380feb50 con 0x7fe338103010 2024-09-16T09:01:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.564+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe338100ac0 con 0x7fe338103010 2024-09-16T09:01:05.565 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.564+0000 7fe3357fa640 1 -- 172.21.15.31:0/4011157725 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fe32c004f80 con 0x7fe338103010 2024-09-16T09:01:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.565+0000 7fe3357fa640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe32007af30 0x7fe32007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.565+0000 7fe3357fa640 1 -- 172.21.15.31:0/4011157725 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fe32c09dbf0 con 0x7fe338103010 2024-09-16T09:01:05.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.565+0000 7fe337fff640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe32007af30 0x7fe32007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:05.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.566+0000 7fe337fff640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe32007af30 0x7fe32007d3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe328004650 tx=0x7fe3280314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:05.571 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.570+0000 7fe3357fa640 1 -- 172.21.15.31:0/4011157725 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fe32c060e50 con 0x7fe338103010 2024-09-16T09:01:05.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.834+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe3381087a0 con 0x7fe338103010 2024-09-16T09:01:05.836 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:05 smithi031 ceph-mon[91884]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:05.836 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:05 smithi031 ceph-mon[91884]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:05.836 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:05 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T09:01:05.836 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:05 smithi031 ceph-mon[91884]: Upgrade: osd.1 is safe to restart 2024-09-16T09:01:05.836 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:05 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2944784438' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:05.836 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:05 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/171526130' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:01:05.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.834+0000 7fe3357fa640 1 -- 172.21.15.31:0/4011157725 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fe32c063f70 con 0x7fe338103010 2024-09-16T09:01:05.836 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:01:05.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe32007af30 msgr2=0x7fe32007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe32007af30 0x7fe32007d3f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe328004650 tx=0x7fe3280314e0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338103010 msgr2=0x7fe338071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:05.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338103010 0x7fe338071a30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe32c00c8c0 tx=0x7fe32c00cd80 comp rx=0 tx=0).stop 2024-09-16T09:01:05.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 shutdown_connections 2024-09-16T09:01:05.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe32007af30 0x7fe32007d3f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe338103010 0x7fe338071a30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 --2- 172.21.15.31:0/4011157725 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3381006e0 0x7fe3380714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:05.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 >> 172.21.15.31:0/4011157725 conn(0x7fe3380fa280 msgr2=0x7fe3380fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:05.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 shutdown_connections 2024-09-16T09:01:05.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:05.837+0000 7fe33e3df640 1 -- 172.21.15.31:0/4011157725 wait complete. 2024-09-16T09:01:06.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:05 smithi097 ceph-mon[79892]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:06.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:05 smithi097 ceph-mon[79892]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:06.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:05 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-16T09:01:06.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:05 smithi097 ceph-mon[79892]: Upgrade: osd.1 is safe to restart 2024-09-16T09:01:06.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:05 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2944784438' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:06.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:05 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/171526130' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:01:06.818 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:06 smithi031 ceph-mon[91884]: pgmap v59: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T09:01:06.818 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:06 smithi031 ceph-mon[91884]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:06.818 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:06 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/4011157725' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:01:06.818 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:06.818 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T09:01:06.818 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:06 smithi097 ceph-mon[79892]: pgmap v59: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-09-16T09:01:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:06 smithi097 ceph-mon[79892]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:06 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/4011157725' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:01:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T09:01:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:08.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:07 smithi097 ceph-mon[79892]: Upgrade: Updating osd.1 2024-09-16T09:01:08.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:07 smithi097 ceph-mon[79892]: Deploying daemon osd.1 on smithi031 2024-09-16T09:01:08.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:07 smithi031 ceph-mon[91884]: Upgrade: Updating osd.1 2024-09-16T09:01:08.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:07 smithi031 ceph-mon[91884]: Deploying daemon osd.1 on smithi031 2024-09-16T09:01:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:08 smithi097 ceph-mon[79892]: pgmap v60: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:09.030 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:08 smithi031 ceph-mon[91884]: pgmap v60: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:09.030 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:09.030 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:09.292 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:09 smithi031 systemd[1]: Stopping Ceph osd.1 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:01:09.623 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:09 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[56981]: 2024-09-16T09:01:09.289+0000 7f8fb2e62640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:01:09.623 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:09 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[56981]: 2024-09-16T09:01:09.289+0000 7f8fb2e62640 -1 osd.1 55 *** Got signal Terminated *** 2024-09-16T09:01:09.623 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:09 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[56981]: 2024-09-16T09:01:09.289+0000 7f8fb2e62640 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:01:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:09 smithi097 ceph-mon[79892]: osd.1 marked itself down and dead 2024-09-16T09:01:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:09 smithi031 ceph-mon[91884]: osd.1 marked itself down and dead 2024-09-16T09:01:10.577 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:10 smithi031 podman[104428]: 2024-09-16 09:01:10.274939127 +0000 UTC m=+1.097351273 container died bdb1a021ae0643e3779fe1595206a00e0d8b2bb1197f0c6f23b209b20300e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T09:01:10.873 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:10 smithi031 podman[104428]: 2024-09-16 09:01:10.665374809 +0000 UTC m=+1.487786955 container cleanup bdb1a021ae0643e3779fe1595206a00e0d8b2bb1197f0c6f23b209b20300e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T09:01:10.873 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:10 smithi031 bash[104428]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1 2024-09-16T09:01:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:10 smithi031 ceph-mon[91884]: pgmap v61: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:10 smithi031 ceph-mon[91884]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:01:10.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:10 smithi031 ceph-mon[91884]: osdmap e56: 6 total, 5 up, 6 in 2024-09-16T09:01:11.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:10 smithi097 ceph-mon[79892]: pgmap v61: 65 pgs: 65 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:11.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:10 smithi097 ceph-mon[79892]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:01:11.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:10 smithi097 ceph-mon[79892]: osdmap e56: 6 total, 5 up, 6 in 2024-09-16T09:01:11.243 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:10 smithi031 podman[104443]: 2024-09-16 09:01:10.912232061 +0000 UTC m=+0.634568208 container remove bdb1a021ae0643e3779fe1595206a00e0d8b2bb1197f0c6f23b209b20300e11c (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, ceph=True, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T09:01:11.496 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:11 smithi031 podman[104536]: 2024-09-16 09:01:11.242934481 +0000 UTC m=+0.105367828 container create 3370ca323ccd5f8aa1e36399f699ef55857949048296de8134c4544149a5e3e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T09:01:11.496 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:11 smithi031 podman[104536]: 2024-09-16 09:01:11.153901684 +0000 UTC m=+0.016335050 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:01:11.496 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:11 smithi031 podman[104536]: 2024-09-16 09:01:11.495521607 +0000 UTC m=+0.357954963 container init 3370ca323ccd5f8aa1e36399f699ef55857949048296de8134c4544149a5e3e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:01:11.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:11 smithi031 ceph-mon[91884]: osdmap e57: 6 total, 5 up, 6 in 2024-09-16T09:01:11.873 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:11 smithi031 podman[104536]: 2024-09-16 09:01:11.499217402 +0000 UTC m=+0.361650765 container start 3370ca323ccd5f8aa1e36399f699ef55857949048296de8134c4544149a5e3e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:01:11.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:11 smithi031 podman[104536]: 2024-09-16 09:01:11.570466952 +0000 UTC m=+0.432900303 container attach 3370ca323ccd5f8aa1e36399f699ef55857949048296de8134c4544149a5e3e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:01:11.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:11 smithi031 podman[104536]: 2024-09-16 09:01:11.699630247 +0000 UTC m=+0.562063600 container died 3370ca323ccd5f8aa1e36399f699ef55857949048296de8134c4544149a5e3e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:01:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:11 smithi097 ceph-mon[79892]: osdmap e57: 6 total, 5 up, 6 in 2024-09-16T09:01:12.485 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 podman[104558]: 2024-09-16 09:01:12.232001689 +0000 UTC m=+0.526566691 container remove 3370ca323ccd5f8aa1e36399f699ef55857949048296de8134c4544149a5e3e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T09:01:12.485 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-16T09:01:12.485 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 systemd[1]: Stopped Ceph osd.1 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:01:12.485 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.1.service: Consumed 1min 10.151s CPU time. 2024-09-16T09:01:12.741 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 systemd[1]: Starting Ceph osd.1 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:01:12.741 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 podman[104657]: 2024-09-16 09:01:12.599642577 +0000 UTC m=+0.016357278 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:01:12.741 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 podman[104657]: 2024-09-16 09:01:12.708170376 +0000 UTC m=+0.124885076 container create fa8f382e224d0c615545f4b84e232cbcbc0e612621a3e9164fd40869d52ce02e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T09:01:13.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:12 smithi031 ceph-mon[91884]: pgmap v64: 65 pgs: 12 stale+active+clean, 53 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:13.011 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 podman[104657]: 2024-09-16 09:01:12.935706299 +0000 UTC m=+0.352421000 container init fa8f382e224d0c615545f4b84e232cbcbc0e612621a3e9164fd40869d52ce02e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T09:01:13.011 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:12 smithi031 podman[104657]: 2024-09-16 09:01:12.939213985 +0000 UTC m=+0.355928693 container start fa8f382e224d0c615545f4b84e232cbcbc0e612621a3e9164fd40869d52ce02e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:01:13.011 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:13 smithi031 podman[104657]: 2024-09-16 09:01:13.010431068 +0000 UTC m=+0.427145795 container attach fa8f382e224d0c615545f4b84e232cbcbc0e612621a3e9164fd40869d52ce02e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T09:01:13.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:12 smithi097 ceph-mon[79892]: pgmap v64: 65 pgs: 12 stale+active+clean, 53 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:13.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:13 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:13.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:13 smithi031 bash[104657]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:13.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:13 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:13.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:13 smithi031 bash[104657]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:15.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:14 smithi097 ceph-mon[79892]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:14 smithi031 ceph-mon[91884]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:15.744 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T09:01:15.744 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 bash[104657]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T09:01:15.744 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T09:01:15.744 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 bash[104657]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T09:01:15.744 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T09:01:15.744 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 bash[104657]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T09:01:16.098 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-mon[91884]: Health check failed: Degraded data redundancy: 43/282 objects degraded (15.248%), 15 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:16.098 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T09:01:16.098 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 bash[104657]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T09:01:16.098 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-16T09:01:16.098 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 bash[104657]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-16T09:01:16.098 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T09:01:16.098 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 bash[104657]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T09:01:16.099 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate[104672]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-16T09:01:16.099 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 bash[104657]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-16T09:01:16.099 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:15 smithi031 podman[104657]: 2024-09-16 09:01:15.777634806 +0000 UTC m=+3.194349518 container died fa8f382e224d0c615545f4b84e232cbcbc0e612621a3e9164fd40869d52ce02e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T09:01:16.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:15 smithi097 ceph-mon[79892]: Health check failed: Degraded data redundancy: 43/282 objects degraded (15.248%), 15 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:16.764 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 podman[104845]: 2024-09-16 09:01:16.414163644 +0000 UTC m=+0.630212186 container remove fa8f382e224d0c615545f4b84e232cbcbc0e612621a3e9164fd40869d52ce02e (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-16T09:01:16.764 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 podman[104905]: 2024-09-16 09:01:16.539347061 +0000 UTC m=+0.017008205 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:01:16.764 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 podman[104905]: 2024-09-16 09:01:16.656634925 +0000 UTC m=+0.134296054 container create 539fc5d4e6cf77ae60e41d4b0849c29ddf41adcb7222f7e5e7bde7b04cfdbeb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:01:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:16 smithi031 ceph-mon[91884]: pgmap v66: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:17.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 podman[104905]: 2024-09-16 09:01:16.892315296 +0000 UTC m=+0.369976433 container init 539fc5d4e6cf77ae60e41d4b0849c29ddf41adcb7222f7e5e7bde7b04cfdbeb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-16T09:01:17.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 podman[104905]: 2024-09-16 09:01:16.896410982 +0000 UTC m=+0.374072109 container start 539fc5d4e6cf77ae60e41d4b0849c29ddf41adcb7222f7e5e7bde7b04cfdbeb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:01:17.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 ceph-osd[104923]: -- 172.21.15.31:0/2356999477 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bd075e1e0 con 0x555bd06ee000 2024-09-16T09:01:17.124 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 bash[104905]: 539fc5d4e6cf77ae60e41d4b0849c29ddf41adcb7222f7e5e7bde7b04cfdbeb6 2024-09-16T09:01:17.124 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:16 smithi031 systemd[1]: Started Ceph osd.1 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:01:17.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:16 smithi097 ceph-mon[79892]: pgmap v66: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:18.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:18 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:18.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:18 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:18.325 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:18 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:18.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:18 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:18 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:18.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:18 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:18.623 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:18 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[104919]: 2024-09-16T09:01:18.540+0000 7f17805b9640 -1 Falling back to public interface 2024-09-16T09:01:19.301 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:19 smithi031 ceph-mon[91884]: pgmap v67: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:19.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:19 smithi097 ceph-mon[79892]: pgmap v67: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:21.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:21 smithi031 ceph-mon[91884]: pgmap v68: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:21.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:21 smithi097 ceph-mon[79892]: pgmap v68: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:23.178 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:22 smithi031 ceph-mon[91884]: pgmap v69: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:23.178 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:23.178 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:23.178 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:23.178 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:22 smithi097 ceph-mon[79892]: pgmap v69: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:24 smithi097 ceph-mon[79892]: pgmap v70: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:25.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:24 smithi031 ceph-mon[91884]: pgmap v70: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:25.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:25.067 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:25.067 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:24 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[104919]: 2024-09-16T09:01:24.859+0000 7f17805b9640 -1 osd.1 55 log_to_monitors true 2024-09-16T09:01:25.732 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:25 smithi031 ceph-mon[91884]: from='osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:01:26.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:25 smithi097 ceph-mon[79892]: from='osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:01:26.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:01:25 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[104919]: 2024-09-16T09:01:25.730+0000 7f1778363640 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:01:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: pgmap v71: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T09:01:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: osdmap e58: 6 total, 5 up, 6 in 2024-09-16T09:01:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-16T09:01:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T09:01:27.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:26 smithi097 ceph-mon[79892]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-16T09:01:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: pgmap v71: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T09:01:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: osdmap e58: 6 total, 5 up, 6 in 2024-09-16T09:01:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-16T09:01:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T09:01:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:26 smithi031 ceph-mon[91884]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-16T09:01:28.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:27 smithi097 ceph-mon[79892]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:01:28.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:27 smithi097 ceph-mon[79892]: osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696] boot 2024-09-16T09:01:28.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:27 smithi097 ceph-mon[79892]: osdmap e59: 6 total, 6 up, 6 in 2024-09-16T09:01:28.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:27 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:01:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:27 smithi031 ceph-mon[91884]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:01:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:27 smithi031 ceph-mon[91884]: osd.1 [v2:172.21.15.31:6810/4124137696,v1:172.21.15.31:6811/4124137696] boot 2024-09-16T09:01:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:27 smithi031 ceph-mon[91884]: osdmap e59: 6 total, 6 up, 6 in 2024-09-16T09:01:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:27 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:01:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:28 smithi097 ceph-mon[79892]: pgmap v74: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:28 smithi097 ceph-mon[79892]: osdmap e60: 6 total, 6 up, 6 in 2024-09-16T09:01:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:28 smithi031 ceph-mon[91884]: pgmap v74: 65 pgs: 19 active+undersized, 15 active+undersized+degraded, 31 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 43/282 objects degraded (15.248%) 2024-09-16T09:01:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:28 smithi031 ceph-mon[91884]: osdmap e60: 6 total, 6 up, 6 in 2024-09-16T09:01:30.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:29 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 22/282 objects degraded (7.801%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:30.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:29 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 22/282 objects degraded (7.801%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:30 smithi031 ceph-mon[91884]: pgmap v76: 65 pgs: 5 active+undersized, 7 active+undersized+degraded, 53 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/282 objects degraded (7.801%) 2024-09-16T09:01:31.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:30 smithi097 ceph-mon[79892]: pgmap v76: 65 pgs: 5 active+undersized, 7 active+undersized+degraded, 53 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22/282 objects degraded (7.801%) 2024-09-16T09:01:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:31 smithi031 ceph-mon[91884]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/282 objects degraded (7.801%), 7 pgs degraded) 2024-09-16T09:01:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:31 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:01:32.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:31 smithi097 ceph-mon[79892]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 22/282 objects degraded (7.801%), 7 pgs degraded) 2024-09-16T09:01:32.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:31 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:01:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:32 smithi031 ceph-mon[91884]: pgmap v77: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:33.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:32 smithi097 ceph-mon[79892]: pgmap v77: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:34 smithi031 ceph-mon[91884]: pgmap v78: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:34 smithi097 ceph-mon[79892]: pgmap v78: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 -- 172.21.15.31:0/681054255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 msgr2=0x7fbb94102ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/681054255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94102ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009880 tx=0x7fbb8402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 -- 172.21.15.31:0/681054255 shutdown_connections 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/681054255 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94103010 0x7fbb94105400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/681054255 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94102ad0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 -- 172.21.15.31:0/681054255 >> 172.21.15.31:0/681054255 conn(0x7fbb940fa280 msgr2=0x7fbb940fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 -- 172.21.15.31:0/681054255 shutdown_connections 2024-09-16T09:01:35.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.935+0000 7fbb99e8a640 1 -- 172.21.15.31:0/681054255 wait complete. 2024-09-16T09:01:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb99e8a640 1 Processor -- start 2024-09-16T09:01:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb99e8a640 1 -- start start 2024-09-16T09:01:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb99e8a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb99e8a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94103010 0x7fbb94195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb99e8a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94196180 con 0x7fbb941006e0 2024-09-16T09:01:35.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb99e8a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941962f0 con 0x7fbb94103010 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb937fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49156/0 (socket says 172.21.15.31:49156) 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.936+0000 7fbb937fe640 1 -- 172.21.15.31:0/436151380 learned_addr learned my addr 172.21.15.31:0/436151380 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb8bfff640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94103010 0x7fbb94195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb937fe640 1 -- 172.21.15.31:0/436151380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94103010 msgr2=0x7fbb94195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb937fe640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94103010 0x7fbb94195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb937fe640 1 -- 172.21.15.31:0/436151380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb84009560 con 0x7fbb941006e0 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb8bfff640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94103010 0x7fbb94195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb937fe640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94195670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb840055a0 tx=0x7fbb840055d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb917fa640 1 -- 172.21.15.31:0/436151380 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb84004b20 con 0x7fbb941006e0 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb917fa640 1 -- 172.21.15.31:0/436151380 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb84004c80 con 0x7fbb941006e0 2024-09-16T09:01:35.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9419ad90 con 0x7fbb941006e0 2024-09-16T09:01:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb917fa640 1 -- 172.21.15.31:0/436151380 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb84003bc0 con 0x7fbb941006e0 2024-09-16T09:01:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.937+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9419b260 con 0x7fbb941006e0 2024-09-16T09:01:35.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.939+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb941087c0 con 0x7fbb941006e0 2024-09-16T09:01:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.939+0000 7fbb917fa640 1 -- 172.21.15.31:0/436151380 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fbb84003d20 con 0x7fbb941006e0 2024-09-16T09:01:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.940+0000 7fbb917fa640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb8007af30 0x7fbb8007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.940+0000 7fbb917fa640 1 -- 172.21.15.31:0/436151380 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbb840bf170 con 0x7fbb941006e0 2024-09-16T09:01:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.940+0000 7fbb8bfff640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb8007af30 0x7fbb8007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:35.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.941+0000 7fbb8bfff640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb8007af30 0x7fbb8007d3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0058c0 tx=0x7fbb7c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:35.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:35.945+0000 7fbb917fa640 1 -- 172.21.15.31:0/436151380 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fbb840823a0 con 0x7fbb941006e0 2024-09-16T09:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.152+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb9419b4d0 con 0x7fbb8007af30 2024-09-16T09:01:36.154 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.154+0000 7fbb917fa640 1 -- 172.21.15.31:0/436151380 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbb9419b4d0 con 0x7fbb8007af30 2024-09-16T09:01:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb8007af30 msgr2=0x7fbb8007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb8007af30 0x7fbb8007d3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0058c0 tx=0x7fbb7c005850 comp rx=0 tx=0).stop 2024-09-16T09:01:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 msgr2=0x7fbb94195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.157 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94195670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbb840055a0 tx=0x7fbb840055d0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 shutdown_connections 2024-09-16T09:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb8007af30 0x7fbb8007d3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb94103010 0x7fbb94195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 --2- 172.21.15.31:0/436151380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb941006e0 0x7fbb94195670 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 >> 172.21.15.31:0/436151380 conn(0x7fbb940fa280 msgr2=0x7fbb940fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 shutdown_connections 2024-09-16T09:01:36.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.156+0000 7fbb99e8a640 1 -- 172.21.15.31:0/436151380 wait complete. 2024-09-16T09:01:36.166 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:01:36.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 -- 172.21.15.31:0/241039443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 msgr2=0x7f90ac101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/241039443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac101210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9094009880 tx=0x7f909402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 -- 172.21.15.31:0/241039443 shutdown_connections 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/241039443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac101210 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/241039443 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac0ffb90 0x7f90ac0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 -- 172.21.15.31:0/241039443 >> 172.21.15.31:0/241039443 conn(0x7f90ac0fb320 msgr2=0x7f90ac0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 -- 172.21.15.31:0/241039443 shutdown_connections 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 -- 172.21.15.31:0/241039443 wait complete. 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.241+0000 7f90b1ac1640 1 Processor -- start 2024-09-16T09:01:36.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90b1ac1640 1 -- start start 2024-09-16T09:01:36.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90b1ac1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac0ffb90 0x7f90ac074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90b1ac1640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90b1ac1640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac072de0 con 0x7f90ac100d90 2024-09-16T09:01:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90b1ac1640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ac072f50 con 0x7f90ac0ffb90 2024-09-16T09:01:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90a2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90a2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49184/0 (socket says 172.21.15.31:49184) 2024-09-16T09:01:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90a2ffd640 1 -- 172.21.15.31:0/454621735 learned_addr learned my addr 172.21.15.31:0/454621735 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:36.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90ab7fe640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac0ffb90 0x7f90ac074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90a2ffd640 1 -- 172.21.15.31:0/454621735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac0ffb90 msgr2=0x7f90ac074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90a2ffd640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac0ffb90 0x7f90ac074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.242+0000 7f90a2ffd640 1 -- 172.21.15.31:0/454621735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9094009560 con 0x7f90ac100d90 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.243+0000 7f90a2ffd640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac0728a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90940054f0 tx=0x7f9094005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.243+0000 7f90a97fa640 1 -- 172.21.15.31:0/454621735 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9094004aa0 con 0x7f90ac100d90 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.243+0000 7f90a97fa640 1 -- 172.21.15.31:0/454621735 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9094004c00 con 0x7f90ac100d90 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.243+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ac0731d0 con 0x7f90ac100d90 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.243+0000 7f90a97fa640 1 -- 172.21.15.31:0/454621735 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9094003ac0 con 0x7f90ac100d90 2024-09-16T09:01:36.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.243+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ac19d690 con 0x7f90ac100d90 2024-09-16T09:01:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.244+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ac0fff90 con 0x7f90ac100d90 2024-09-16T09:01:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.245+0000 7f90a97fa640 1 -- 172.21.15.31:0/454621735 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f9094031c70 con 0x7f90ac100d90 2024-09-16T09:01:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.245+0000 7f90a97fa640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f909807af30 0x7f909807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.246+0000 7f90a97fa640 1 -- 172.21.15.31:0/454621735 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f90940bf160 con 0x7f90ac100d90 2024-09-16T09:01:36.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.246+0000 7f90ab7fe640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f909807af30 0x7f909807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.246+0000 7f90ab7fe640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f909807af30 0x7f909807d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f909c005fd0 tx=0x7f909c00b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:36.251 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.250+0000 7f90a97fa640 1 -- 172.21.15.31:0/454621735 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f90940823c0 con 0x7f90ac100d90 2024-09-16T09:01:36.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.455+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90ac0608f0 con 0x7f909807af30 2024-09-16T09:01:36.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.456+0000 7f90a97fa640 1 -- 172.21.15.31:0/454621735 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f90ac0608f0 con 0x7f909807af30 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.458+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f909807af30 msgr2=0x7f909807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.458+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f909807af30 0x7f909807d3f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f909c005fd0 tx=0x7f909c00b390 comp rx=0 tx=0).stop 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.458+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 msgr2=0x7f90ac0728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.458+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac0728a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f90940054f0 tx=0x7f9094005520 comp rx=0 tx=0).stop 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.459+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 shutdown_connections 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.459+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f909807af30 0x7f909807d3f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.459+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90ac100d90 0x7f90ac0728a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.459+0000 7f90b1ac1640 1 --2- 172.21.15.31:0/454621735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f90ac0ffb90 0x7f90ac074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.459+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 >> 172.21.15.31:0/454621735 conn(0x7f90ac0fb320 msgr2=0x7f90ac104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:36.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.459+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 shutdown_connections 2024-09-16T09:01:36.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.459+0000 7f90b1ac1640 1 -- 172.21.15.31:0/454621735 wait complete. 2024-09-16T09:01:36.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.542+0000 7fcb25c5c640 1 -- 172.21.15.31:0/4278369483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 msgr2=0x7fcb20101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.542+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/4278369483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb20101240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcb08009880 tx=0x7fcb0802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 -- 172.21.15.31:0/4278369483 shutdown_connections 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/4278369483 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb20101240 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/4278369483 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb200ffbc0 0x7fcb200fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 -- 172.21.15.31:0/4278369483 >> 172.21.15.31:0/4278369483 conn(0x7fcb200fb350 msgr2=0x7fcb200fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 -- 172.21.15.31:0/4278369483 shutdown_connections 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 -- 172.21.15.31:0/4278369483 wait complete. 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 Processor -- start 2024-09-16T09:01:36.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.543+0000 7fcb25c5c640 1 -- start start 2024-09-16T09:01:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb25c5c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb200ffbc0 0x7fcb20074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb25c5c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb200728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb25c5c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20072de0 con 0x7fcb20100dc0 2024-09-16T09:01:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb25c5c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb20072f50 con 0x7fcb200ffbc0 2024-09-16T09:01:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb16ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb200728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb16ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb200728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49208/0 (socket says 172.21.15.31:49208) 2024-09-16T09:01:36.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb16ffd640 1 -- 172.21.15.31:0/3445312968 learned_addr learned my addr 172.21.15.31:0/3445312968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb1f7fe640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb200ffbc0 0x7fcb20074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb16ffd640 1 -- 172.21.15.31:0/3445312968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb200ffbc0 msgr2=0x7fcb20074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb16ffd640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb200ffbc0 0x7fcb20074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb16ffd640 1 -- 172.21.15.31:0/3445312968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb08009560 con 0x7fcb20100dc0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.544+0000 7fcb16ffd640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb200728a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcb080054f0 tx=0x7fcb08005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.545+0000 7fcb1d7fa640 1 -- 172.21.15.31:0/3445312968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb08004aa0 con 0x7fcb20100dc0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.545+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb200731d0 con 0x7fcb20100dc0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.545+0000 7fcb1d7fa640 1 -- 172.21.15.31:0/3445312968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb08004c00 con 0x7fcb20100dc0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.545+0000 7fcb1d7fa640 1 -- 172.21.15.31:0/3445312968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb08003ac0 con 0x7fcb20100dc0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.545+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb200736c0 con 0x7fcb20100dc0 2024-09-16T09:01:36.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.546+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2018f740 con 0x7fcb20100dc0 2024-09-16T09:01:36.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.547+0000 7fcb1d7fa640 1 -- 172.21.15.31:0/3445312968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fcb08031c70 con 0x7fcb20100dc0 2024-09-16T09:01:36.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.547+0000 7fcb1d7fa640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcb0c07af30 0x7fcb0c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.548+0000 7fcb1f7fe640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcb0c07af30 0x7fcb0c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.548+0000 7fcb1d7fa640 1 -- 172.21.15.31:0/3445312968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fcb080bf160 con 0x7fcb20100dc0 2024-09-16T09:01:36.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.548+0000 7fcb1f7fe640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcb0c07af30 0x7fcb0c07d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb10005fd0 tx=0x7fcb1000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:36.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.552+0000 7fcb1d7fa640 1 -- 172.21.15.31:0/3445312968 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fcb080823c0 con 0x7fcb20100dc0 2024-09-16T09:01:36.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.745+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb2002c4f0 con 0x7fcb0c07af30 2024-09-16T09:01:36.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.754+0000 7fcb1d7fa640 1 -- 172.21.15.31:0/3445312968 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcb2002c4f0 con 0x7fcb0c07af30 2024-09-16T09:01:36.755 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (11m) 14s ago 13m 25.3M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (4m) 14s ago 13m 8212k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (3m) 75s ago 12m 7578k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (83s) 14s ago 13m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (78s) 75s ago 12m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (11m) 14s ago 12m 85.5M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (6m) 14s ago 6m 16.1M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (6m) 14s ago 6m 153M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (6m) 75s ago 6m 26.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (6m) 75s ago 6m 17.4M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (3m) 14s ago 14m 511M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:01:36.756 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (2m) 75s ago 12m 452M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (2m) 14s ago 14m 47.1M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (2m) 75s ago 12m 33.6M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (13m) 14s ago 13m 15.6M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (12m) 75s ago 12m 16.4M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (56s) 14s ago 10m 93.2M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (20s) 14s ago 10m 11.1M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (9m) 14s ago 9m 219M 1217M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 e669f8865fde 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (9m) 75s ago 9m 278M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (8m) 75s ago 8m 250M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (8m) 75s ago 8m 233M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T09:01:36.757 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (2m) 14s ago 12m 61.0M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:01:36.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcb0c07af30 msgr2=0x7fcb0c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.758 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcb0c07af30 0x7fcb0c07d3f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fcb10005fd0 tx=0x7fcb1000b410 comp rx=0 tx=0).stop 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 msgr2=0x7fcb200728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb200728a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcb080054f0 tx=0x7fcb08005520 comp rx=0 tx=0).stop 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 shutdown_connections 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcb0c07af30 0x7fcb0c07d3f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb20100dc0 0x7fcb200728a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 --2- 172.21.15.31:0/3445312968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcb200ffbc0 0x7fcb20074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 >> 172.21.15.31:0/3445312968 conn(0x7fcb200fb350 msgr2=0x7fcb20104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 shutdown_connections 2024-09-16T09:01:36.759 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.757+0000 7fcb25c5c640 1 -- 172.21.15.31:0/3445312968 wait complete. 2024-09-16T09:01:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.840+0000 7f1b64832640 1 -- 172.21.15.31:0/675183139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 msgr2=0x7f1b600fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.840+0000 7f1b64832640 1 --2- 172.21.15.31:0/675183139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b600fff90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1b50009880 tx=0x7f1b5002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 -- 172.21.15.31:0/675183139 shutdown_connections 2024-09-16T09:01:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 --2- 172.21.15.31:0/675183139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b60100d90 0x7f1b60101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 --2- 172.21.15.31:0/675183139 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b600fff90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 -- 172.21.15.31:0/675183139 >> 172.21.15.31:0/675183139 conn(0x7f1b600fb320 msgr2=0x7f1b600fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 -- 172.21.15.31:0/675183139 shutdown_connections 2024-09-16T09:01:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 -- 172.21.15.31:0/675183139 wait complete. 2024-09-16T09:01:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 Processor -- start 2024-09-16T09:01:36.843 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.841+0000 7f1b64832640 1 -- start start 2024-09-16T09:01:36.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b64832640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b60195670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b64832640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b60100d90 0x7f1b60195bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b64832640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b60196180 con 0x7f1b600ffb90 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b64832640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b601962f0 con 0x7f1b60100d90 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b60195670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b60195670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49224/0 (socket says 172.21.15.31:49224) 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5e59c640 1 -- 172.21.15.31:0/3808644427 learned_addr learned my addr 172.21.15.31:0/3808644427 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5dd9b640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b60100d90 0x7f1b60195bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5e59c640 1 -- 172.21.15.31:0/3808644427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b60100d90 msgr2=0x7f1b60195bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5e59c640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b60100d90 0x7f1b60195bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5e59c640 1 -- 172.21.15.31:0/3808644427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b50009560 con 0x7f1b600ffb90 2024-09-16T09:01:36.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.842+0000 7f1b5e59c640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b60195670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b5002a9c0 tx=0x7f1b500055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:36.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.843+0000 7f1b4f7fe640 1 -- 172.21.15.31:0/3808644427 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b50004aa0 con 0x7f1b600ffb90 2024-09-16T09:01:36.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.843+0000 7f1b4f7fe640 1 -- 172.21.15.31:0/3808644427 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b50004c00 con 0x7f1b600ffb90 2024-09-16T09:01:36.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.843+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6019ad30 con 0x7f1b600ffb90 2024-09-16T09:01:36.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.843+0000 7f1b4f7fe640 1 -- 172.21.15.31:0/3808644427 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b500039b0 con 0x7f1b600ffb90 2024-09-16T09:01:36.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.843+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6019b1f0 con 0x7f1b600ffb90 2024-09-16T09:01:36.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.844+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b6018f710 con 0x7f1b600ffb90 2024-09-16T09:01:36.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.845+0000 7f1b4f7fe640 1 -- 172.21.15.31:0/3808644427 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f1b50031c00 con 0x7f1b600ffb90 2024-09-16T09:01:36.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.845+0000 7f1b4f7fe640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b4807af30 0x7f1b4807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:36.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.845+0000 7f1b4f7fe640 1 -- 172.21.15.31:0/3808644427 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1b500bf070 con 0x7f1b600ffb90 2024-09-16T09:01:36.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.845+0000 7f1b5dd9b640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b4807af30 0x7f1b4807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:36.847 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.846+0000 7f1b5dd9b640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b4807af30 0x7f1b4807d3f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b60196b90 tx=0x7f1b5400b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:36.851 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:36.850+0000 7f1b4f7fe640 1 -- 172.21.15.31:0/3808644427 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f1b500822d0 con 0x7f1b600ffb90 2024-09-16T09:01:37.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.116+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1b6002c530 con 0x7f1b600ffb90 2024-09-16T09:01:37.118 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:36 smithi031 ceph-mon[91884]: pgmap v79: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:37.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.117+0000 7f1b4f7fe640 1 -- 172.21.15.31:0/3808644427 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f1b500853f0 con 0x7f1b600ffb90 2024-09-16T09:01:37.118 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:01:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:01:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:01:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:37.118 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4, 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 8, 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:01:37.119 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:01:37.120 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b4807af30 msgr2=0x7f1b4807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b4807af30 0x7f1b4807d3f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1b60196b90 tx=0x7f1b5400b390 comp rx=0 tx=0).stop 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 msgr2=0x7f1b60195670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b60195670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b5002a9c0 tx=0x7f1b500055a0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 shutdown_connections 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1b4807af30 0x7f1b4807d3f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b60100d90 0x7f1b60195bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 --2- 172.21.15.31:0/3808644427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1b600ffb90 0x7f1b60195670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 >> 172.21.15.31:0/3808644427 conn(0x7f1b600fb320 msgr2=0x7f1b60104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 shutdown_connections 2024-09-16T09:01:37.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.119+0000 7f1b64832640 1 -- 172.21.15.31:0/3808644427 wait complete. 2024-09-16T09:01:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 -- 172.21.15.31:0/2238379408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 msgr2=0x7f94a0101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 --2- 172.21.15.31:0/2238379408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0101250 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9494009880 tx=0x7f949402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 -- 172.21.15.31:0/2238379408 shutdown_connections 2024-09-16T09:01:37.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 --2- 172.21.15.31:0/2238379408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0101250 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 --2- 172.21.15.31:0/2238379408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a00ffbd0 0x7f94a00fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 -- 172.21.15.31:0/2238379408 >> 172.21.15.31:0/2238379408 conn(0x7f94a00fb360 msgr2=0x7f94a00fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 -- 172.21.15.31:0/2238379408 shutdown_connections 2024-09-16T09:01:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 -- 172.21.15.31:0/2238379408 wait complete. 2024-09-16T09:01:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 Processor -- start 2024-09-16T09:01:37.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.203+0000 7f94a645b640 1 -- start start 2024-09-16T09:01:37.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f94a645b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a00ffbd0 0x7f94a0195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f94a645b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f94a645b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0196190 con 0x7f94a0100dd0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f94a645b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0196300 con 0x7f94a00ffbd0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0195bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49232/0 (socket says 172.21.15.31:49232) 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949f7fe640 1 -- 172.21.15.31:0/1894392143 learned_addr learned my addr 172.21.15.31:0/1894392143 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949ffff640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a00ffbd0 0x7f94a0195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949f7fe640 1 -- 172.21.15.31:0/1894392143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a00ffbd0 msgr2=0x7f94a0195680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949f7fe640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a00ffbd0 0x7f94a0195680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949f7fe640 1 -- 172.21.15.31:0/1894392143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494009560 con 0x7f94a0100dd0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.204+0000 7f949f7fe640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0195bc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9494005640 tx=0x7f9494005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.205+0000 7f949d7fa640 1 -- 172.21.15.31:0/1894392143 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9494004ba0 con 0x7f94a0100dd0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.205+0000 7f949d7fa640 1 -- 172.21.15.31:0/1894392143 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9494004d00 con 0x7f94a0100dd0 2024-09-16T09:01:37.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.205+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a019acf0 con 0x7f94a0100dd0 2024-09-16T09:01:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.205+0000 7f949d7fa640 1 -- 172.21.15.31:0/1894392143 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9494003c30 con 0x7f94a0100dd0 2024-09-16T09:01:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.205+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a019b260 con 0x7f94a0100dd0 2024-09-16T09:01:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.206+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a018f770 con 0x7f94a0100dd0 2024-09-16T09:01:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.206+0000 7f949d7fa640 1 -- 172.21.15.31:0/1894392143 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f9494031640 con 0x7f94a0100dd0 2024-09-16T09:01:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.207+0000 7f949d7fa640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f948807af30 0x7f948807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.207+0000 7f949d7fa640 1 -- 172.21.15.31:0/1894392143 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f94940bf1b0 con 0x7f94a0100dd0 2024-09-16T09:01:37.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.207+0000 7f949ffff640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f948807af30 0x7f948807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.209 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.207+0000 7f949ffff640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f948807af30 0x7f948807d3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94900058c0 tx=0x7f949000a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:37.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.211+0000 7f949d7fa640 1 -- 172.21.15.31:0/1894392143 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f9494082410 con 0x7f94a0100dd0 2024-09-16T09:01:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:36 smithi097 ceph-mon[79892]: pgmap v79: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:37.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.432+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f94a002c480 con 0x7f94a0100dd0 2024-09-16T09:01:37.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.432+0000 7f949d7fa640 1 -- 172.21.15.31:0/1894392143 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7f9494085530 con 0x7f94a0100dd0 2024-09-16T09:01:37.434 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:01:37.434 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:01:37.434 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:01:37.434 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:01:37.434 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:01:37.434 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:01:37.435 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:01:37.436 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:37.437 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:01:37.438 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:01:37.438 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:37.438 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:01:37.438 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.434+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f948807af30 msgr2=0x7f948807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.434+0000 7f94a645b640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f948807af30 0x7f948807d3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f94900058c0 tx=0x7f949000a2d0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.434+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 msgr2=0x7f94a0195bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.434+0000 7f94a645b640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0195bc0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9494005640 tx=0x7f9494005670 comp rx=0 tx=0).stop 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.435+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 shutdown_connections 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.435+0000 7f94a645b640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f948807af30 0x7f948807d3f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.435+0000 7f94a645b640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a0100dd0 0x7f94a0195bc0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.435+0000 7f94a645b640 1 --2- 172.21.15.31:0/1894392143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f94a00ffbd0 0x7f94a0195680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.435+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 >> 172.21.15.31:0/1894392143 conn(0x7f94a00fb360 msgr2=0x7f94a0104450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.435+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 shutdown_connections 2024-09-16T09:01:37.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.435+0000 7f94a645b640 1 -- 172.21.15.31:0/1894392143 wait complete. 2024-09-16T09:01:37.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 -- 172.21.15.31:0/778807019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 msgr2=0x7ff694101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 --2- 172.21.15.31:0/778807019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694101210 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff684009880 tx=0x7ff68402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 -- 172.21.15.31:0/778807019 shutdown_connections 2024-09-16T09:01:37.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 --2- 172.21.15.31:0/778807019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694101210 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 --2- 172.21.15.31:0/778807019 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6940ffb90 0x7ff6940fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 -- 172.21.15.31:0/778807019 >> 172.21.15.31:0/778807019 conn(0x7ff6940fb320 msgr2=0x7ff6940fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:37.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 -- 172.21.15.31:0/778807019 shutdown_connections 2024-09-16T09:01:37.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.519+0000 7ff69c213640 1 -- 172.21.15.31:0/778807019 wait complete. 2024-09-16T09:01:37.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff69c213640 1 Processor -- start 2024-09-16T09:01:37.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff69c213640 1 -- start start 2024-09-16T09:01:37.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff69c213640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6940ffb90 0x7ff6941979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff69c213640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff69c213640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6941984d0 con 0x7ff694100d90 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff69c213640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff694198640 con 0x7ff6940ffb90 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff6997ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff6997ae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49254/0 (socket says 172.21.15.31:49254) 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff6997ae640 1 -- 172.21.15.31:0/2164744522 learned_addr learned my addr 172.21.15.31:0/2164744522 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff699faf640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6940ffb90 0x7ff6941979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff6997ae640 1 -- 172.21.15.31:0/2164744522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6940ffb90 msgr2=0x7ff6941979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff6997ae640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6940ffb90 0x7ff6941979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff6997ae640 1 -- 172.21.15.31:0/2164744522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff684009560 con 0x7ff694100d90 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.520+0000 7ff699faf640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6940ffb90 0x7ff6941979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.521+0000 7ff6997ae640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694197f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff684005640 tx=0x7ff684005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.521+0000 7ff68affd640 1 -- 172.21.15.31:0/2164744522 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff684004ba0 con 0x7ff694100d90 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.521+0000 7ff68affd640 1 -- 172.21.15.31:0/2164744522 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff684004d00 con 0x7ff694100d90 2024-09-16T09:01:37.523 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.521+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69419d030 con 0x7ff694100d90 2024-09-16T09:01:37.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.521+0000 7ff68affd640 1 -- 172.21.15.31:0/2164744522 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff684003c30 con 0x7ff694100d90 2024-09-16T09:01:37.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.521+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69419d5a0 con 0x7ff694100d90 2024-09-16T09:01:37.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.522+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6941056d0 con 0x7ff694100d90 2024-09-16T09:01:37.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.522+0000 7ff68affd640 1 -- 172.21.15.31:0/2164744522 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7ff684031640 con 0x7ff694100d90 2024-09-16T09:01:37.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.523+0000 7ff68affd640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff68007af30 0x7ff68007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.523+0000 7ff68affd640 1 -- 172.21.15.31:0/2164744522 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7ff6840bf230 con 0x7ff694100d90 2024-09-16T09:01:37.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.523+0000 7ff699faf640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff68007af30 0x7ff68007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.523+0000 7ff699faf640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff68007af30 0x7ff68007d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff690006fd0 tx=0x7ff690008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:37.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.527+0000 7ff68affd640 1 -- 172.21.15.31:0/2164744522 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ff6840823e0 con 0x7ff694100d90 2024-09-16T09:01:37.732 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.731+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff69402c4f0 con 0x7ff68007af30 2024-09-16T09:01:37.733 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.732+0000 7ff68affd640 1 -- 172.21.15.31:0/2164744522 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff69402c4f0 con 0x7ff68007af30 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "8/23 daemons upgraded", 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:01:37.734 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:01:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff68007af30 msgr2=0x7ff68007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff68007af30 0x7ff68007d3f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff690006fd0 tx=0x7ff690008040 comp rx=0 tx=0).stop 2024-09-16T09:01:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 msgr2=0x7ff694197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.736 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694197f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff684005640 tx=0x7ff684005670 comp rx=0 tx=0).stop 2024-09-16T09:01:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 shutdown_connections 2024-09-16T09:01:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff68007af30 0x7ff68007d3f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff694100d90 0x7ff694197f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 --2- 172.21.15.31:0/2164744522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff6940ffb90 0x7ff6941979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 >> 172.21.15.31:0/2164744522 conn(0x7ff6940fb320 msgr2=0x7ff694072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 shutdown_connections 2024-09-16T09:01:37.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.735+0000 7ff69c213640 1 -- 172.21.15.31:0/2164744522 wait complete. 2024-09-16T09:01:37.820 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 -- 172.21.15.31:0/884133283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe800ffb90 msgr2=0x7fbe800fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 --2- 172.21.15.31:0/884133283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe800ffb90 0x7fbe800fff90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009880 tx=0x7fbe7002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 -- 172.21.15.31:0/884133283 shutdown_connections 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 --2- 172.21.15.31:0/884133283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe80100d90 0x7fbe80101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 --2- 172.21.15.31:0/884133283 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe800ffb90 0x7fbe800fff90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 -- 172.21.15.31:0/884133283 >> 172.21.15.31:0/884133283 conn(0x7fbe800fb320 msgr2=0x7fbe800fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 -- 172.21.15.31:0/884133283 shutdown_connections 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.819+0000 7fbe852ec640 1 -- 172.21.15.31:0/884133283 wait complete. 2024-09-16T09:01:37.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe852ec640 1 Processor -- start 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe852ec640 1 -- start start 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe852ec640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe800ffb90 0x7fbe80195690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe852ec640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80100d90 0x7fbe80195bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe852ec640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801961a0 con 0x7fbe80100d90 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe852ec640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80196310 con 0x7fbe800ffb90 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe7e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80100d90 0x7fbe80195bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe7e7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80100d90 0x7fbe80195bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:49282/0 (socket says 172.21.15.31:49282) 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe7e7fc640 1 -- 172.21.15.31:0/1879757378 learned_addr learned my addr 172.21.15.31:0/1879757378 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe7effd640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe800ffb90 0x7fbe80195690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe7e7fc640 1 -- 172.21.15.31:0/1879757378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe800ffb90 msgr2=0x7fbe80195690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:37.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.820+0000 7fbe7e7fc640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe800ffb90 0x7fbe80195690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.821+0000 7fbe7e7fc640 1 -- 172.21.15.31:0/1879757378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe70009560 con 0x7fbe80100d90 2024-09-16T09:01:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.821+0000 7fbe7e7fc640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80100d90 0x7fbe80195bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400d720 tx=0x7fbe7400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.821+0000 7fbe67fff640 1 -- 172.21.15.31:0/1879757378 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe740051f0 con 0x7fbe80100d90 2024-09-16T09:01:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.821+0000 7fbe67fff640 1 -- 172.21.15.31:0/1879757378 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe74009ce0 con 0x7fbe80100d90 2024-09-16T09:01:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.821+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8019ad60 con 0x7fbe80100d90 2024-09-16T09:01:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.821+0000 7fbe67fff640 1 -- 172.21.15.31:0/1879757378 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe74010460 con 0x7fbe80100d90 2024-09-16T09:01:37.824 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.821+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8019b330 con 0x7fbe80100d90 2024-09-16T09:01:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.822+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8018f730 con 0x7fbe80100d90 2024-09-16T09:01:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.823+0000 7fbe67fff640 1 -- 172.21.15.31:0/1879757378 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fbe74005350 con 0x7fbe80100d90 2024-09-16T09:01:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.823+0000 7fbe67fff640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbe6807af30 0x7fbe6807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:01:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.824+0000 7fbe67fff640 1 -- 172.21.15.31:0/1879757378 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fbe7409dad0 con 0x7fbe80100d90 2024-09-16T09:01:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.824+0000 7fbe7effd640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbe6807af30 0x7fbe6807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:01:37.825 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.824+0000 7fbe7effd640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbe6807af30 0x7fbe6807d3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbe70006010 tx=0x7fbe70031500 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:01:37.829 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:37.828+0000 7fbe67fff640 1 -- 172.21.15.31:0/1879757378 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fbe74060c80 con 0x7fbe80100d90 2024-09-16T09:01:38.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.088+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbe801089c0 con 0x7fbe80100d90 2024-09-16T09:01:38.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:37 smithi031 ceph-mon[91884]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:38.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:37 smithi031 ceph-mon[91884]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:38.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:37 smithi031 ceph-mon[91884]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:38.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:37 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3808644427' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:38.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:37 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1894392143' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:01:38.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.089+0000 7fbe67fff640 1 -- 172.21.15.31:0/1879757378 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fbe74063da0 con 0x7fbe80100d90 2024-09-16T09:01:38.091 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbe6807af30 msgr2=0x7fbe6807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbe6807af30 0x7fbe6807d3f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbe70006010 tx=0x7fbe70031500 comp rx=0 tx=0).stop 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80100d90 msgr2=0x7fbe80195bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80100d90 0x7fbe80195bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400d720 tx=0x7fbe7400dbe0 comp rx=0 tx=0).stop 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 shutdown_connections 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbe6807af30 0x7fbe6807d3f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe80100d90 0x7fbe80195bd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 --2- 172.21.15.31:0/1879757378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbe800ffb90 0x7fbe80195690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 >> 172.21.15.31:0/1879757378 conn(0x7fbe800fb320 msgr2=0x7fbe801044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.091+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 shutdown_connections 2024-09-16T09:01:38.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:01:38.092+0000 7fbe852ec640 1 -- 172.21.15.31:0/1879757378 wait complete. 2024-09-16T09:01:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:37 smithi097 ceph-mon[79892]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:37 smithi097 ceph-mon[79892]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:37 smithi097 ceph-mon[79892]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:37 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3808644427' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:01:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:37 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1894392143' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:01:38.862 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:38 smithi097 ceph-mon[79892]: pgmap v80: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:38.863 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:38 smithi097 ceph-mon[79892]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:38.863 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:38 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1879757378' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:01:39.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:38 smithi031 ceph-mon[91884]: pgmap v80: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:39.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:38 smithi031 ceph-mon[91884]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:01:39.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:38 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1879757378' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:01:39.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:39.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:39.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:41.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:40 smithi031 ceph-mon[91884]: pgmap v81: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:41.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:40 smithi097 ceph-mon[79892]: pgmap v81: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-16T09:01:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:41 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T09:01:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:41 smithi031 ceph-mon[91884]: Upgrade: osd.2 is safe to restart 2024-09-16T09:01:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16} : dispatch 2024-09-16T09:01:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:41 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-16T09:01:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:41 smithi097 ceph-mon[79892]: Upgrade: osd.2 is safe to restart 2024-09-16T09:01:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:42 smithi031 ceph-mon[91884]: pgmap v82: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T09:01:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:42 smithi097 ceph-mon[79892]: pgmap v82: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T09:01:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:01:44.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:43 smithi031 ceph-mon[91884]: Upgrade: Updating osd.2 2024-09-16T09:01:44.011 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:43 smithi031 ceph-mon[91884]: Deploying daemon osd.2 on smithi031 2024-09-16T09:01:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:43 smithi097 ceph-mon[79892]: Upgrade: Updating osd.2 2024-09-16T09:01:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:43 smithi097 ceph-mon[79892]: Deploying daemon osd.2 on smithi031 2024-09-16T09:01:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:44 smithi097 ceph-mon[79892]: pgmap v83: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:44 smithi031 ceph-mon[91884]: pgmap v83: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:45.373 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:45 smithi031 systemd[1]: Stopping Ceph osd.2 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:01:45.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:45 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[64315]: 2024-09-16T09:01:45.409+0000 7f2773822640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:01:45.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:45 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[64315]: 2024-09-16T09:01:45.409+0000 7f2773822640 -1 osd.2 60 *** Got signal Terminated *** 2024-09-16T09:01:45.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:45 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[64315]: 2024-09-16T09:01:45.409+0000 7f2773822640 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:01:46.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:45 smithi097 ceph-mon[79892]: osd.2 marked itself down and dead 2024-09-16T09:01:46.188 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:45 smithi031 ceph-mon[91884]: osd.2 marked itself down and dead 2024-09-16T09:01:46.612 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:46 smithi031 podman[110050]: 2024-09-16 09:01:46.300439309 +0000 UTC m=+1.001641732 container died e669f8865fde831b4998e3758b6b39d5e9382b9e611a8d2ec48e265afcd7cf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-16T09:01:46.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:46 smithi031 podman[110050]: 2024-09-16 09:01:46.718235886 +0000 UTC m=+1.419438307 container cleanup e669f8865fde831b4998e3758b6b39d5e9382b9e611a8d2ec48e265afcd7cf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-16T09:01:46.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:46 smithi031 bash[110050]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2 2024-09-16T09:01:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:46 smithi097 ceph-mon[79892]: pgmap v84: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:46 smithi097 ceph-mon[79892]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:01:47.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:46 smithi097 ceph-mon[79892]: osdmap e61: 6 total, 5 up, 6 in 2024-09-16T09:01:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:46 smithi031 ceph-mon[91884]: pgmap v84: 65 pgs: 65 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:46 smithi031 ceph-mon[91884]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:01:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:46 smithi031 ceph-mon[91884]: osdmap e61: 6 total, 5 up, 6 in 2024-09-16T09:01:47.373 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:46 smithi031 podman[110064]: 2024-09-16 09:01:46.96139885 +0000 UTC m=+0.658843244 container remove e669f8865fde831b4998e3758b6b39d5e9382b9e611a8d2ec48e265afcd7cf1f (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:01:47.639 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:47 smithi031 podman[110156]: 2024-09-16 09:01:47.378627059 +0000 UTC m=+0.096715940 container create a7f59d536c9f8ab2058e3a5d7c5639d86709b80596193640ccb1065a0efa1f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-09-16T09:01:47.640 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:47 smithi031 podman[110156]: 2024-09-16 09:01:47.298268023 +0000 UTC m=+0.016356920 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:01:47.640 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:47 smithi031 podman[110156]: 2024-09-16 09:01:47.639336934 +0000 UTC m=+0.357425817 container init a7f59d536c9f8ab2058e3a5d7c5639d86709b80596193640ccb1065a0efa1f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731) 2024-09-16T09:01:47.915 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:47 smithi031 podman[110156]: 2024-09-16 09:01:47.642894285 +0000 UTC m=+0.360983171 container start a7f59d536c9f8ab2058e3a5d7c5639d86709b80596193640ccb1065a0efa1f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T09:01:47.915 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:47 smithi031 podman[110156]: 2024-09-16 09:01:47.72260142 +0000 UTC m=+0.440690310 container attach a7f59d536c9f8ab2058e3a5d7c5639d86709b80596193640ccb1065a0efa1f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T09:01:47.915 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:47 smithi031 podman[110156]: 2024-09-16 09:01:47.84332114 +0000 UTC m=+0.561410029 container died a7f59d536c9f8ab2058e3a5d7c5639d86709b80596193640ccb1065a0efa1f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T09:01:48.184 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:47 smithi031 ceph-mon[91884]: osdmap e62: 6 total, 5 up, 6 in 2024-09-16T09:01:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:47 smithi097 ceph-mon[79892]: osdmap e62: 6 total, 5 up, 6 in 2024-09-16T09:01:48.714 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:48 smithi031 podman[110179]: 2024-09-16 09:01:48.459607768 +0000 UTC m=+0.610412785 container remove a7f59d536c9f8ab2058e3a5d7c5639d86709b80596193640ccb1065a0efa1f3f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-16T09:01:48.714 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:48 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-16T09:01:48.714 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:48 smithi031 systemd[1]: Stopped Ceph osd.2 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:01:48.714 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:48 smithi031 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.2.service: Consumed 57.680s CPU time. 2024-09-16T09:01:49.025 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:48 smithi031 ceph-mon[91884]: pgmap v87: 65 pgs: 9 stale+active+clean, 56 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:49.025 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:48 smithi031 systemd[1]: Starting Ceph osd.2 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:01:49.026 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:48 smithi031 podman[110277]: 2024-09-16 09:01:48.828119345 +0000 UTC m=+0.016328752 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:01:49.026 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:48 smithi031 podman[110277]: 2024-09-16 09:01:48.944192531 +0000 UTC m=+0.132401937 container create 591696adb2d028d81072e9e1970951a77e4f6c3167366bbf97e20e80cdbf516a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:01:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:48 smithi097 ceph-mon[79892]: pgmap v87: 65 pgs: 9 stale+active+clean, 56 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:01:49.278 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:49 smithi031 podman[110277]: 2024-09-16 09:01:49.138269269 +0000 UTC m=+0.326478675 container init 591696adb2d028d81072e9e1970951a77e4f6c3167366bbf97e20e80cdbf516a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T09:01:49.278 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:49 smithi031 podman[110277]: 2024-09-16 09:01:49.141891523 +0000 UTC m=+0.330100933 container start 591696adb2d028d81072e9e1970951a77e4f6c3167366bbf97e20e80cdbf516a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T09:01:49.278 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:49 smithi031 podman[110277]: 2024-09-16 09:01:49.238046138 +0000 UTC m=+0.426255556 container attach 591696adb2d028d81072e9e1970951a77e4f6c3167366bbf97e20e80cdbf516a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:01:49.623 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:49 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:49.623 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:49 smithi031 bash[110277]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:49.623 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:49 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:49.623 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:49 smithi031 bash[110277]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:01:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:49 smithi097 ceph-mon[79892]: Health check failed: Degraded data redundancy: 24/282 objects degraded (8.511%), 9 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:49 smithi031 ceph-mon[91884]: Health check failed: Degraded data redundancy: 24/282 objects degraded (8.511%), 9 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:51.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:50 smithi097 ceph-mon[79892]: pgmap v88: 65 pgs: 11 active+undersized, 4 stale+active+clean, 9 active+undersized+degraded, 41 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24/282 objects degraded (8.511%) 2024-09-16T09:01:51.272 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:50 smithi031 ceph-mon[91884]: pgmap v88: 65 pgs: 11 active+undersized, 4 stale+active+clean, 9 active+undersized+degraded, 41 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24/282 objects degraded (8.511%) 2024-09-16T09:01:51.927 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:01:51.927 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 bash[110277]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:01:51.927 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T09:01:51.927 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 bash[110277]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T09:01:51.927 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T09:01:51.927 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 bash[110277]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T09:01:52.268 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T09:01:52.268 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 bash[110277]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T09:01:52.268 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T09:01:52.268 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 bash[110277]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T09:01:52.268 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:01:52.269 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 bash[110277]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:01:52.269 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate[110292]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T09:01:52.269 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 bash[110277]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T09:01:52.269 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:51 smithi031 podman[110277]: 2024-09-16 09:01:51.963471827 +0000 UTC m=+3.151681234 container died 591696adb2d028d81072e9e1970951a77e4f6c3167366bbf97e20e80cdbf516a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T09:01:52.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:52 smithi031 podman[110463]: 2024-09-16 09:01:52.574805355 +0000 UTC m=+0.600566637 container remove 591696adb2d028d81072e9e1970951a77e4f6c3167366bbf97e20e80cdbf516a (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-activate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T09:01:52.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:52 smithi031 podman[110521]: 2024-09-16 09:01:52.775689585 +0000 UTC m=+0.096969005 container create cd7af82048da4de1c21ebde7f11f05ef398f32080e0d944675f02b1108ecc708 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-09-16T09:01:52.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:52 smithi031 podman[110521]: 2024-09-16 09:01:52.695677426 +0000 UTC m=+0.016956855 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:01:53.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:52 smithi097 ceph-mon[79892]: pgmap v89: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:52 smithi031 ceph-mon[91884]: pgmap v89: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:53.458 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:53 smithi031 podman[110521]: 2024-09-16 09:01:53.002899442 +0000 UTC m=+0.324178861 container init cd7af82048da4de1c21ebde7f11f05ef398f32080e0d944675f02b1108ecc708 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T09:01:53.458 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:53 smithi031 podman[110521]: 2024-09-16 09:01:53.00627832 +0000 UTC m=+0.327557740 container start cd7af82048da4de1c21ebde7f11f05ef398f32080e0d944675f02b1108ecc708 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:01:53.458 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:53 smithi031 ceph-osd[110541]: -- 172.21.15.31:0/665913740 <== mon.0 v2:172.21.15.31:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583e409e1e0 con 0x5583e32a9c00 2024-09-16T09:01:53.458 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:53 smithi031 bash[110521]: cd7af82048da4de1c21ebde7f11f05ef398f32080e0d944675f02b1108ecc708 2024-09-16T09:01:53.458 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:53 smithi031 systemd[1]: Started Ceph osd.2 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:01:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:54 smithi031 ceph-mon[91884]: pgmap v90: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:54.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:01:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:54 smithi097 ceph-mon[79892]: pgmap v90: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:01:54.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:01:54 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[110537]: 2024-09-16T09:01:54.662+0000 7fa8f4ed2640 -1 Falling back to public interface 2024-09-16T09:01:56.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:56 smithi031 ceph-mon[91884]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:56.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:56 smithi097 ceph-mon[79892]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:57.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:57 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 38/282 objects degraded (13.475%), 12 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:57.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:57 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 38/282 objects degraded (13.475%), 12 pgs degraded (PG_DEGRADED) 2024-09-16T09:01:58.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:58 smithi031 ceph-mon[91884]: pgmap v92: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:58.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:58.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:58.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:58 smithi097 ceph-mon[79892]: pgmap v92: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:01:58.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:01:58.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:00.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:59 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:00.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:01:59 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:00.305 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:59 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:00.305 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:01:59 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:00.873 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:02:00 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[110537]: 2024-09-16T09:02:00.630+0000 7fa8f4ed2640 -1 osd.2 60 log_to_monitors true 2024-09-16T09:02:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:00 smithi097 ceph-mon[79892]: pgmap v93: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:02:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:00 smithi097 ceph-mon[79892]: from='osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:02:01.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:00 smithi031 ceph-mon[91884]: pgmap v93: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:02:01.371 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:00 smithi031 ceph-mon[91884]: from='osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:02:01.371 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:02:00 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[110537]: 2024-09-16T09:02:00.930+0000 7fa8ecc7c640 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:02:02.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:01 smithi031 ceph-mon[91884]: from='osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:02:02.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:01 smithi031 ceph-mon[91884]: osdmap e63: 6 total, 5 up, 6 in 2024-09-16T09:02:02.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:01 smithi031 ceph-mon[91884]: from='osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-16T09:02:02.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:01 smithi097 ceph-mon[79892]: from='osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:02:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:01 smithi097 ceph-mon[79892]: osdmap e63: 6 total, 5 up, 6 in 2024-09-16T09:02:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:01 smithi097 ceph-mon[79892]: from='osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi031", "root=default"]} : dispatch 2024-09-16T09:02:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: pgmap v95: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:02:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:02:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774] boot 2024-09-16T09:02:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: osdmap e64: 6 total, 6 up, 6 in 2024-09-16T09:02:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-16T09:02:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: pgmap v95: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:02:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: osd.2 [v2:172.21.15.31:6818/3196153774,v1:172.21.15.31:6819/3196153774] boot 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: osdmap e64: 6 total, 6 up, 6 in 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-16T09:02:04.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:03 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T09:02:04.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:03 smithi097 ceph-mon[79892]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T09:02:04.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:03 smithi097 ceph-mon[79892]: osdmap e65: 6 total, 6 up, 6 in 2024-09-16T09:02:04.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:03 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T09:02:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:03 smithi031 ceph-mon[91884]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-16T09:02:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:03 smithi031 ceph-mon[91884]: osdmap e65: 6 total, 6 up, 6 in 2024-09-16T09:02:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:04 smithi097 ceph-mon[79892]: pgmap v98: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:02:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:04 smithi031 ceph-mon[91884]: pgmap v98: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 38/282 objects degraded (13.475%) 2024-09-16T09:02:06.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:05 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 21/282 objects degraded (7.447%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:06.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:05 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 21/282 objects degraded (7.447%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:06 smithi097 ceph-mon[79892]: pgmap v99: 65 pgs: 6 active+undersized, 6 active+undersized+degraded, 53 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/282 objects degraded (7.447%) 2024-09-16T09:02:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:06 smithi031 ceph-mon[91884]: pgmap v99: 65 pgs: 6 active+undersized, 6 active+undersized+degraded, 53 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 21/282 objects degraded (7.447%) 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 -- 172.21.15.31:0/990442573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 msgr2=0x7efdac0feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 --2- 172.21.15.31:0/990442573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac0feef0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009880 tx=0x7efd9c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 -- 172.21.15.31:0/990442573 shutdown_connections 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 --2- 172.21.15.31:0/990442573 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdac0ffcf0 0x7efdac100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 --2- 172.21.15.31:0/990442573 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac0feef0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 -- 172.21.15.31:0/990442573 >> 172.21.15.31:0/990442573 conn(0x7efdac0fa280 msgr2=0x7efdac0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 -- 172.21.15.31:0/990442573 shutdown_connections 2024-09-16T09:02:08.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.189+0000 7efdb22b0640 1 -- 172.21.15.31:0/990442573 wait complete. 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdb22b0640 1 Processor -- start 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdb22b0640 1 -- start start 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdb22b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdb22b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdac0ffcf0 0x7efdac197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdb22b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac1983d0 con 0x7efdac0feaf0 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdb22b0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdac198540 con 0x7efdac0ffcf0 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdabfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdabfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43170/0 (socket says 172.21.15.31:43170) 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.190+0000 7efdabfff640 1 -- 172.21.15.31:0/1999928973 learned_addr learned my addr 172.21.15.31:0/1999928973 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdab7fe640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdac0ffcf0 0x7efdac197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdabfff640 1 -- 172.21.15.31:0/1999928973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdac0ffcf0 msgr2=0x7efdac197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdabfff640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdac0ffcf0 0x7efdac197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdabfff640 1 -- 172.21.15.31:0/1999928973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c009560 con 0x7efdac0feaf0 2024-09-16T09:02:08.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdab7fe640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdac0ffcf0 0x7efdac197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:02:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdabfff640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac1978c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0054f0 tx=0x7efd9c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efda97fa640 1 -- 172.21.15.31:0/1999928973 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c004aa0 con 0x7efdac0feaf0 2024-09-16T09:02:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efda97fa640 1 -- 172.21.15.31:0/1999928973 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd9c004c00 con 0x7efdac0feaf0 2024-09-16T09:02:08.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdac19cf80 con 0x7efdac0feaf0 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efda97fa640 1 -- 172.21.15.31:0/1999928973 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd9c003ac0 con 0x7efdac0feaf0 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.191+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdac19d3f0 con 0x7efdac0feaf0 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.192+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdac107700 con 0x7efdac0feaf0 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.193+0000 7efda97fa640 1 -- 172.21.15.31:0/1999928973 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7efd9c003d30 con 0x7efdac0feaf0 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.194+0000 7efda97fa640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efd9407af30 0x7efd9407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.194+0000 7efda97fa640 1 -- 172.21.15.31:0/1999928973 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6500+0+0 (secure 0 0 0) 0x7efd9c0bf1a0 con 0x7efdac0feaf0 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.194+0000 7efdab7fe640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efd9407af30 0x7efd9407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.195 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.194+0000 7efdab7fe640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efd9407af30 0x7efd9407d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efda00058c0 tx=0x7efda0005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:08.199 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.198+0000 7efda97fa640 1 -- 172.21.15.31:0/1999928973 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7efd9c0823d0 con 0x7efdac0feaf0 2024-09-16T09:02:08.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:08 smithi031 ceph-mon[91884]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/282 objects degraded (7.447%), 6 pgs degraded) 2024-09-16T09:02:08.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:08 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:02:08.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.405+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efdac068430 con 0x7efd9407af30 2024-09-16T09:02:08.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.406+0000 7efda97fa640 1 -- 172.21.15.31:0/1999928973 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7efdac068430 con 0x7efd9407af30 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efd9407af30 msgr2=0x7efd9407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efd9407af30 0x7efd9407d3f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efda00058c0 tx=0x7efda0005850 comp rx=0 tx=0).stop 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 msgr2=0x7efdac1978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac1978c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd9c0054f0 tx=0x7efd9c005520 comp rx=0 tx=0).stop 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 shutdown_connections 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efd9407af30 0x7efd9407d3f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efdac0ffcf0 0x7efdac197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 --2- 172.21.15.31:0/1999928973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efdac0feaf0 0x7efdac1978c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 >> 172.21.15.31:0/1999928973 conn(0x7efdac0fa280 msgr2=0x7efdac067d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 shutdown_connections 2024-09-16T09:02:08.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.409+0000 7efdb22b0640 1 -- 172.21.15.31:0/1999928973 wait complete. 2024-09-16T09:02:08.419 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.493+0000 7f91648da640 1 -- 172.21.15.31:0/62274025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160100680 msgr2=0x7f9160102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.493+0000 7f91648da640 1 --2- 172.21.15.31:0/62274025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160100680 0x7f9160102a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f91500098b0 tx=0x7f915002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 -- 172.21.15.31:0/62274025 shutdown_connections 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 --2- 172.21.15.31:0/62274025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9160102fb0 0x7f91601053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 --2- 172.21.15.31:0/62274025 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160100680 0x7f9160102a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 -- 172.21.15.31:0/62274025 >> 172.21.15.31:0/62274025 conn(0x7f91600fa280 msgr2=0x7f91600fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 -- 172.21.15.31:0/62274025 shutdown_connections 2024-09-16T09:02:08.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 -- 172.21.15.31:0/62274025 wait complete. 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 Processor -- start 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.494+0000 7f91648da640 1 -- start start 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f91648da640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9160100680 0x7f91601956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f91648da640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160102fb0 0x7f9160195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f91648da640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9160196200 con 0x7f9160102fb0 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f91648da640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9160196370 con 0x7f9160100680 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160102fb0 0x7f9160195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.497 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915dd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160102fb0 0x7f9160195c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43186/0 (socket says 172.21.15.31:43186) 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915dd9b640 1 -- 172.21.15.31:0/458049484 learned_addr learned my addr 172.21.15.31:0/458049484 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915e59c640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9160100680 0x7f91601956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915dd9b640 1 -- 172.21.15.31:0/458049484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9160100680 msgr2=0x7f91601956f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915dd9b640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9160100680 0x7f91601956f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915dd9b640 1 -- 172.21.15.31:0/458049484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9150009560 con 0x7f9160102fb0 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.495+0000 7f915dd9b640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160102fb0 0x7f9160195c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f915400c920 tx=0x7f915400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.496+0000 7f914f7fe640 1 -- 172.21.15.31:0/458049484 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9154007b80 con 0x7f9160102fb0 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.496+0000 7f914f7fe640 1 -- 172.21.15.31:0/458049484 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f91540054e0 con 0x7f9160102fb0 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.496+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916019ae10 con 0x7f9160102fb0 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.496+0000 7f914f7fe640 1 -- 172.21.15.31:0/458049484 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f915400f450 con 0x7f9160102fb0 2024-09-16T09:02:08.498 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.496+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916019b330 con 0x7f9160102fb0 2024-09-16T09:02:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.497+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9160100a60 con 0x7f9160102fb0 2024-09-16T09:02:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.497+0000 7f914f7fe640 1 -- 172.21.15.31:0/458049484 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f91540046c0 con 0x7f9160102fb0 2024-09-16T09:02:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.498+0000 7f914f7fe640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f914807af30 0x7f914807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.498+0000 7f914f7fe640 1 -- 172.21.15.31:0/458049484 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f915409dd10 con 0x7f9160102fb0 2024-09-16T09:02:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.498+0000 7f915e59c640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f914807af30 0x7f914807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.498+0000 7f915e59c640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f914807af30 0x7f914807d3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9150006010 tx=0x7f9150031500 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:08.503 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.502+0000 7f914f7fe640 1 -- 172.21.15.31:0/458049484 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f9154060ec0 con 0x7f9160102fb0 2024-09-16T09:02:08.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:08 smithi097 ceph-mon[79892]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/282 objects degraded (7.447%), 6 pgs degraded) 2024-09-16T09:02:08.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:08 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:02:08.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.706+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f91601039d0 con 0x7f914807af30 2024-09-16T09:02:08.709 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.708+0000 7f914f7fe640 1 -- 172.21.15.31:0/458049484 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f91601039d0 con 0x7f914807af30 2024-09-16T09:02:08.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f914807af30 msgr2=0x7f914807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f914807af30 0x7f914807d3f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9150006010 tx=0x7f9150031500 comp rx=0 tx=0).stop 2024-09-16T09:02:08.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160102fb0 msgr2=0x7f9160195c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160102fb0 0x7f9160195c30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f915400c920 tx=0x7f915400cde0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 shutdown_connections 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f914807af30 0x7f914807d3f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9160102fb0 0x7f9160195c30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 --2- 172.21.15.31:0/458049484 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9160100680 0x7f91601956f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 >> 172.21.15.31:0/458049484 conn(0x7f91600fa280 msgr2=0x7f91600fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 shutdown_connections 2024-09-16T09:02:08.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.710+0000 7f91648da640 1 -- 172.21.15.31:0/458049484 wait complete. 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 -- 172.21.15.31:0/2531526392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128100680 msgr2=0x7f5128102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 --2- 172.21.15.31:0/2531526392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128100680 0x7f5128102a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f51180098b0 tx=0x7f511802a4d0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 -- 172.21.15.31:0/2531526392 shutdown_connections 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 --2- 172.21.15.31:0/2531526392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5128102fb0 0x7f51281053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 --2- 172.21.15.31:0/2531526392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128100680 0x7f5128102a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 -- 172.21.15.31:0/2531526392 >> 172.21.15.31:0/2531526392 conn(0x7f51280fa280 msgr2=0x7f51280fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 -- 172.21.15.31:0/2531526392 shutdown_connections 2024-09-16T09:02:08.796 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.793+0000 7f512f5f9640 1 -- 172.21.15.31:0/2531526392 wait complete. 2024-09-16T09:02:08.797 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512f5f9640 1 Processor -- start 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512f5f9640 1 -- start start 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512f5f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5128100680 0x7f51280fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512f5f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128102fb0 0x7f51280fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512f5f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51280feb70 con 0x7f5128102fb0 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512f5f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51280fece0 con 0x7f5128100680 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512cb94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128102fb0 0x7f51280fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512cb94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128102fb0 0x7f51280fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43198/0 (socket says 172.21.15.31:43198) 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.794+0000 7f512cb94640 1 -- 172.21.15.31:0/3781141980 learned_addr learned my addr 172.21.15.31:0/3781141980 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f512d395640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5128100680 0x7f51280fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f512cb94640 1 -- 172.21.15.31:0/3781141980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5128100680 msgr2=0x7f51280fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f512cb94640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5128100680 0x7f51280fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f512cb94640 1 -- 172.21.15.31:0/3781141980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009560 con 0x7f5128102fb0 2024-09-16T09:02:08.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f512cb94640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128102fb0 0x7f51280fe630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f512400c8f0 tx=0x7f512400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f511e7fc640 1 -- 172.21.15.31:0/3781141980 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5124007b80 con 0x7f5128102fb0 2024-09-16T09:02:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f511e7fc640 1 -- 172.21.15.31:0/3781141980 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5124007ce0 con 0x7f5128102fb0 2024-09-16T09:02:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51280fefc0 con 0x7f5128102fb0 2024-09-16T09:02:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f511e7fc640 1 -- 172.21.15.31:0/3781141980 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51240187b0 con 0x7f5128102fb0 2024-09-16T09:02:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.795+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512819d770 con 0x7f5128102fb0 2024-09-16T09:02:08.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.797+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5128100a60 con 0x7f5128102fb0 2024-09-16T09:02:08.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.800+0000 7f511e7fc640 1 -- 172.21.15.31:0/3781141980 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f512400f450 con 0x7f5128102fb0 2024-09-16T09:02:08.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.800+0000 7f511e7fc640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f511407af30 0x7f511407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:08.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.800+0000 7f511e7fc640 1 -- 172.21.15.31:0/3781141980 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f512409e390 con 0x7f5128102fb0 2024-09-16T09:02:08.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.800+0000 7f512d395640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f511407af30 0x7f511407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:08.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.801+0000 7f512d395640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f511407af30 0x7f511407d3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5118006010 tx=0x7f5118005b50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:08.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.803+0000 7f511e7fc640 1 -- 172.21.15.31:0/3781141980 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f5124061660 con 0x7f5128102fb0 2024-09-16T09:02:08.998 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:08.995+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51281039d0 con 0x7f511407af30 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.006+0000 7f511e7fc640 1 -- 172.21.15.31:0/3781141980 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f51281039d0 con 0x7f511407af30 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (11m) 10s ago 13m 25.3M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (4m) 10s ago 13m 8476k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (4m) 107s ago 12m 7578k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (115s) 10s ago 13m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (110s) 107s ago 12m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (11m) 10s ago 13m 85.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (7m) 10s ago 7m 16.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (7m) 10s ago 7m 153M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:02:09.008 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (7m) 107s ago 7m 26.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:02:09.009 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (7m) 107s ago 7m 17.4M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:02:09.009 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (4m) 10s ago 14m 516M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:02:09.009 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (3m) 107s ago 12m 452M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:02:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (2m) 10s ago 15m 48.2M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:02:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (2m) 107s ago 12m 33.6M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:02:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (13m) 10s ago 13m 15.8M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:02:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (12m) 107s ago 12m 16.4M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:02:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (88s) 10s ago 11m 93.6M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:02:09.012 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (52s) 10s ago 10m 98.7M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:02:09.013 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (16s) 10s ago 10m 13.3M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:02:09.013 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (9m) 107s ago 9m 278M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 eeedb0a2c952 2024-09-16T09:02:09.013 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (9m) 107s ago 9m 250M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T09:02:09.013 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (8m) 107s ago 8m 233M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T09:02:09.013 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 10s ago 13m 65.2M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:02:09.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f511407af30 msgr2=0x7f511407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f511407af30 0x7f511407d3f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5118006010 tx=0x7f5118005b50 comp rx=0 tx=0).stop 2024-09-16T09:02:09.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128102fb0 msgr2=0x7f51280fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128102fb0 0x7f51280fe630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f512400c8f0 tx=0x7f512400cdb0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 shutdown_connections 2024-09-16T09:02:09.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f511407af30 0x7f511407d3f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5128102fb0 0x7f51280fe630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 --2- 172.21.15.31:0/3781141980 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5128100680 0x7f51280fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 >> 172.21.15.31:0/3781141980 conn(0x7f51280fa280 msgr2=0x7f51280fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:09.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 shutdown_connections 2024-09-16T09:02:09.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.008+0000 7f512f5f9640 1 -- 172.21.15.31:0/3781141980 wait complete. 2024-09-16T09:02:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 -- 172.21.15.31:0/1451769007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 msgr2=0x7f23bc0feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 --2- 172.21.15.31:0/1451769007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc0feea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009880 tx=0x7f23ac02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 -- 172.21.15.31:0/1451769007 shutdown_connections 2024-09-16T09:02:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 --2- 172.21.15.31:0/1451769007 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0ffca0 0x7f23bc100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 --2- 172.21.15.31:0/1451769007 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc0feea0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 -- 172.21.15.31:0/1451769007 >> 172.21.15.31:0/1451769007 conn(0x7f23bc0fa230 msgr2=0x7f23bc0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 -- 172.21.15.31:0/1451769007 shutdown_connections 2024-09-16T09:02:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 -- 172.21.15.31:0/1451769007 wait complete. 2024-09-16T09:02:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 Processor -- start 2024-09-16T09:02:09.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.092+0000 7f23c2940640 1 -- start start 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23c2940640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc1029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23c2940640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0ffca0 0x7f23bc101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23c2940640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc1015c0 con 0x7f23bc0feaa0 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23c2940640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc101730 con 0x7f23bc0ffca0 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc1029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc1029e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43208/0 (socket says 172.21.15.31:43208) 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bbfff640 1 -- 172.21.15.31:0/43341563 learned_addr learned my addr 172.21.15.31:0/43341563 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bb7fe640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0ffca0 0x7f23bc101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bbfff640 1 -- 172.21.15.31:0/43341563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0ffca0 msgr2=0x7f23bc101080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bbfff640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0ffca0 0x7f23bc101080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bbfff640 1 -- 172.21.15.31:0/43341563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac009560 con 0x7f23bc0feaa0 2024-09-16T09:02:09.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.093+0000 7f23bbfff640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc1029e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02a9c0 tx=0x7f23ac0054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.094+0000 7f23b97fa640 1 -- 172.21.15.31:0/43341563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ac004aa0 con 0x7f23bc0feaa0 2024-09-16T09:02:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.094+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc1019b0 con 0x7f23bc0feaa0 2024-09-16T09:02:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.094+0000 7f23b97fa640 1 -- 172.21.15.31:0/43341563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23ac004c00 con 0x7f23bc0feaa0 2024-09-16T09:02:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.094+0000 7f23b97fa640 1 -- 172.21.15.31:0/43341563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ac003a80 con 0x7f23bc0feaa0 2024-09-16T09:02:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.094+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc19d740 con 0x7f23bc0feaa0 2024-09-16T09:02:09.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.095+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc0feea0 con 0x7f23bc0feaa0 2024-09-16T09:02:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.096+0000 7f23b97fa640 1 -- 172.21.15.31:0/43341563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f23ac003d40 con 0x7f23bc0feaa0 2024-09-16T09:02:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.096+0000 7f23b97fa640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f23a407af30 0x7f23a407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.096+0000 7f23b97fa640 1 -- 172.21.15.31:0/43341563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f23ac0bf1a0 con 0x7f23bc0feaa0 2024-09-16T09:02:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.096+0000 7f23bb7fe640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f23a407af30 0x7f23a407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.099 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.097+0000 7f23bb7fe640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f23a407af30 0x7f23a407d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005fd0 tx=0x7f23b0005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:09.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.100+0000 7f23b97fa640 1 -- 172.21.15.31:0/43341563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f23ac082350 con 0x7f23bc0feaa0 2024-09-16T09:02:09.361 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.359+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f23bc19d9d0 con 0x7f23bc0feaa0 2024-09-16T09:02:09.361 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:09 smithi031 ceph-mon[91884]: pgmap v100: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:09.361 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:09.361 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.359+0000 7f23b97fa640 1 -- 172.21.15.31:0/43341563 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f23ac085470 con 0x7f23bc0feaa0 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:02:09.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 3, 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 3 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 7, 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 7 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:02:09.363 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.361+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f23a407af30 msgr2=0x7f23a407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.361+0000 7f23c2940640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f23a407af30 0x7f23a407d3f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f23b0005fd0 tx=0x7f23b0005850 comp rx=0 tx=0).stop 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.361+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 msgr2=0x7f23bc1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.361+0000 7f23c2940640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc1029e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f23ac02a9c0 tx=0x7f23ac0054f0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.362+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 shutdown_connections 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.362+0000 7f23c2940640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f23a407af30 0x7f23a407d3f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.362+0000 7f23c2940640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f23bc0ffca0 0x7f23bc101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.362+0000 7f23c2940640 1 --2- 172.21.15.31:0/43341563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23bc0feaa0 0x7f23bc1029e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.362+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 >> 172.21.15.31:0/43341563 conn(0x7f23bc0fa230 msgr2=0x7f23bc068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:09.364 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.362+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 shutdown_connections 2024-09-16T09:02:09.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.362+0000 7f23c2940640 1 -- 172.21.15.31:0/43341563 wait complete. 2024-09-16T09:02:09.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 -- 172.21.15.31:0/3074605913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578102fc0 msgr2=0x7f55781053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 --2- 172.21.15.31:0/3074605913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578102fc0 0x7f55781053b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f556c009880 tx=0x7f556c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 -- 172.21.15.31:0/3074605913 shutdown_connections 2024-09-16T09:02:09.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 --2- 172.21.15.31:0/3074605913 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578102fc0 0x7f55781053b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 --2- 172.21.15.31:0/3074605913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578100690 0x7f5578102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 -- 172.21.15.31:0/3074605913 >> 172.21.15.31:0/3074605913 conn(0x7f55780fa250 msgr2=0x7f55780fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:09.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 -- 172.21.15.31:0/3074605913 shutdown_connections 2024-09-16T09:02:09.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.445+0000 7f557e4af640 1 -- 172.21.15.31:0/3074605913 wait complete. 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f557e4af640 1 Processor -- start 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f557e4af640 1 -- start start 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f557e4af640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578100690 0x7f5578195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f557e4af640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578102fc0 0x7f5578195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f557e4af640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5578196140 con 0x7f5578100690 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f557e4af640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55781962b0 con 0x7f5578102fc0 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f5577fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578100690 0x7f5578195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f5577fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578100690 0x7f5578195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43222/0 (socket says 172.21.15.31:43222) 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.446+0000 7f5577fff640 1 -- 172.21.15.31:0/2584412295 learned_addr learned my addr 172.21.15.31:0/2584412295 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f55777fe640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578102fc0 0x7f5578195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f5577fff640 1 -- 172.21.15.31:0/2584412295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578102fc0 msgr2=0x7f5578195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.450 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f5577fff640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578102fc0 0x7f5578195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f5577fff640 1 -- 172.21.15.31:0/2584412295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c009560 con 0x7f5578100690 2024-09-16T09:02:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f5577fff640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578100690 0x7f5578195630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f556800d720 tx=0x7f556800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f55757fa640 1 -- 172.21.15.31:0/2584412295 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55680051f0 con 0x7f5578100690 2024-09-16T09:02:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f55757fa640 1 -- 172.21.15.31:0/2584412295 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5568009ce0 con 0x7f5578100690 2024-09-16T09:02:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557819ad00 con 0x7f5578100690 2024-09-16T09:02:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f55757fa640 1 -- 172.21.15.31:0/2584412295 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5568010460 con 0x7f5578100690 2024-09-16T09:02:09.451 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.447+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557819b240 con 0x7f5578100690 2024-09-16T09:02:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.448+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5578108740 con 0x7f5578100690 2024-09-16T09:02:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.449+0000 7f55757fa640 1 -- 172.21.15.31:0/2584412295 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f5568005350 con 0x7f5578100690 2024-09-16T09:02:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.449+0000 7f55757fa640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f556007af30 0x7f556007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.450+0000 7f55757fa640 1 -- 172.21.15.31:0/2584412295 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f556809da10 con 0x7f5578100690 2024-09-16T09:02:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.450+0000 7f55777fe640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f556007af30 0x7f556007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.450+0000 7f55777fe640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f556007af30 0x7f556007d3f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f556c009230 tx=0x7f556c0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:09.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.454+0000 7f55757fa640 1 -- 172.21.15.31:0/2584412295 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f5568060c50 con 0x7f5578100690 2024-09-16T09:02:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:09 smithi097 ceph-mon[79892]: pgmap v100: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.675+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f557819b770 con 0x7f5578100690 2024-09-16T09:02:09.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.676+0000 7f55757fa640 1 -- 172.21.15.31:0/2584412295 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7f5568063d70 con 0x7f5578100690 2024-09-16T09:02:09.678 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:02:09.678 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:02:09.678 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:02:09.679 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:02:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:09.681 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:02:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f556007af30 msgr2=0x7f556007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f556007af30 0x7f556007d3f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f556c009230 tx=0x7f556c0314e0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578100690 msgr2=0x7f5578195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.682 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578100690 0x7f5578195630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f556800d720 tx=0x7f556800dbe0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 shutdown_connections 2024-09-16T09:02:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f556007af30 0x7f556007d3f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5578102fc0 0x7f5578195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 --2- 172.21.15.31:0/2584412295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5578100690 0x7f5578195630 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 >> 172.21.15.31:0/2584412295 conn(0x7f55780fa250 msgr2=0x7f55780fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 shutdown_connections 2024-09-16T09:02:09.683 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.678+0000 7f557e4af640 1 -- 172.21.15.31:0/2584412295 wait complete. 2024-09-16T09:02:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.762+0000 7f1ea7479640 1 -- 172.21.15.31:0/4019492334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0100680 msgr2=0x7f1ea0102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.762+0000 7f1ea7479640 1 --2- 172.21.15.31:0/4019492334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0100680 0x7f1ea0102a70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0098b0 tx=0x7f1e9c02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.762+0000 7f1ea7479640 1 -- 172.21.15.31:0/4019492334 shutdown_connections 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.762+0000 7f1ea7479640 1 --2- 172.21.15.31:0/4019492334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea0102fb0 0x7f1ea01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.762+0000 7f1ea7479640 1 --2- 172.21.15.31:0/4019492334 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0100680 0x7f1ea0102a70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.762+0000 7f1ea7479640 1 -- 172.21.15.31:0/4019492334 >> 172.21.15.31:0/4019492334 conn(0x7f1ea00fa280 msgr2=0x7f1ea00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 -- 172.21.15.31:0/4019492334 shutdown_connections 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 -- 172.21.15.31:0/4019492334 wait complete. 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 Processor -- start 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 -- start start 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea0100680 0x7f1ea00fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0102fb0 0x7f1ea00fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00feb70 con 0x7f1ea0102fb0 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea7479640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00fece0 con 0x7f1ea0100680 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea4a14640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0102fb0 0x7f1ea00fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.763+0000 7f1ea4a14640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0102fb0 0x7f1ea00fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43240/0 (socket says 172.21.15.31:43240) 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea4a14640 1 -- 172.21.15.31:0/150570436 learned_addr learned my addr 172.21.15.31:0/150570436 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea5215640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea0100680 0x7f1ea00fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea4a14640 1 -- 172.21.15.31:0/150570436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea0100680 msgr2=0x7f1ea00fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea4a14640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea0100680 0x7f1ea00fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea4a14640 1 -- 172.21.15.31:0/150570436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c009560 con 0x7f1ea0102fb0 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea4a14640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0102fb0 0x7f1ea00fe630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000d6c0 tx=0x7f1e9000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1e967fc640 1 -- 172.21.15.31:0/150570436 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e900051f0 con 0x7f1ea0102fb0 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea00fefc0 con 0x7f1ea0102fb0 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1e967fc640 1 -- 172.21.15.31:0/150570436 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e90009ce0 con 0x7f1ea0102fb0 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1e967fc640 1 -- 172.21.15.31:0/150570436 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e90010460 con 0x7f1ea0102fb0 2024-09-16T09:02:09.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.764+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea019d770 con 0x7f1ea0102fb0 2024-09-16T09:02:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.765+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea0100a60 con 0x7f1ea0102fb0 2024-09-16T09:02:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.766+0000 7f1e967fc640 1 -- 172.21.15.31:0/150570436 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f1e90005350 con 0x7f1ea0102fb0 2024-09-16T09:02:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.766+0000 7f1e967fc640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1e8c07af30 0x7f1e8c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.767+0000 7f1e967fc640 1 -- 172.21.15.31:0/150570436 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f1e9009dab0 con 0x7f1ea0102fb0 2024-09-16T09:02:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.767+0000 7f1ea5215640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1e8c07af30 0x7f1e8c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:09.769 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.767+0000 7f1ea5215640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1e8c07af30 0x7f1e8c07d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c031540 tx=0x7f1e9c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:09.773 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.771+0000 7f1e967fc640 1 -- 172.21.15.31:0/150570436 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f1e9009fe60 con 0x7f1ea0102fb0 2024-09-16T09:02:09.978 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.976+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ea01039d0 con 0x7f1e8c07af30 2024-09-16T09:02:09.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.977+0000 7f1e967fc640 1 -- 172.21.15.31:0/150570436 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1ea01039d0 con 0x7f1e8c07af30 2024-09-16T09:02:09.979 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:02:09.979 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:02:09.979 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:02:09.979 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:02:09.979 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "9/23 daemons upgraded", 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:02:09.980 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:02:09.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1e8c07af30 msgr2=0x7f1e8c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1e8c07af30 0x7f1e8c07d3f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c031540 tx=0x7f1e9c000bc0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0102fb0 msgr2=0x7f1ea00fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0102fb0 0x7f1ea00fe630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000d6c0 tx=0x7f1e9000db80 comp rx=0 tx=0).stop 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 shutdown_connections 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1e8c07af30 0x7f1e8c07d3f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ea0102fb0 0x7f1ea00fe630 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 --2- 172.21.15.31:0/150570436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ea0100680 0x7f1ea00fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 >> 172.21.15.31:0/150570436 conn(0x7f1ea00fa280 msgr2=0x7f1ea00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 shutdown_connections 2024-09-16T09:02:09.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:09.979+0000 7f1ea7479640 1 -- 172.21.15.31:0/150570436 wait complete. 2024-09-16T09:02:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 -- 172.21.15.31:0/4080021870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c100d90 msgr2=0x7f379c101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 --2- 172.21.15.31:0/4080021870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c100d90 0x7f379c101210 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f378c009880 tx=0x7f378c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 -- 172.21.15.31:0/4080021870 shutdown_connections 2024-09-16T09:02:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 --2- 172.21.15.31:0/4080021870 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c100d90 0x7f379c101210 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 --2- 172.21.15.31:0/4080021870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379c0ffb90 0x7f379c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 -- 172.21.15.31:0/4080021870 >> 172.21.15.31:0/4080021870 conn(0x7f379c0fb320 msgr2=0x7f379c0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:10.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 -- 172.21.15.31:0/4080021870 shutdown_connections 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.066+0000 7f37a39b0640 1 -- 172.21.15.31:0/4080021870 wait complete. 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a39b0640 1 Processor -- start 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a39b0640 1 -- start start 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a39b0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0ffb90 0x7f379c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a39b0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379c100d90 0x7f379c0728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a39b0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c072de0 con 0x7f379c0ffb90 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a39b0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379c072f50 con 0x7f379c100d90 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a174c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0ffb90 0x7f379c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a174c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0ffb90 0x7f379c074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43252/0 (socket says 172.21.15.31:43252) 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a174c640 1 -- 172.21.15.31:0/1286964182 learned_addr learned my addr 172.21.15.31:0/1286964182 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a0f4b640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379c100d90 0x7f379c0728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a174c640 1 -- 172.21.15.31:0/1286964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379c100d90 msgr2=0x7f379c0728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:10.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a174c640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379c100d90 0x7f379c0728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.067+0000 7f37a174c640 1 -- 172.21.15.31:0/1286964182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c009560 con 0x7f379c0ffb90 2024-09-16T09:02:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.068+0000 7f37a174c640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0ffb90 0x7f379c074250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f379800eaf0 tx=0x7f379800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.068+0000 7f37927fc640 1 -- 172.21.15.31:0/1286964182 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3798003970 con 0x7f379c0ffb90 2024-09-16T09:02:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.068+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f379c073230 con 0x7f379c0ffb90 2024-09-16T09:02:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.068+0000 7f37927fc640 1 -- 172.21.15.31:0/1286964182 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3798004020 con 0x7f379c0ffb90 2024-09-16T09:02:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.068+0000 7f37927fc640 1 -- 172.21.15.31:0/1286964182 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3798010660 con 0x7f379c0ffb90 2024-09-16T09:02:10.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.068+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379c19d690 con 0x7f379c0ffb90 2024-09-16T09:02:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.069+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379c0fff90 con 0x7f379c0ffb90 2024-09-16T09:02:10.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.070+0000 7f37927fc640 1 -- 172.21.15.31:0/1286964182 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f3798004190 con 0x7f379c0ffb90 2024-09-16T09:02:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.070+0000 7f37927fc640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f378807af30 0x7f378807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.070+0000 7f37927fc640 1 -- 172.21.15.31:0/1286964182 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3798014070 con 0x7f379c0ffb90 2024-09-16T09:02:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.070+0000 7f37a0f4b640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f378807af30 0x7f378807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:10.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.071+0000 7f37a0f4b640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f378807af30 0x7f378807d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f378c031540 tx=0x7f378c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:10.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.075+0000 7f37927fc640 1 -- 172.21.15.31:0/1286964182 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f3798061a70 con 0x7f379c0ffb90 2024-09-16T09:02:10.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.333+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f379c02c3e0 con 0x7f379c0ffb90 2024-09-16T09:02:10.335 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:10 smithi031 ceph-mon[91884]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:10.336 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:10 smithi031 ceph-mon[91884]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:10.336 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:10 smithi031 ceph-mon[91884]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:10.336 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:10 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/43341563' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:10.336 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:10 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2584412295' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:02:10.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.333+0000 7f37927fc640 1 -- 172.21.15.31:0/1286964182 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f3798004440 con 0x7f379c0ffb90 2024-09-16T09:02:10.336 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.335+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f378807af30 msgr2=0x7f378807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.335+0000 7f37a39b0640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f378807af30 0x7f378807d3f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f378c031540 tx=0x7f378c000bc0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0ffb90 msgr2=0x7f379c074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0ffb90 0x7f379c074250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f379800eaf0 tx=0x7f379800c2d0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 shutdown_connections 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f378807af30 0x7f378807d3f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f379c100d90 0x7f379c0728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 --2- 172.21.15.31:0/1286964182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379c0ffb90 0x7f379c074250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 >> 172.21.15.31:0/1286964182 conn(0x7f379c0fb320 msgr2=0x7f379c104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 shutdown_connections 2024-09-16T09:02:10.338 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:10.336+0000 7f37a39b0640 1 -- 172.21.15.31:0/1286964182 wait complete. 2024-09-16T09:02:10.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:10 smithi097 ceph-mon[79892]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:10.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:10 smithi097 ceph-mon[79892]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:10.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:10 smithi097 ceph-mon[79892]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:10.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:10 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/43341563' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:10.414 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:10 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2584412295' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:02:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:11 smithi031 ceph-mon[91884]: pgmap v101: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:11 smithi031 ceph-mon[91884]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:11 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1286964182' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:02:11.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:11 smithi097 ceph-mon[79892]: pgmap v101: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:11.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:11 smithi097 ceph-mon[79892]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:11.425 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:11 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1286964182' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:02:13.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:13 smithi031 ceph-mon[91884]: pgmap v102: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:13.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:13 smithi097 ceph-mon[79892]: pgmap v102: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:14 smithi031 ceph-mon[91884]: pgmap v103: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:14 smithi097 ceph-mon[79892]: pgmap v103: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:16 smithi031 ceph-mon[91884]: pgmap v104: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:17.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:16 smithi097 ceph-mon[79892]: pgmap v104: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:17 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-16T09:02:18.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:17 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16} : dispatch 2024-09-16T09:02:19.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:18 smithi031 ceph-mon[91884]: pgmap v105: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:19.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:18 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T09:02:19.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:18 smithi031 ceph-mon[91884]: Upgrade: osd.3 is safe to restart 2024-09-16T09:02:19.169 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:18 smithi097 ceph-mon[79892]: pgmap v105: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:19.170 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:18 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-16T09:02:19.170 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:18 smithi097 ceph-mon[79892]: Upgrade: osd.3 is safe to restart 2024-09-16T09:02:20.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:19 smithi031 ceph-mon[91884]: Upgrade: Updating osd.3 2024-09-16T09:02:20.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:20.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T09:02:20.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:20.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:19 smithi031 ceph-mon[91884]: Deploying daemon osd.3 on smithi097 2024-09-16T09:02:20.219 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:19 smithi097 ceph-mon[79892]: Upgrade: Updating osd.3 2024-09-16T09:02:20.219 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:20.219 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T09:02:20.219 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:20.219 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:19 smithi097 ceph-mon[79892]: Deploying daemon osd.3 on smithi097 2024-09-16T09:02:21.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:20 smithi031 ceph-mon[91884]: pgmap v106: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:21.143 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:20 smithi097 ceph-mon[79892]: pgmap v106: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:22.117 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:21 smithi097 systemd[1]: Stopping Ceph osd.3 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:02:22.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:22 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[41513]: 2024-09-16T09:02:22.115+0000 7f6e9109d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:02:22.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:22 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[41513]: 2024-09-16T09:02:22.115+0000 7f6e9109d640 -1 osd.3 65 *** Got signal Terminated *** 2024-09-16T09:02:22.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:22 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[41513]: 2024-09-16T09:02:22.115+0000 7f6e9109d640 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:02:23.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:22 smithi031 ceph-mon[91884]: pgmap v107: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:23.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:22 smithi031 ceph-mon[91884]: osd.3 marked itself down and dead 2024-09-16T09:02:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:22 smithi097 ceph-mon[79892]: pgmap v107: 65 pgs: 65 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:22 smithi097 ceph-mon[79892]: osd.3 marked itself down and dead 2024-09-16T09:02:23.642 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:23 smithi097 podman[88286]: 2024-09-16 09:02:23.383532236 +0000 UTC m=+1.418018091 container died eeedb0a2c952db735141393c43ce0c73a9ff14fa76a4c58529db47a1a435c170 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=reef-ae7f373, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T09:02:24.012 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:23 smithi097 podman[88286]: 2024-09-16 09:02:23.740209392 +0000 UTC m=+1.774695231 container cleanup eeedb0a2c952db735141393c43ce0c73a9ff14fa76a4c58529db47a1a435c170 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T09:02:24.012 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:23 smithi097 bash[88286]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3 2024-09-16T09:02:24.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:02:24.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: osdmap e66: 6 total, 5 up, 6 in 2024-09-16T09:02:24.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 3]} : dispatch 2024-09-16T09:02:24.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 3]} : dispatch 2024-09-16T09:02:24.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 3]} : dispatch 2024-09-16T09:02:24.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [1, 0, 4, 3]} : dispatch 2024-09-16T09:02:24.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]} : dispatch 2024-09-16T09:02:24.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [1, 5]} : dispatch 2024-09-16T09:02:24.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:24.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:24.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:02:24.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: osdmap e66: 6 total, 5 up, 6 in 2024-09-16T09:02:24.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 3]} : dispatch 2024-09-16T09:02:24.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 3]} : dispatch 2024-09-16T09:02:24.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 3]} : dispatch 2024-09-16T09:02:24.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [1, 0, 4, 3]} : dispatch 2024-09-16T09:02:24.129 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]} : dispatch 2024-09-16T09:02:24.130 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [1, 5]} : dispatch 2024-09-16T09:02:24.130 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:24.130 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:24.375 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:24 smithi097 podman[88301]: 2024-09-16 09:02:24.038509437 +0000 UTC m=+0.648461615 container remove eeedb0a2c952db735141393c43ce0c73a9ff14fa76a4c58529db47a1a435c170 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T09:02:24.654 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:24 smithi097 podman[88391]: 2024-09-16 09:02:24.278172901 +0000 UTC m=+0.016115735 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:02:24.654 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:24 smithi097 podman[88391]: 2024-09-16 09:02:24.375955815 +0000 UTC m=+0.113898643 container create e329b221f330cca2e8f9fe9e4f2aa4c5edaadc85af4e1f6c39fae1ad5ac3436f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T09:02:24.654 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:24 smithi097 podman[88391]: 2024-09-16 09:02:24.65355308 +0000 UTC m=+0.391495905 container init e329b221f330cca2e8f9fe9e4f2aa4c5edaadc85af4e1f6c39fae1ad5ac3436f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True) 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: pgmap v109: 65 pgs: 16 stale+active+clean, 49 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 3]}]': finished 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 3]}]': finished 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 3]}]': finished 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [1, 0, 4, 3]}]': finished 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]}]': finished 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [1, 5]}]': finished 2024-09-16T09:02:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:24 smithi097 ceph-mon[79892]: osdmap e67: 6 total, 5 up, 6 in 2024-09-16T09:02:25.013 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:24 smithi097 podman[88391]: 2024-09-16 09:02:24.657159452 +0000 UTC m=+0.395102281 container start e329b221f330cca2e8f9fe9e4f2aa4c5edaadc85af4e1f6c39fae1ad5ac3436f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T09:02:25.013 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:24 smithi097 podman[88391]: 2024-09-16 09:02:24.737399242 +0000 UTC m=+0.475342076 container attach e329b221f330cca2e8f9fe9e4f2aa4c5edaadc85af4e1f6c39fae1ad5ac3436f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T09:02:25.013 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:24 smithi097 podman[88391]: 2024-09-16 09:02:24.842518769 +0000 UTC m=+0.580461597 container died e329b221f330cca2e8f9fe9e4f2aa4c5edaadc85af4e1f6c39fae1ad5ac3436f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:02:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: pgmap v109: 65 pgs: 16 stale+active+clean, 49 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.2", "id": [0, 3]}]': finished 2024-09-16T09:02:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [0, 3]}]': finished 2024-09-16T09:02:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [0, 3]}]': finished 2024-09-16T09:02:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [1, 0, 4, 3]}]': finished 2024-09-16T09:02:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 3]}]': finished 2024-09-16T09:02:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [1, 5]}]': finished 2024-09-16T09:02:25.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:24 smithi031 ceph-mon[91884]: osdmap e67: 6 total, 5 up, 6 in 2024-09-16T09:02:25.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:25 smithi097 podman[88414]: 2024-09-16 09:02:25.408718894 +0000 UTC m=+0.557980856 container remove e329b221f330cca2e8f9fe9e4f2aa4c5edaadc85af4e1f6c39fae1ad5ac3436f (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T09:02:25.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:25 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-16T09:02:25.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:25 smithi097 systemd[1]: Stopped Ceph osd.3 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:02:25.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:25 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.3.service: Consumed 1min 11.041s CPU time. 2024-09-16T09:02:25.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:25 smithi097 systemd[1]: Starting Ceph osd.3 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:02:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:25 smithi031 ceph-mon[91884]: osdmap e68: 6 total, 5 up, 6 in 2024-09-16T09:02:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:25 smithi031 ceph-mon[91884]: osdmap e69: 6 total, 5 up, 6 in 2024-09-16T09:02:26.148 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:25 smithi097 ceph-mon[79892]: osdmap e68: 6 total, 5 up, 6 in 2024-09-16T09:02:26.149 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:25 smithi097 ceph-mon[79892]: osdmap e69: 6 total, 5 up, 6 in 2024-09-16T09:02:26.149 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:25 smithi097 podman[88516]: 2024-09-16 09:02:25.74800458 +0000 UTC m=+0.015932612 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:02:26.149 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:25 smithi097 podman[88516]: 2024-09-16 09:02:25.852686489 +0000 UTC m=+0.120614506 container create aa1ab175a5989a5c100f5402b63d69be785f2e487386dd8c85fa4ad052bb9f85 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T09:02:26.149 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:26 smithi097 podman[88516]: 2024-09-16 09:02:26.148533784 +0000 UTC m=+0.416461829 container init aa1ab175a5989a5c100f5402b63d69be785f2e487386dd8c85fa4ad052bb9f85 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T09:02:26.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:26 smithi097 podman[88516]: 2024-09-16 09:02:26.152376707 +0000 UTC m=+0.420304731 container start aa1ab175a5989a5c100f5402b63d69be785f2e487386dd8c85fa4ad052bb9f85 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:02:26.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:26 smithi097 podman[88516]: 2024-09-16 09:02:26.256816683 +0000 UTC m=+0.524744724 container attach aa1ab175a5989a5c100f5402b63d69be785f2e487386dd8c85fa4ad052bb9f85 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T09:02:26.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:26 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:02:26.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:26 smithi097 bash[88516]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:02:26.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:26 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:02:26.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:26 smithi097 bash[88516]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:02:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:26 smithi031 ceph-mon[91884]: pgmap v112: 65 pgs: 10 active+undersized, 11 peering, 3 stale+active+clean, 8 active+undersized+degraded, 33 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/282 objects degraded (10.284%) 2024-09-16T09:02:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:26 smithi031 ceph-mon[91884]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T09:02:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:26 smithi031 ceph-mon[91884]: Health check failed: Degraded data redundancy: 29/282 objects degraded (10.284%), 8 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:27.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:26 smithi097 ceph-mon[79892]: pgmap v112: 65 pgs: 10 active+undersized, 11 peering, 3 stale+active+clean, 8 active+undersized+degraded, 33 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 29/282 objects degraded (10.284%) 2024-09-16T09:02:27.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:26 smithi097 ceph-mon[79892]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-09-16T09:02:27.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:26 smithi097 ceph-mon[79892]: Health check failed: Degraded data redundancy: 29/282 objects degraded (10.284%), 8 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:28.928 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-mon[79892]: pgmap v114: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 14 active+undersized, 11 peering, 16 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 49/282 objects degraded (17.376%); 0 B/s, 1 objects/s recovering 2024-09-16T09:02:28.929 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-mon[79892]: osdmap e70: 6 total, 5 up, 6 in 2024-09-16T09:02:28.929 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T09:02:28.929 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 bash[88516]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T09:02:28.929 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T09:02:28.929 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 bash[88516]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-16T09:02:28.929 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T09:02:28.929 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 bash[88516]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-16T09:02:29.235 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 bash[88516]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 bash[88516]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 bash[88516]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate[88531]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 bash[88516]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-16T09:02:29.236 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:28 smithi097 podman[88516]: 2024-09-16 09:02:28.956231941 +0000 UTC m=+3.224159962 container died aa1ab175a5989a5c100f5402b63d69be785f2e487386dd8c85fa4ad052bb9f85 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux ) 2024-09-16T09:02:29.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:28 smithi031 ceph-mon[91884]: pgmap v114: 65 pgs: 1 active+recovering+undersized+remapped, 1 remapped+peering, 14 active+undersized, 11 peering, 16 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 49/282 objects degraded (17.376%); 0 B/s, 1 objects/s recovering 2024-09-16T09:02:29.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:28 smithi031 ceph-mon[91884]: osdmap e70: 6 total, 5 up, 6 in 2024-09-16T09:02:29.884 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:29 smithi097 podman[88705]: 2024-09-16 09:02:29.527135287 +0000 UTC m=+0.565110325 container remove aa1ab175a5989a5c100f5402b63d69be785f2e487386dd8c85fa4ad052bb9f85 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-activate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0) 2024-09-16T09:02:29.884 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:29 smithi097 podman[88766]: 2024-09-16 09:02:29.637111572 +0000 UTC m=+0.016168131 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:02:29.884 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:29 smithi097 podman[88766]: 2024-09-16 09:02:29.773417272 +0000 UTC m=+0.152473822 container create 9d23936a9f41b9ac67e67f502af668cc92406d70b29dcab364a861ea7eebd3b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T09:02:30.141 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:29 smithi097 ceph-mon[79892]: osdmap e71: 6 total, 5 up, 6 in 2024-09-16T09:02:30.141 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:30 smithi097 podman[88766]: 2024-09-16 09:02:30.038884896 +0000 UTC m=+0.417941448 container init 9d23936a9f41b9ac67e67f502af668cc92406d70b29dcab364a861ea7eebd3b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T09:02:30.141 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:30 smithi097 podman[88766]: 2024-09-16 09:02:30.042188471 +0000 UTC m=+0.421245029 container start 9d23936a9f41b9ac67e67f502af668cc92406d70b29dcab364a861ea7eebd3b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T09:02:30.141 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:30 smithi097 ceph-osd[88784]: -- 172.21.15.97:0/1955498389 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55760d9701e0 con 0x55760cb65c00 2024-09-16T09:02:30.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:29 smithi031 ceph-mon[91884]: osdmap e71: 6 total, 5 up, 6 in 2024-09-16T09:02:30.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:30 smithi097 bash[88766]: 9d23936a9f41b9ac67e67f502af668cc92406d70b29dcab364a861ea7eebd3b2 2024-09-16T09:02:30.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:30 smithi097 systemd[1]: Started Ceph osd.3 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:02:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:30 smithi097 ceph-mon[79892]: pgmap v117: 65 pgs: 3 active+clean+remapped, 1 active+recovering+undersized+remapped, 2 remapped+peering, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 3/282 objects misplaced (1.064%); 0 B/s, 164 objects/s recovering 2024-09-16T09:02:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:30 smithi097 ceph-mon[79892]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T09:02:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:30 smithi097 ceph-mon[79892]: osdmap e72: 6 total, 5 up, 6 in 2024-09-16T09:02:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:30 smithi031 ceph-mon[91884]: pgmap v117: 65 pgs: 3 active+clean+remapped, 1 active+recovering+undersized+remapped, 2 remapped+peering, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 3/282 objects misplaced (1.064%); 0 B/s, 164 objects/s recovering 2024-09-16T09:02:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:30 smithi031 ceph-mon[91884]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-09-16T09:02:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:30 smithi031 ceph-mon[91884]: osdmap e72: 6 total, 5 up, 6 in 2024-09-16T09:02:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:31.681 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:31 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:02:31.435+0000 7f22cc6e7640 -1 Falling back to public interface 2024-09-16T09:02:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:31 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 59/282 objects degraded (20.922%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:32.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:31 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 59/282 objects degraded (20.922%), 19 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:33.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:32 smithi097 ceph-mon[79892]: pgmap v119: 65 pgs: 5 active+clean+remapped, 1 remapped+peering, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 263 objects/s recovering 2024-09-16T09:02:33.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:32 smithi031 ceph-mon[91884]: pgmap v119: 65 pgs: 5 active+clean+remapped, 1 remapped+peering, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 263 objects/s recovering 2024-09-16T09:02:34.112 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:33 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:34.112 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:33 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:34.112 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:33 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:34.112 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:33 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:33 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:33 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:33 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:34.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:33 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:35.105 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:34 smithi097 ceph-mon[79892]: pgmap v120: 65 pgs: 5 active+clean+remapped, 1 remapped+peering, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 237 objects/s recovering 2024-09-16T09:02:35.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:34 smithi031 ceph-mon[91884]: pgmap v120: 65 pgs: 5 active+clean+remapped, 1 remapped+peering, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 237 objects/s recovering 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: pgmap v121: 65 pgs: 6 active+clean+remapped, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 192 objects/s recovering 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:37.069 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: pgmap v121: 65 pgs: 6 active+clean+remapped, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 192 objects/s recovering 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:02:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:37.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:37.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:37.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-16T09:02:37.761 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:37 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:02:37.393+0000 7f22cc6e7640 -1 osd.3 65 log_to_monitors true 2024-09-16T09:02:38.262 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:02:37 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:02:37.938+0000 7f22c4491640 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:02:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:37 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T09:02:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:37 smithi097 ceph-mon[79892]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-16T09:02:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:37 smithi097 ceph-mon[79892]: from='osd.3 [v2:172.21.15.97:6800/3637276624,v1:172.21.15.97:6801/3637276624]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T09:02:38.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:37 smithi097 ceph-mon[79892]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T09:02:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:37 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T09:02:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:37 smithi031 ceph-mon[91884]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-16T09:02:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:37 smithi031 ceph-mon[91884]: from='osd.3 [v2:172.21.15.97:6800/3637276624,v1:172.21.15.97:6801/3637276624]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T09:02:38.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:37 smithi031 ceph-mon[91884]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T09:02:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:38 smithi097 ceph-mon[79892]: pgmap v122: 65 pgs: 6 active+clean+remapped, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 169 objects/s recovering 2024-09-16T09:02:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:38 smithi097 ceph-mon[79892]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T09:02:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:38 smithi097 ceph-mon[79892]: from='osd.3 [v2:172.21.15.97:6800/3637276624,v1:172.21.15.97:6801/3637276624]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:02:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:38 smithi097 ceph-mon[79892]: osdmap e73: 6 total, 5 up, 6 in 2024-09-16T09:02:39.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:38 smithi097 ceph-mon[79892]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:02:39.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:39.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:38 smithi031 ceph-mon[91884]: pgmap v122: 65 pgs: 6 active+clean+remapped, 18 active+undersized, 19 active+undersized+degraded, 22 active+clean; 201 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 59/282 objects degraded (20.922%); 6/282 objects misplaced (2.128%); 0 B/s, 169 objects/s recovering 2024-09-16T09:02:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:38 smithi031 ceph-mon[91884]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T09:02:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:38 smithi031 ceph-mon[91884]: from='osd.3 [v2:172.21.15.97:6800/3637276624,v1:172.21.15.97:6801/3637276624]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:02:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:38 smithi031 ceph-mon[91884]: osdmap e73: 6 total, 5 up, 6 in 2024-09-16T09:02:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:38 smithi031 ceph-mon[91884]: from='osd.3 ' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:02:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:39.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:40.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:39 smithi097 ceph-mon[79892]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:02:40.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:39 smithi097 ceph-mon[79892]: osd.3 [v2:172.21.15.97:6800/3637276624,v1:172.21.15.97:6801/3637276624] boot 2024-09-16T09:02:40.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:39 smithi097 ceph-mon[79892]: osdmap e74: 6 total, 6 up, 6 in 2024-09-16T09:02:40.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:02:40.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:39 smithi097 ceph-mon[79892]: osdmap e75: 6 total, 6 up, 6 in 2024-09-16T09:02:40.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:39 smithi031 ceph-mon[91884]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:02:40.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:39 smithi031 ceph-mon[91884]: osd.3 [v2:172.21.15.97:6800/3637276624,v1:172.21.15.97:6801/3637276624] boot 2024-09-16T09:02:40.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:39 smithi031 ceph-mon[91884]: osdmap e74: 6 total, 6 up, 6 in 2024-09-16T09:02:40.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:02:40.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:39 smithi031 ceph-mon[91884]: osdmap e75: 6 total, 6 up, 6 in 2024-09-16T09:02:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 -- 172.21.15.31:0/19953798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 msgr2=0x7f9774102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 --2- 172.21.15.31:0/19953798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774102a20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f97640098b0 tx=0x7f976402a4d0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 -- 172.21.15.31:0/19953798 shutdown_connections 2024-09-16T09:02:40.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 --2- 172.21.15.31:0/19953798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9774102f60 0x7f9774105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 --2- 172.21.15.31:0/19953798 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774102a20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 -- 172.21.15.31:0/19953798 >> 172.21.15.31:0/19953798 conn(0x7f97740fa250 msgr2=0x7f97740fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 -- 172.21.15.31:0/19953798 shutdown_connections 2024-09-16T09:02:40.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.435+0000 7f9779cd6640 1 -- 172.21.15.31:0/19953798 wait complete. 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f9779cd6640 1 Processor -- start 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f9779cd6640 1 -- start start 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f9779cd6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f9779cd6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9774102f60 0x7f9774195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f9779cd6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9774196150 con 0x7f9774100630 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f9779cd6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97741962c0 con 0x7f9774102f60 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f97737fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:40.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f97737fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774195640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45502/0 (socket says 172.21.15.31:45502) 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.436+0000 7f97737fe640 1 -- 172.21.15.31:0/3521308501 learned_addr learned my addr 172.21.15.31:0/3521308501 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f976affd640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9774102f60 0x7f9774195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f97737fe640 1 -- 172.21.15.31:0/3521308501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9774102f60 msgr2=0x7f9774195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f97737fe640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9774102f60 0x7f9774195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f97737fe640 1 -- 172.21.15.31:0/3521308501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764009560 con 0x7f9774100630 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f976affd640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9774102f60 0x7f9774195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f97737fe640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774195640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f97640054f0 tx=0x7f9764005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f97717fa640 1 -- 172.21.15.31:0/3521308501 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9764004aa0 con 0x7f9774100630 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f97717fa640 1 -- 172.21.15.31:0/3521308501 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9764004c00 con 0x7f9774100630 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977419ad00 con 0x7f9774100630 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f97717fa640 1 -- 172.21.15.31:0/3521308501 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9764003b10 con 0x7f9774100630 2024-09-16T09:02:40.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.437+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977419b1a0 con 0x7f9774100630 2024-09-16T09:02:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.438+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97741087c0 con 0x7f9774100630 2024-09-16T09:02:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.439+0000 7f97717fa640 1 -- 172.21.15.31:0/3521308501 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f9764003d80 con 0x7f9774100630 2024-09-16T09:02:40.441 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.440+0000 7f97717fa640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f976007af30 0x7f976007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:40.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.440+0000 7f976affd640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f976007af30 0x7f976007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:40.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.440+0000 7f97717fa640 1 -- 172.21.15.31:0/3521308501 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6827+0+0 (secure 0 0 0) 0x7f97640bf230 con 0x7f9774100630 2024-09-16T09:02:40.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.440+0000 7f976affd640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f976007af30 0x7f976007d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f975c005d00 tx=0x7f975c005ed0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:40.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.444+0000 7f97717fa640 1 -- 172.21.15.31:0/3521308501 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f9764082320 con 0x7f9774100630 2024-09-16T09:02:40.650 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.649+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f977419b3e0 con 0x7f976007af30 2024-09-16T09:02:40.651 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.650+0000 7f97717fa640 1 -- 172.21.15.31:0/3521308501 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f977419b3e0 con 0x7f976007af30 2024-09-16T09:02:40.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f976007af30 msgr2=0x7f976007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f976007af30 0x7f976007d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f975c005d00 tx=0x7f975c005ed0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.653 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 msgr2=0x7f9774195640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774195640 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f97640054f0 tx=0x7f9764005520 comp rx=0 tx=0).stop 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 shutdown_connections 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f976007af30 0x7f976007d3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9774102f60 0x7f9774195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 --2- 172.21.15.31:0/3521308501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9774100630 0x7f9774195640 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 >> 172.21.15.31:0/3521308501 conn(0x7f97740fa250 msgr2=0x7f97740fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 shutdown_connections 2024-09-16T09:02:40.654 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.652+0000 7f9779cd6640 1 -- 172.21.15.31:0/3521308501 wait complete. 2024-09-16T09:02:40.662 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:02:40.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 -- 172.21.15.31:0/185062380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 msgr2=0x7f8e6c105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 --2- 172.21.15.31:0/185062380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c105350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8e600098b0 tx=0x7f8e6002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 -- 172.21.15.31:0/185062380 shutdown_connections 2024-09-16T09:02:40.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 --2- 172.21.15.31:0/185062380 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c105350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 --2- 172.21.15.31:0/185062380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c100630 0x7f8e6c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 -- 172.21.15.31:0/185062380 >> 172.21.15.31:0/185062380 conn(0x7f8e6c0fa250 msgr2=0x7f8e6c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:40.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 -- 172.21.15.31:0/185062380 shutdown_connections 2024-09-16T09:02:40.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.737+0000 7f8e71219640 1 -- 172.21.15.31:0/185062380 wait complete. 2024-09-16T09:02:40.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.738+0000 7f8e71219640 1 Processor -- start 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.738+0000 7f8e71219640 1 -- start start 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.738+0000 7f8e71219640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c100630 0x7f8e6c0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.738+0000 7f8e71219640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c0fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.738+0000 7f8e71219640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c1004d0 con 0x7f8e6c102f60 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.738+0000 7f8e71219640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0feb50 con 0x7f8e6c100630 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.738+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c0fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e6a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c0fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45512/0 (socket says 172.21.15.31:45512) 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e6a59c640 1 -- 172.21.15.31:0/3184245276 learned_addr learned my addr 172.21.15.31:0/3184245276 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e6ad9d640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c100630 0x7f8e6c0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e6a59c640 1 -- 172.21.15.31:0/3184245276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c100630 msgr2=0x7f8e6c0fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e6a59c640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c100630 0x7f8e6c0fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e6a59c640 1 -- 172.21.15.31:0/3184245276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e60009560 con 0x7f8e6c102f60 2024-09-16T09:02:40.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e6a59c640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c0fe5e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e6002a9f0 tx=0x7f8e60005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:40.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e53fff640 1 -- 172.21.15.31:0/3184245276 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e60004b20 con 0x7f8e6c102f60 2024-09-16T09:02:40.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e53fff640 1 -- 172.21.15.31:0/3184245276 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e60004c80 con 0x7f8e6c102f60 2024-09-16T09:02:40.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c0fedd0 con 0x7f8e6c102f60 2024-09-16T09:02:40.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.739+0000 7f8e53fff640 1 -- 172.21.15.31:0/3184245276 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e60003b50 con 0x7f8e6c102f60 2024-09-16T09:02:40.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.740+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c0ff2f0 con 0x7f8e6c102f60 2024-09-16T09:02:40.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.741+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c108790 con 0x7f8e6c102f60 2024-09-16T09:02:40.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.742+0000 7f8e53fff640 1 -- 172.21.15.31:0/3184245276 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f8e60031d00 con 0x7f8e6c102f60 2024-09-16T09:02:40.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.742+0000 7f8e53fff640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8e5407af30 0x7f8e5407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:40.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.742+0000 7f8e6ad9d640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8e5407af30 0x7f8e5407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:40.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.743+0000 7f8e53fff640 1 -- 172.21.15.31:0/3184245276 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6827+0+0 (secure 0 0 0) 0x7f8e600bf330 con 0x7f8e6c102f60 2024-09-16T09:02:40.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.743+0000 7f8e6ad9d640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8e5407af30 0x7f8e5407d3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00abe0 tx=0x7f8e5c00a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:40.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.747+0000 7f8e53fff640 1 -- 172.21.15.31:0/3184245276 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f8e600823a0 con 0x7f8e6c102f60 2024-09-16T09:02:40.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.959+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e6c1039c0 con 0x7f8e5407af30 2024-09-16T09:02:40.961 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.960+0000 7f8e53fff640 1 -- 172.21.15.31:0/3184245276 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8e6c1039c0 con 0x7f8e5407af30 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.962+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8e5407af30 msgr2=0x7f8e5407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8e5407af30 0x7f8e5407d3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c00abe0 tx=0x7f8e5c00a2d0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 msgr2=0x7f8e6c0fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c0fe5e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8e6002a9f0 tx=0x7f8e60005610 comp rx=0 tx=0).stop 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 shutdown_connections 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8e5407af30 0x7f8e5407d3f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e6c102f60 0x7f8e6c0fe5e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 --2- 172.21.15.31:0/3184245276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e6c100630 0x7f8e6c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:40.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 >> 172.21.15.31:0/3184245276 conn(0x7f8e6c0fa250 msgr2=0x7f8e6c0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 shutdown_connections 2024-09-16T09:02:40.965 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:40.963+0000 7f8e71219640 1 -- 172.21.15.31:0/3184245276 wait complete. 2024-09-16T09:02:41.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 -- 172.21.15.31:0/3977053969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 msgr2=0x7fc9581053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 --2- 172.21.15.31:0/3977053969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc9581053a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0098b0 tx=0x7fc94c02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 -- 172.21.15.31:0/3977053969 shutdown_connections 2024-09-16T09:02:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 --2- 172.21.15.31:0/3977053969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc9581053a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 --2- 172.21.15.31:0/3977053969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100680 0x7fc958102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 -- 172.21.15.31:0/3977053969 >> 172.21.15.31:0/3977053969 conn(0x7fc9580fa280 msgr2=0x7fc9580fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 -- 172.21.15.31:0/3977053969 shutdown_connections 2024-09-16T09:02:41.052 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.050+0000 7fc95cb99640 1 -- 172.21.15.31:0/3977053969 wait complete. 2024-09-16T09:02:41.052 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:40 smithi031 ceph-mon[91884]: pgmap v125: 65 pgs: 8 peering, 4 remapped+peering, 2 active+clean+remapped, 12 active+undersized, 17 active+undersized+degraded, 22 active+clean; 201 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 51/282 objects degraded (18.085%); 3/282 objects misplaced (1.064%) 2024-09-16T09:02:41.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:40 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 51/282 objects degraded (18.085%), 17 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:41.053 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:40 smithi031 ceph-mon[91884]: osdmap e76: 6 total, 6 up, 6 in 2024-09-16T09:02:41.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc95cb99640 1 Processor -- start 2024-09-16T09:02:41.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc95cb99640 1 -- start start 2024-09-16T09:02:41.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc95cb99640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100680 0x7fc958197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc95cb99640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc958197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc95cb99640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958198450 con 0x7fc958102fb0 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc95cb99640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9581985c0 con 0x7fc958100680 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc955d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc958197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc955d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc958197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45520/0 (socket says 172.21.15.31:45520) 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc955d9b640 1 -- 172.21.15.31:0/2790012606 learned_addr learned my addr 172.21.15.31:0/2790012606 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.051+0000 7fc95659c640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100680 0x7fc958197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc955d9b640 1 -- 172.21.15.31:0/2790012606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100680 msgr2=0x7fc958197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc955d9b640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100680 0x7fc958197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc955d9b640 1 -- 172.21.15.31:0/2790012606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c009560 con 0x7fc958102fb0 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc95659c640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100680 0x7fc958197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc955d9b640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc958197e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009230 tx=0x7fc94c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:41.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc9477fe640 1 -- 172.21.15.31:0/2790012606 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc94c004aa0 con 0x7fc958102fb0 2024-09-16T09:02:41.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc9477fe640 1 -- 172.21.15.31:0/2790012606 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc94c005d50 con 0x7fc958102fb0 2024-09-16T09:02:41.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95819d000 con 0x7fc958102fb0 2024-09-16T09:02:41.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95819d4f0 con 0x7fc958102fb0 2024-09-16T09:02:41.056 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.052+0000 7fc9477fe640 1 -- 172.21.15.31:0/2790012606 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc94c0039b0 con 0x7fc958102fb0 2024-09-16T09:02:41.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.054+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc938005250 con 0x7fc958102fb0 2024-09-16T09:02:41.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.054+0000 7fc9477fe640 1 -- 172.21.15.31:0/2790012606 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fc94c003c20 con 0x7fc958102fb0 2024-09-16T09:02:41.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.055+0000 7fc9477fe640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007af30 0x7fc94007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.055+0000 7fc9477fe640 1 -- 172.21.15.31:0/2790012606 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6827+0+0 (secure 0 0 0) 0x7fc94c0bef50 con 0x7fc958102fb0 2024-09-16T09:02:41.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.055+0000 7fc95659c640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007af30 0x7fc94007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.057 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.055+0000 7fc95659c640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007af30 0x7fc94007d3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc948005fd0 tx=0x7fc948009500 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:41.060 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.059+0000 7fc9477fe640 1 -- 172.21.15.31:0/2790012606 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fc94c0f6190 con 0x7fc958102fb0 2024-09-16T09:02:41.257 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.255+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc938000bc0 con 0x7fc94007af30 2024-09-16T09:02:41.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:40 smithi097 ceph-mon[79892]: pgmap v125: 65 pgs: 8 peering, 4 remapped+peering, 2 active+clean+remapped, 12 active+undersized, 17 active+undersized+degraded, 22 active+clean; 201 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 51/282 objects degraded (18.085%); 3/282 objects misplaced (1.064%) 2024-09-16T09:02:41.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:40 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 51/282 objects degraded (18.085%), 17 pgs degraded (PG_DEGRADED) 2024-09-16T09:02:41.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:40 smithi097 ceph-mon[79892]: osdmap e76: 6 total, 6 up, 6 in 2024-09-16T09:02:41.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.265+0000 7fc9477fe640 1 -- 172.21.15.31:0/2790012606 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc938000bc0 con 0x7fc94007af30 2024-09-16T09:02:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:02:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (12m) 42s ago 14m 25.3M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:02:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (5m) 42s ago 14m 8476k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:02:41.266 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (4m) 8s ago 13m 7755k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (2m) 42s ago 14m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (2m) 8s ago 13m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (12m) 42s ago 13m 85.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (8m) 42s ago 7m 16.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (7m) 42s ago 7m 153M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (7m) 8s ago 7m 26.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (7m) 8s ago 7m 18.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (4m) 42s ago 15m 516M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (3m) 8s ago 13m 454M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (3m) 42s ago 15m 48.2M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (3m) 8s ago 13m 39.4M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:02:41.267 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (14m) 42s ago 14m 15.8M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (13m) 8s ago 13m 16.7M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (2m) 42s ago 11m 93.6M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (84s) 42s ago 11m 98.7M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (48s) 42s ago 10m 13.3M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (11s) 8s ago 10m 12.2M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (9m) 8s ago 9m 252M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 90bf97c38fb9 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (9m) 8s ago 9m 265M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T09:02:41.268 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (3m) 42s ago 13m 65.2M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:02:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007af30 msgr2=0x7fc94007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.269 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007af30 0x7fc94007d3f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc948005fd0 tx=0x7fc948009500 comp rx=0 tx=0).stop 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 msgr2=0x7fc958197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc958197e80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009230 tx=0x7fc94c0055a0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 shutdown_connections 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc94007af30 0x7fc94007d3f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc958102fb0 0x7fc958197e80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 --2- 172.21.15.31:0/2790012606 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc958100680 0x7fc958197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 >> 172.21.15.31:0/2790012606 conn(0x7fc9580fa280 msgr2=0x7fc9580ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 shutdown_connections 2024-09-16T09:02:41.270 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.268+0000 7fc95cb99640 1 -- 172.21.15.31:0/2790012606 wait complete. 2024-09-16T09:02:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.350+0000 7f003edc5640 1 -- 172.21.15.31:0/1817824598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 msgr2=0x7f0038100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.350+0000 7f003edc5640 1 --2- 172.21.15.31:0/1817824598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038100170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f002c009880 tx=0x7f002c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 -- 172.21.15.31:0/1817824598 shutdown_connections 2024-09-16T09:02:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 --2- 172.21.15.31:0/1817824598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038100170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 --2- 172.21.15.31:0/1817824598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380feaf0 0x7f00380feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 -- 172.21.15.31:0/1817824598 >> 172.21.15.31:0/1817824598 conn(0x7f00380fa280 msgr2=0x7f00380fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:41.352 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 -- 172.21.15.31:0/1817824598 shutdown_connections 2024-09-16T09:02:41.353 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 -- 172.21.15.31:0/1817824598 wait complete. 2024-09-16T09:02:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 Processor -- start 2024-09-16T09:02:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.351+0000 7f003edc5640 1 -- start start 2024-09-16T09:02:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003edc5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380feaf0 0x7f0038195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003edc5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003edc5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038196220 con 0x7f00380ffcf0 2024-09-16T09:02:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003edc5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0038196390 con 0x7f00380feaf0 2024-09-16T09:02:41.354 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f0037fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f0037fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038195c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45538/0 (socket says 172.21.15.31:45538) 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003cb61640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380feaf0 0x7f0038195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f0037fff640 1 -- 172.21.15.31:0/1715905559 learned_addr learned my addr 172.21.15.31:0/1715905559 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003cb61640 1 -- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 msgr2=0x7f0038195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003cb61640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003cb61640 1 -- 172.21.15.31:0/1715905559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c009560 con 0x7f00380feaf0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f0037fff640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.352+0000 7f003cb61640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380feaf0 0x7f0038195710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f002800e940 tx=0x7f002800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.353+0000 7f0035ffb640 1 -- 172.21.15.31:0/1715905559 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f002800cc90 con 0x7f00380feaf0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.353+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003819ae30 con 0x7f00380feaf0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.353+0000 7f0035ffb640 1 -- 172.21.15.31:0/1715905559 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00280044f0 con 0x7f00380feaf0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.353+0000 7f0035ffb640 1 -- 172.21.15.31:0/1715905559 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0028010430 con 0x7f00380feaf0 2024-09-16T09:02:41.355 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.353+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003819b2f0 con 0x7f00380feaf0 2024-09-16T09:02:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.354+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00380feef0 con 0x7f00380feaf0 2024-09-16T09:02:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.355+0000 7f0035ffb640 1 -- 172.21.15.31:0/1715905559 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f0028004000 con 0x7f00380feaf0 2024-09-16T09:02:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.355+0000 7f0035ffb640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f002007af30 0x7f002007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.355+0000 7f0035ffb640 1 -- 172.21.15.31:0/1715905559 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6827+0+0 (secure 0 0 0) 0x7f0028014070 con 0x7f00380feaf0 2024-09-16T09:02:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.355+0000 7f0037fff640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f002007af30 0x7f002007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.356+0000 7f0037fff640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f002007af30 0x7f002007d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0038196c30 tx=0x7f002c0058d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:41.365 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.364+0000 7f0035ffb640 1 -- 172.21.15.31:0/1715905559 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f00280614b0 con 0x7f00380feaf0 2024-09-16T09:02:41.626 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.625+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f003819b8b0 con 0x7f00380feaf0 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.626+0000 7f0035ffb640 1 -- 172.21.15.31:0/1715905559 <== mon.1 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f00280645d0 con 0x7f00380feaf0 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:41.627 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2, 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 6, 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 8 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:02:41.628 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:02:41.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f002007af30 msgr2=0x7f002007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f002007af30 0x7f002007d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0038196c30 tx=0x7f002c0058d0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380feaf0 msgr2=0x7f0038195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380feaf0 0x7f0038195710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f002800e940 tx=0x7f002800ee00 comp rx=0 tx=0).stop 2024-09-16T09:02:41.629 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 shutdown_connections 2024-09-16T09:02:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f002007af30 0x7f002007d3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f00380ffcf0 0x7f0038195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 --2- 172.21.15.31:0/1715905559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00380feaf0 0x7f0038195710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 >> 172.21.15.31:0/1715905559 conn(0x7f00380fa280 msgr2=0x7f0038067cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 shutdown_connections 2024-09-16T09:02:41.630 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.628+0000 7f003edc5640 1 -- 172.21.15.31:0/1715905559 wait complete. 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 -- 172.21.15.31:0/1305985898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4102fb0 msgr2=0x7f46b41053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 --2- 172.21.15.31:0/1305985898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4102fb0 0x7f46b41053a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f46a40098b0 tx=0x7f46a402a4d0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 -- 172.21.15.31:0/1305985898 shutdown_connections 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 --2- 172.21.15.31:0/1305985898 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4102fb0 0x7f46b41053a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 --2- 172.21.15.31:0/1305985898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4100680 0x7f46b4102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 -- 172.21.15.31:0/1305985898 >> 172.21.15.31:0/1305985898 conn(0x7f46b40fa280 msgr2=0x7f46b40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 -- 172.21.15.31:0/1305985898 shutdown_connections 2024-09-16T09:02:41.712 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 -- 172.21.15.31:0/1305985898 wait complete. 2024-09-16T09:02:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.711+0000 7f46bc12d640 1 Processor -- start 2024-09-16T09:02:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46bc12d640 1 -- start start 2024-09-16T09:02:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46bc12d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4100680 0x7f46b40fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46bc12d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4102fb0 0x7f46b40fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46bc12d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40feb70 con 0x7f46b4100680 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46bc12d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b40fece0 con 0x7f46b4102fb0 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b9ec9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4100680 0x7f46b40fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b9ec9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4100680 0x7f46b40fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43688/0 (socket says 172.21.15.31:43688) 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b9ec9640 1 -- 172.21.15.31:0/3188700713 learned_addr learned my addr 172.21.15.31:0/3188700713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b96c8640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4102fb0 0x7f46b40fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b9ec9640 1 -- 172.21.15.31:0/3188700713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4102fb0 msgr2=0x7f46b40fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b9ec9640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4102fb0 0x7f46b40fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b9ec9640 1 -- 172.21.15.31:0/3188700713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a4009560 con 0x7f46b4100680 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b96c8640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4102fb0 0x7f46b40fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.712+0000 7f46b9ec9640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4100680 0x7f46b40fffe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46b000d6f0 tx=0x7f46b000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.713+0000 7f46aaffd640 1 -- 172.21.15.31:0/3188700713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46b0009a30 con 0x7f46b4100680 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.713+0000 7f46aaffd640 1 -- 172.21.15.31:0/3188700713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46b00054e0 con 0x7f46b4100680 2024-09-16T09:02:41.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.713+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b40fefc0 con 0x7f46b4100680 2024-09-16T09:02:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.713+0000 7f46aaffd640 1 -- 172.21.15.31:0/3188700713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46b0010460 con 0x7f46b4100680 2024-09-16T09:02:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.713+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b419d780 con 0x7f46b4100680 2024-09-16T09:02:41.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.714+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4109190 con 0x7f46b4100680 2024-09-16T09:02:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.714+0000 7f46aaffd640 1 -- 172.21.15.31:0/3188700713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f46b0009b90 con 0x7f46b4100680 2024-09-16T09:02:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.715+0000 7f46aaffd640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46a007af30 0x7f46a007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.715+0000 7f46aaffd640 1 -- 172.21.15.31:0/3188700713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6740+0+0 (secure 0 0 0) 0x7f46b009db10 con 0x7f46b4100680 2024-09-16T09:02:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.715+0000 7f46b96c8640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46a007af30 0x7f46a007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:41.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.715+0000 7f46b96c8640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46a007af30 0x7f46a007d3f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009880 tx=0x7f46a40314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:41.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.719+0000 7f46aaffd640 1 -- 172.21.15.31:0/3188700713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f46b0060c50 con 0x7f46b4100680 2024-09-16T09:02:41.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.942+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46b4100a60 con 0x7f46b4100680 2024-09-16T09:02:41.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.942+0000 7f46aaffd640 1 -- 172.21.15.31:0/3188700713 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7f46b0020070 con 0x7f46b4100680 2024-09-16T09:02:41.944 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:02:41.944 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:02:41.944 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:02:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:02:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:02:41.945 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:02:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:02:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:02:41.945 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:02:41.946 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:02:41.947 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:41.948 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46a007af30 msgr2=0x7f46a007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46a007af30 0x7f46a007d3f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009880 tx=0x7f46a40314e0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4100680 msgr2=0x7f46b40fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4100680 0x7f46b40fffe0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f46b000d6f0 tx=0x7f46b000dbb0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 shutdown_connections 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46a007af30 0x7f46a007d3f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46b4102fb0 0x7f46b40fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 --2- 172.21.15.31:0/3188700713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46b4100680 0x7f46b40fffe0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:41.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 >> 172.21.15.31:0/3188700713 conn(0x7f46b40fa280 msgr2=0x7f46b40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:41.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 shutdown_connections 2024-09-16T09:02:41.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:41.945+0000 7f46bc12d640 1 -- 172.21.15.31:0/3188700713 wait complete. 2024-09-16T09:02:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 -- 172.21.15.31:0/1693715526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600ffca0 msgr2=0x7fa160100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 --2- 172.21.15.31:0/1693715526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600ffca0 0x7fa160100120 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa150009880 tx=0x7fa15002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 -- 172.21.15.31:0/1693715526 shutdown_connections 2024-09-16T09:02:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 --2- 172.21.15.31:0/1693715526 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600ffca0 0x7fa160100120 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 --2- 172.21.15.31:0/1693715526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1600feaa0 0x7fa1600feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.033 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 -- 172.21.15.31:0/1693715526 >> 172.21.15.31:0/1693715526 conn(0x7fa1600fa250 msgr2=0x7fa1600fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 -- 172.21.15.31:0/1693715526 shutdown_connections 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.030+0000 7fa167951640 1 -- 172.21.15.31:0/1693715526 wait complete. 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa167951640 1 Processor -- start 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa167951640 1 -- start start 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa167951640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600feaa0 0x7fa160195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa167951640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1600ffca0 0x7fa160195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa167951640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa160196160 con 0x7fa1600feaa0 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa167951640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1601962d0 con 0x7fa1600ffca0 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa1656ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600feaa0 0x7fa160195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa1656ed640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600feaa0 0x7fa160195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43710/0 (socket says 172.21.15.31:43710) 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa1656ed640 1 -- 172.21.15.31:0/2472796640 learned_addr learned my addr 172.21.15.31:0/2472796640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa164eec640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1600ffca0 0x7fa160195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:42.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa1656ed640 1 -- 172.21.15.31:0/2472796640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1600ffca0 msgr2=0x7fa160195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa1656ed640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1600ffca0 0x7fa160195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.031+0000 7fa1656ed640 1 -- 172.21.15.31:0/2472796640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150009560 con 0x7fa1600feaa0 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.032+0000 7fa1656ed640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600feaa0 0x7fa160195650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00e970 tx=0x7fa15c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.032+0000 7fa1567fc640 1 -- 172.21.15.31:0/2472796640 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa15c0096e0 con 0x7fa1600feaa0 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.032+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16019ad70 con 0x7fa1600feaa0 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.032+0000 7fa1567fc640 1 -- 172.21.15.31:0/2472796640 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa15c0044f0 con 0x7fa1600feaa0 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.032+0000 7fa1567fc640 1 -- 172.21.15.31:0/2472796640 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa15c010430 con 0x7fa1600feaa0 2024-09-16T09:02:42.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.032+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16019b2b0 con 0x7fa1600feaa0 2024-09-16T09:02:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.033+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1601076b0 con 0x7fa1600feaa0 2024-09-16T09:02:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.033+0000 7fa1567fc640 1 -- 172.21.15.31:0/2472796640 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fa15c00cc90 con 0x7fa1600feaa0 2024-09-16T09:02:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.034+0000 7fa1567fc640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa14c07af30 0x7fa14c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.034+0000 7fa1567fc640 1 -- 172.21.15.31:0/2472796640 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6740+0+0 (secure 0 0 0) 0x7fa15c014070 con 0x7fa1600feaa0 2024-09-16T09:02:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.034+0000 7fa164eec640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa14c07af30 0x7fa14c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:42.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.035+0000 7fa164eec640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa14c07af30 0x7fa14c07d3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa150000c00 tx=0x7fa1500314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:42.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.038+0000 7fa1567fc640 1 -- 172.21.15.31:0/2472796640 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fa15c061590 con 0x7fa1600feaa0 2024-09-16T09:02:42.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.244+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa16019b550 con 0x7fa14c07af30 2024-09-16T09:02:42.246 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:41 smithi031 ceph-mon[91884]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:42.246 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:41 smithi031 ceph-mon[91884]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:42.246 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:41 smithi031 ceph-mon[91884]: osdmap e77: 6 total, 6 up, 6 in 2024-09-16T09:02:42.246 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:41 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1715905559' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:42.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.245+0000 7fa1567fc640 1 -- 172.21.15.31:0/2472796640 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa16019b550 con 0x7fa14c07af30 2024-09-16T09:02:42.246 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:02:42.246 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "10/23 daemons upgraded", 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:02:42.247 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.247+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa14c07af30 msgr2=0x7fa14c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.247+0000 7fa167951640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa14c07af30 0x7fa14c07d3f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa150000c00 tx=0x7fa1500314e0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.247+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600feaa0 msgr2=0x7fa160195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.247+0000 7fa167951640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600feaa0 0x7fa160195650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00e970 tx=0x7fa15c00ee30 comp rx=0 tx=0).stop 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.248+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 shutdown_connections 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.248+0000 7fa167951640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa14c07af30 0x7fa14c07d3f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.248+0000 7fa167951640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa1600ffca0 0x7fa160195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.248+0000 7fa167951640 1 --2- 172.21.15.31:0/2472796640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1600feaa0 0x7fa160195650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.248+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 >> 172.21.15.31:0/2472796640 conn(0x7fa1600fa250 msgr2=0x7fa160068520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:42.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.248+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 shutdown_connections 2024-09-16T09:02:42.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.248+0000 7fa167951640 1 -- 172.21.15.31:0/2472796640 wait complete. 2024-09-16T09:02:42.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:41 smithi097 ceph-mon[79892]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:42.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:41 smithi097 ceph-mon[79892]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:41 smithi097 ceph-mon[79892]: osdmap e77: 6 total, 6 up, 6 in 2024-09-16T09:02:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:41 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1715905559' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:02:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 -- 172.21.15.31:0/843991085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 msgr2=0x7fac600fe620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.333 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 --2- 172.21.15.31:0/843991085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac600fe620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fac50009c20 tx=0x7fac5002a970 comp rx=0 tx=0).stop 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 -- 172.21.15.31:0/843991085 shutdown_connections 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 --2- 172.21.15.31:0/843991085 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac600feb60 0x7fac600fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 --2- 172.21.15.31:0/843991085 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac600fe620 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 -- 172.21.15.31:0/843991085 >> 172.21.15.31:0/843991085 conn(0x7fac600fa010 msgr2=0x7fac600fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 -- 172.21.15.31:0/843991085 shutdown_connections 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.331+0000 7fac66fa0640 1 -- 172.21.15.31:0/843991085 wait complete. 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac66fa0640 1 Processor -- start 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac66fa0640 1 -- start start 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac66fa0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac60197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac66fa0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac600feb60 0x7fac60197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac66fa0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac60198140 con 0x7fac600fe220 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac66fa0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac601982b0 con 0x7fac600feb60 2024-09-16T09:02:42.334 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac65f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac60197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:42.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac65f9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac60197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43726/0 (socket says 172.21.15.31:43726) 2024-09-16T09:02:42.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac65f9e640 1 -- 172.21.15.31:0/197811397 learned_addr learned my addr 172.21.15.31:0/197811397 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:02:42.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.332+0000 7fac6579d640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac600feb60 0x7fac60197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:42.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac65f9e640 1 -- 172.21.15.31:0/197811397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac600feb60 msgr2=0x7fac60197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac65f9e640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac600feb60 0x7fac60197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac65f9e640 1 -- 172.21.15.31:0/197811397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac50009900 con 0x7fac600fe220 2024-09-16T09:02:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac65f9e640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac60197630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac50009d40 tx=0x7fac5002f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac5effd640 1 -- 172.21.15.31:0/197811397 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac50030840 con 0x7fac600fe220 2024-09-16T09:02:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac5effd640 1 -- 172.21.15.31:0/197811397 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac5002fa20 con 0x7fac600fe220 2024-09-16T09:02:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6019cca0 con 0x7fac600fe220 2024-09-16T09:02:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac5effd640 1 -- 172.21.15.31:0/197811397 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac5003a490 con 0x7fac600fe220 2024-09-16T09:02:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.333+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6019d190 con 0x7fac600fe220 2024-09-16T09:02:42.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.334+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac60191700 con 0x7fac600fe220 2024-09-16T09:02:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.335+0000 7fac5effd640 1 -- 172.21.15.31:0/197811397 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fac5000a040 con 0x7fac600fe220 2024-09-16T09:02:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.335+0000 7fac5effd640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fac4807af30 0x7fac4807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:02:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.336+0000 7fac6579d640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fac4807af30 0x7fac4807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:02:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.336+0000 7fac5effd640 1 -- 172.21.15.31:0/197811397 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6740+0+0 (secure 0 0 0) 0x7fac5002c040 con 0x7fac600fe220 2024-09-16T09:02:42.337 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.336+0000 7fac6579d640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fac4807af30 0x7fac4807d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac60198b00 tx=0x7fac54005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:02:42.341 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.340+0000 7fac5effd640 1 -- 172.21.15.31:0/197811397 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fac50082800 con 0x7fac600fe220 2024-09-16T09:02:42.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.596+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fac6010a770 con 0x7fac600fe220 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.597+0000 7fac5effd640 1 -- 172.21.15.31:0/197811397 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+795 (secure 0 0 0) 0x7fac50085920 con 0x7fac600fe220 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN Degraded data redundancy: 33/282 objects degraded (11.702%), 11 pgs degraded 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 33/282 objects degraded (11.702%), 11 pgs degraded 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-09-16T09:02:42.598 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.12 is active+undersized+degraded, acting [1,0] 2024-09-16T09:02:42.599 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-09-16T09:02:42.599 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-09-16T09:02:42.599 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-09-16T09:02:42.599 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fac4807af30 msgr2=0x7fac4807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fac4807af30 0x7fac4807d3f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac60198b00 tx=0x7fac54005f90 comp rx=0 tx=0).stop 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 msgr2=0x7fac60197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac60197630 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fac50009d40 tx=0x7fac5002f5f0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 shutdown_connections 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fac4807af30 0x7fac4807d3f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac600feb60 0x7fac60197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 --2- 172.21.15.31:0/197811397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac600fe220 0x7fac60197630 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:02:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 >> 172.21.15.31:0/197811397 conn(0x7fac600fa010 msgr2=0x7fac60106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:02:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 shutdown_connections 2024-09-16T09:02:42.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:02:42.600+0000 7fac66fa0640 1 -- 172.21.15.31:0/197811397 wait complete. 2024-09-16T09:02:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:42 smithi097 ceph-mon[79892]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:42 smithi097 ceph-mon[79892]: pgmap v128: 65 pgs: 9 peering, 5 remapped+peering, 9 active+undersized, 11 active+undersized+degraded, 31 active+clean; 201 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 33/282 objects degraded (11.702%) 2024-09-16T09:02:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:42 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3188700713' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:02:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:42 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/197811397' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:02:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:42 smithi097 ceph-mon[79892]: osdmap e78: 6 total, 6 up, 6 in 2024-09-16T09:02:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:42 smithi031 ceph-mon[91884]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:42 smithi031 ceph-mon[91884]: pgmap v128: 65 pgs: 9 peering, 5 remapped+peering, 9 active+undersized, 11 active+undersized+degraded, 31 active+clean; 201 MiB data, 971 MiB used, 535 GiB / 536 GiB avail; 33/282 objects degraded (11.702%) 2024-09-16T09:02:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:42 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3188700713' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:02:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:42 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/197811397' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:02:43.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:42 smithi031 ceph-mon[91884]: osdmap e78: 6 total, 6 up, 6 in 2024-09-16T09:02:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:43 smithi097 ceph-mon[79892]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:43 smithi097 ceph-mon[79892]: osdmap e79: 6 total, 6 up, 6 in 2024-09-16T09:02:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:43 smithi031 ceph-mon[91884]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:02:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:43 smithi031 ceph-mon[91884]: osdmap e79: 6 total, 6 up, 6 in 2024-09-16T09:02:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:44 smithi097 ceph-mon[79892]: pgmap v131: 65 pgs: 9 peering, 5 remapped+peering, 3 active+undersized, 1 active+undersized+degraded, 47 active+clean; 201 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 3/282 objects degraded (1.064%) 2024-09-16T09:02:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:44 smithi097 ceph-mon[79892]: osdmap e80: 6 total, 6 up, 6 in 2024-09-16T09:02:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:44 smithi031 ceph-mon[91884]: pgmap v131: 65 pgs: 9 peering, 5 remapped+peering, 3 active+undersized, 1 active+undersized+degraded, 47 active+clean; 201 MiB data, 991 MiB used, 535 GiB / 536 GiB avail; 3/282 objects degraded (1.064%) 2024-09-16T09:02:45.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:44 smithi031 ceph-mon[91884]: osdmap e80: 6 total, 6 up, 6 in 2024-09-16T09:02:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:46 smithi097 ceph-mon[79892]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/282 objects degraded (1.064%), 1 pg degraded) 2024-09-16T09:02:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:46 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:02:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:46 smithi031 ceph-mon[91884]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3/282 objects degraded (1.064%), 1 pg degraded) 2024-09-16T09:02:46.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:46 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:02:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:47 smithi097 ceph-mon[79892]: pgmap v134: 65 pgs: 2 peering, 1 remapped+peering, 62 active+clean; 201 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 370 objects/s recovering 2024-09-16T09:02:47.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:47 smithi031 ceph-mon[91884]: pgmap v134: 65 pgs: 2 peering, 1 remapped+peering, 62 active+clean; 201 MiB data, 999 MiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s, 370 objects/s recovering 2024-09-16T09:02:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:49 smithi097 ceph-mon[79892]: pgmap v135: 65 pgs: 1 peering, 64 active+clean; 201 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s, 264 objects/s recovering 2024-09-16T09:02:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:49 smithi031 ceph-mon[91884]: pgmap v135: 65 pgs: 1 peering, 64 active+clean; 201 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s, 264 objects/s recovering 2024-09-16T09:02:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:51 smithi031 ceph-mon[91884]: pgmap v136: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s, 225 objects/s recovering 2024-09-16T09:02:51.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:51 smithi097 ceph-mon[79892]: pgmap v136: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s, 225 objects/s recovering 2024-09-16T09:02:52.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:52 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-16T09:02:52.339 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:52 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16} : dispatch 2024-09-16T09:02:53.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:53 smithi097 ceph-mon[79892]: pgmap v137: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s, 188 objects/s recovering 2024-09-16T09:02:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:53 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T09:02:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:53 smithi097 ceph-mon[79892]: Upgrade: osd.4 is safe to restart 2024-09-16T09:02:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T09:02:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:53 smithi031 ceph-mon[91884]: pgmap v137: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s, 188 objects/s recovering 2024-09-16T09:02:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:53 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-16T09:02:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:53 smithi031 ceph-mon[91884]: Upgrade: osd.4 is safe to restart 2024-09-16T09:02:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T09:02:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:02:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:54 smithi031 ceph-mon[91884]: Upgrade: Updating osd.4 2024-09-16T09:02:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:54 smithi031 ceph-mon[91884]: Deploying daemon osd.4 on smithi097 2024-09-16T09:02:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:54.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:54.447 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:54 smithi097 ceph-mon[79892]: Upgrade: Updating osd.4 2024-09-16T09:02:54.447 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:54 smithi097 ceph-mon[79892]: Deploying daemon osd.4 on smithi097 2024-09-16T09:02:54.447 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:54.447 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:02:55.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:55 smithi031 ceph-mon[91884]: pgmap v138: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s, 155 objects/s recovering 2024-09-16T09:02:55.482 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:55 smithi097 ceph-mon[79892]: pgmap v138: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s, 155 objects/s recovering 2024-09-16T09:02:56.084 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:55 smithi097 systemd[1]: Stopping Ceph osd.4 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:02:56.511 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:56 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[48369]: 2024-09-16T09:02:56.119+0000 7f6a2bd03640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:02:56.512 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:56 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[48369]: 2024-09-16T09:02:56.119+0000 7f6a2bd03640 -1 osd.4 80 *** Got signal Terminated *** 2024-09-16T09:02:56.512 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:56 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[48369]: 2024-09-16T09:02:56.119+0000 7f6a2bd03640 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:02:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:57 smithi031 ceph-mon[91884]: pgmap v139: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 383 KiB/s, 2 objects/s recovering 2024-09-16T09:02:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:57 smithi031 ceph-mon[91884]: osd.4 marked itself down and dead 2024-09-16T09:02:57.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:57 smithi097 ceph-mon[79892]: pgmap v139: 65 pgs: 65 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 383 KiB/s, 2 objects/s recovering 2024-09-16T09:02:57.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:57 smithi097 ceph-mon[79892]: osd.4 marked itself down and dead 2024-09-16T09:02:57.845 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:57 smithi097 podman[93293]: 2024-09-16 09:02:57.575518083 +0000 UTC m=+1.595956982 container died 90bf97c38fb96a8446390ad4db97a0f11e44168826e9070ac6f703bcacad9923 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T09:02:58.221 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:58 smithi097 ceph-mon[79892]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:02:58.221 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:58 smithi097 ceph-mon[79892]: osdmap e81: 6 total, 5 up, 6 in 2024-09-16T09:02:58.221 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:57 smithi097 podman[93293]: 2024-09-16 09:02:57.952897543 +0000 UTC m=+1.973336432 container cleanup 90bf97c38fb96a8446390ad4db97a0f11e44168826e9070ac6f703bcacad9923 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:02:58.221 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:57 smithi097 bash[93293]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4 2024-09-16T09:02:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:58 smithi031 ceph-mon[91884]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:02:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:58 smithi031 ceph-mon[91884]: osdmap e81: 6 total, 5 up, 6 in 2024-09-16T09:02:58.511 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:58 smithi097 podman[93319]: 2024-09-16 09:02:58.22071058 +0000 UTC m=+0.646817002 container remove 90bf97c38fb96a8446390ad4db97a0f11e44168826e9070ac6f703bcacad9923 (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T09:02:58.833 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:58 smithi097 podman[93409]: 2024-09-16 09:02:58.484441887 +0000 UTC m=+0.015863900 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:02:58.833 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:58 smithi097 podman[93409]: 2024-09-16 09:02:58.597028025 +0000 UTC m=+0.128450038 container create d4d853c05bf4f53c6598fe2918c940513ea175a844ba6589a0fb8faab4ac6057 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:02:58.833 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:58 smithi097 podman[93409]: 2024-09-16 09:02:58.832956358 +0000 UTC m=+0.364378374 container init d4d853c05bf4f53c6598fe2918c940513ea175a844ba6589a0fb8faab4ac6057 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T09:02:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:59 smithi097 ceph-mon[79892]: pgmap v141: 65 pgs: 6 stale+active+clean, 59 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:59 smithi097 ceph-mon[79892]: osdmap e82: 6 total, 5 up, 6 in 2024-09-16T09:02:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:02:59 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:02:59.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:58 smithi097 podman[93409]: 2024-09-16 09:02:58.83659808 +0000 UTC m=+0.368020098 container start d4d853c05bf4f53c6598fe2918c940513ea175a844ba6589a0fb8faab4ac6057 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T09:02:59.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:58 smithi097 podman[93409]: 2024-09-16 09:02:58.924334288 +0000 UTC m=+0.455756310 container attach d4d853c05bf4f53c6598fe2918c940513ea175a844ba6589a0fb8faab4ac6057 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD) 2024-09-16T09:02:59.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:59 smithi097 podman[93409]: 2024-09-16 09:02:59.023799752 +0000 UTC m=+0.555221773 container died d4d853c05bf4f53c6598fe2918c940513ea175a844ba6589a0fb8faab4ac6057 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:02:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:59 smithi031 ceph-mon[91884]: pgmap v141: 65 pgs: 6 stale+active+clean, 59 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail 2024-09-16T09:02:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:59 smithi031 ceph-mon[91884]: osdmap e82: 6 total, 5 up, 6 in 2024-09-16T09:02:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:02:59 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:00.012 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:59 smithi097 podman[93432]: 2024-09-16 09:02:59.602894893 +0000 UTC m=+0.568228200 container remove d4d853c05bf4f53c6598fe2918c940513ea175a844ba6589a0fb8faab4ac6057 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T09:03:00.013 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:59 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.4.service: Deactivated successfully. 2024-09-16T09:03:00.013 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:59 smithi097 systemd[1]: Stopped Ceph osd.4 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:03:00.013 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:59 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.4.service: Consumed 1min 862ms CPU time. 2024-09-16T09:03:00.013 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:02:59 smithi097 systemd[1]: Starting Ceph osd.4 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:03:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:00 smithi031 ceph-mon[91884]: pgmap v143: 65 pgs: 6 active+undersized, 12 peering, 2 stale+active+clean, 4 active+undersized+degraded, 41 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 12/282 objects degraded (4.255%) 2024-09-16T09:03:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:00 smithi031 ceph-mon[91884]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:03:00.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:00 smithi031 ceph-mon[91884]: Health check failed: Degraded data redundancy: 12/282 objects degraded (4.255%), 4 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:00.374 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:00 smithi097 ceph-mon[79892]: pgmap v143: 65 pgs: 6 active+undersized, 12 peering, 2 stale+active+clean, 4 active+undersized+degraded, 41 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 12/282 objects degraded (4.255%) 2024-09-16T09:03:00.374 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:00 smithi097 ceph-mon[79892]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:03:00.374 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:00 smithi097 ceph-mon[79892]: Health check failed: Degraded data redundancy: 12/282 objects degraded (4.255%), 4 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:00.374 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 podman[93534]: 2024-09-16 09:02:59.935446745 +0000 UTC m=+0.016537452 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:03:00.374 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 podman[93534]: 2024-09-16 09:03:00.11264233 +0000 UTC m=+0.193733077 container create df972715ccd4db509256b7a8d268083936d31927b6d465af8493f1d80a436c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T09:03:00.374 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 podman[93534]: 2024-09-16 09:03:00.373661091 +0000 UTC m=+0.454751802 container init df972715ccd4db509256b7a8d268083936d31927b6d465af8493f1d80a436c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T09:03:00.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 podman[93534]: 2024-09-16 09:03:00.376912284 +0000 UTC m=+0.458002994 container start df972715ccd4db509256b7a8d268083936d31927b6d465af8493f1d80a436c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=) 2024-09-16T09:03:00.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 podman[93534]: 2024-09-16 09:03:00.481997597 +0000 UTC m=+0.563088310 container attach df972715ccd4db509256b7a8d268083936d31927b6d465af8493f1d80a436c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T09:03:00.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:00.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 bash[93534]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:00.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:00.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:00 smithi097 bash[93534]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:02 smithi031 ceph-mon[91884]: pgmap v144: 65 pgs: 12 active+undersized, 12 peering, 8 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 21/282 objects degraded (7.447%) 2024-09-16T09:03:02.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:02 smithi097 ceph-mon[79892]: pgmap v144: 65 pgs: 12 active+undersized, 12 peering, 8 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 21/282 objects degraded (7.447%) 2024-09-16T09:03:03.136 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:02 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T09:03:03.136 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:02 smithi097 bash[93534]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T09:03:03.136 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:02 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T09:03:03.136 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:02 smithi097 bash[93534]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-16T09:03:03.137 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T09:03:03.137 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 bash[93534]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 bash[93534]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 bash[93534]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 bash[93534]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate[93549]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-16T09:03:03.416 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 bash[93534]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-16T09:03:03.417 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 podman[93534]: 2024-09-16 09:03:03.171005777 +0000 UTC m=+3.252096479 container died df972715ccd4db509256b7a8d268083936d31927b6d465af8493f1d80a436c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:03:03.752 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:03 smithi097 ceph-mon[79892]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T09:03:03.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:03 smithi031 ceph-mon[91884]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T09:03:04.011 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 podman[93722]: 2024-09-16 09:03:03.752316831 +0000 UTC m=+0.569167902 container remove df972715ccd4db509256b7a8d268083936d31927b6d465af8493f1d80a436c82 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-activate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T09:03:04.012 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 podman[93782]: 2024-09-16 09:03:03.870412392 +0000 UTC m=+0.016430460 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:03:04.012 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:03 smithi097 podman[93782]: 2024-09-16 09:03:03.978226567 +0000 UTC m=+0.124244678 container create 3c82b6b87cd9abfceee7117bb7e55914dd05350e0161ff1824b10bb4f2161880 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T09:03:04.417 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:04 smithi097 podman[93782]: 2024-09-16 09:03:04.205470413 +0000 UTC m=+0.351488486 container init 3c82b6b87cd9abfceee7117bb7e55914dd05350e0161ff1824b10bb4f2161880 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True) 2024-09-16T09:03:04.417 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:04 smithi097 podman[93782]: 2024-09-16 09:03:04.208584631 +0000 UTC m=+0.354602706 container start 3c82b6b87cd9abfceee7117bb7e55914dd05350e0161ff1824b10bb4f2161880 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:03:04.417 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:04 smithi097 ceph-osd[93802]: -- 172.21.15.97:0/2980669702 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d2300e1e0 con 0x555d2221bc00 2024-09-16T09:03:04.417 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:04 smithi097 bash[93782]: 3c82b6b87cd9abfceee7117bb7e55914dd05350e0161ff1824b10bb4f2161880 2024-09-16T09:03:04.417 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:04 smithi097 systemd[1]: Started Ceph osd.4 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:03:04.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:04 smithi097 ceph-mon[79892]: pgmap v145: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:04.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:04.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:04.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:04.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:04 smithi031 ceph-mon[91884]: pgmap v145: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:04.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:04.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:04.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:06.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:05 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:03:05.860+0000 7faf1fd5e640 -1 Falling back to public interface 2024-09-16T09:03:06.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:06 smithi097 ceph-mon[79892]: pgmap v146: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:06 smithi031 ceph-mon[91884]: pgmap v146: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:07.511 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:07 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 46/282 objects degraded (16.312%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:07 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 46/282 objects degraded (16.312%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:08.861 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:08 smithi097 ceph-mon[79892]: pgmap v147: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:08.862 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:08.862 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:08.862 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:08.862 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:08 smithi031 ceph-mon[91884]: pgmap v147: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:09.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:09.939 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:09.939 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:11.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:10 smithi097 ceph-mon[79892]: pgmap v148: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:11.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:10 smithi031 ceph-mon[91884]: pgmap v148: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:12.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:03:12.135+0000 7faf1fd5e640 -1 osd.4 80 log_to_monitors true 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.698+0000 7f10f897f640 1 -- 172.21.15.31:0/1904506164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4100630 msgr2=0x7f10f4102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.698+0000 7f10f897f640 1 --2- 172.21.15.31:0/1904506164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4100630 0x7f10f4102a20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f10e40098b0 tx=0x7f10e402a4d0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.698+0000 7f10f897f640 1 -- 172.21.15.31:0/1904506164 shutdown_connections 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.698+0000 7f10f897f640 1 --2- 172.21.15.31:0/1904506164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f4102f60 0x7f10f4105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.698+0000 7f10f897f640 1 --2- 172.21.15.31:0/1904506164 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4100630 0x7f10f4102a20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.698+0000 7f10f897f640 1 -- 172.21.15.31:0/1904506164 >> 172.21.15.31:0/1904506164 conn(0x7f10f40fa250 msgr2=0x7f10f40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.698+0000 7f10f897f640 1 -- 172.21.15.31:0/1904506164 shutdown_connections 2024-09-16T09:03:12.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.699+0000 7f10f897f640 1 -- 172.21.15.31:0/1904506164 wait complete. 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: pgmap v149: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:12.702 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-16T09:03:12.703 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T09:03:12.703 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-09-16T09:03:12.703 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='osd.4 [v2:172.21.15.97:6808/54702930,v1:172.21.15.97:6809/54702930]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:03:12.703 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:12 smithi031 ceph-mon[91884]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:03:12.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.699+0000 7f10f897f640 1 Processor -- start 2024-09-16T09:03:12.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f897f640 1 -- start start 2024-09-16T09:03:12.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f897f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f4100630 0x7f10f4195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:12.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f897f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4102f60 0x7f10f4195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f897f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f4196150 con 0x7f10f4102f60 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f897f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f41962c0 con 0x7f10f4100630 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4102f60 0x7f10f4195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f1d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4102f60 0x7f10f4195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51536/0 (socket says 172.21.15.31:51536) 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f1d9b640 1 -- 172.21.15.31:0/2492017664 learned_addr learned my addr 172.21.15.31:0/2492017664 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f259c640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f4100630 0x7f10f4195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f1d9b640 1 -- 172.21.15.31:0/2492017664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f4100630 msgr2=0x7f10f4195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f1d9b640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f4100630 0x7f10f4195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.700+0000 7f10f1d9b640 1 -- 172.21.15.31:0/2492017664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009560 con 0x7f10f4102f60 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.701+0000 7f10f1d9b640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4102f60 0x7f10f4195b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f10e800dba0 tx=0x7f10e800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.701+0000 7f10e37fe640 1 -- 172.21.15.31:0/2492017664 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e800dea0 con 0x7f10f4102f60 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.701+0000 7f10e37fe640 1 -- 172.21.15.31:0/2492017664 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f10e8003970 con 0x7f10f4102f60 2024-09-16T09:03:12.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.701+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f419ad60 con 0x7f10f4102f60 2024-09-16T09:03:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.701+0000 7f10e37fe640 1 -- 172.21.15.31:0/2492017664 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10e80047a0 con 0x7f10f4102f60 2024-09-16T09:03:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.701+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f419b2b0 con 0x7f10f4102f60 2024-09-16T09:03:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.702+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f41087c0 con 0x7f10f4102f60 2024-09-16T09:03:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.704+0000 7f10e37fe640 1 -- 172.21.15.31:0/2492017664 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f10e8003ae0 con 0x7f10f4102f60 2024-09-16T09:03:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.704+0000 7f10e37fe640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f10dc07af30 0x7f10dc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.704+0000 7f10e37fe640 1 -- 172.21.15.31:0/2492017664 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6682+0+0 (secure 0 0 0) 0x7f10e809e1f0 con 0x7f10f4102f60 2024-09-16T09:03:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.704+0000 7f10f259c640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f10dc07af30 0x7f10dc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:12.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.705+0000 7f10f259c640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f10dc07af30 0x7f10dc07d3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f10e4006010 tx=0x7f10e40314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:12.711 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.709+0000 7f10e37fe640 1 -- 172.21.15.31:0/2492017664 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f10e8061250 con 0x7f10f4102f60 2024-09-16T09:03:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: pgmap v149: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 992 MiB used, 535 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:03:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: Upgrade: unsafe to stop osd(s) at this time (9 PGs are or would become offline) 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='osd.4 [v2:172.21.15.97:6808/54702930,v1:172.21.15.97:6809/54702930]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:03:12.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-mon[79892]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:03:12.922 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.920+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f10f4103980 con 0x7f10dc07af30 2024-09-16T09:03:12.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.921+0000 7f10e37fe640 1 -- 172.21.15.31:0/2492017664 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f10f4103980 con 0x7f10dc07af30 2024-09-16T09:03:12.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f10dc07af30 msgr2=0x7f10dc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:12.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f10dc07af30 0x7f10dc07d3f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f10e4006010 tx=0x7f10e40314e0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4102f60 msgr2=0x7f10f4195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:12.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4102f60 0x7f10f4195b80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f10e800dba0 tx=0x7f10e800b630 comp rx=0 tx=0).stop 2024-09-16T09:03:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 shutdown_connections 2024-09-16T09:03:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f10dc07af30 0x7f10dc07d3f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10f4102f60 0x7f10f4195b80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 --2- 172.21.15.31:0/2492017664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10f4100630 0x7f10f4195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 >> 172.21.15.31:0/2492017664 conn(0x7f10f40fa250 msgr2=0x7f10f40fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 shutdown_connections 2024-09-16T09:03:12.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:12.923+0000 7f10f897f640 1 -- 172.21.15.31:0/2492017664 wait complete. 2024-09-16T09:03:12.934 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:03:13.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.008+0000 7fe7184b5640 1 -- 172.21.15.31:0/1355737929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710102f60 msgr2=0x7fe710105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.008+0000 7fe7184b5640 1 --2- 172.21.15.31:0/1355737929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710102f60 0x7fe710105350 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe7000098b0 tx=0x7fe70002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.008+0000 7fe7184b5640 1 -- 172.21.15.31:0/1355737929 shutdown_connections 2024-09-16T09:03:13.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.008+0000 7fe7184b5640 1 --2- 172.21.15.31:0/1355737929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710102f60 0x7fe710105350 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.008+0000 7fe7184b5640 1 --2- 172.21.15.31:0/1355737929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710100630 0x7fe710102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.008+0000 7fe7184b5640 1 -- 172.21.15.31:0/1355737929 >> 172.21.15.31:0/1355737929 conn(0x7fe7100fa250 msgr2=0x7fe7100fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:13.011 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.008+0000 7fe7184b5640 1 -- 172.21.15.31:0/1355737929 shutdown_connections 2024-09-16T09:03:13.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.009+0000 7fe7184b5640 1 -- 172.21.15.31:0/1355737929 wait complete. 2024-09-16T09:03:13.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.009+0000 7fe7184b5640 1 Processor -- start 2024-09-16T09:03:13.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.009+0000 7fe7184b5640 1 -- start start 2024-09-16T09:03:13.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe7184b5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710100630 0x7fe7100fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe7184b5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102f60 0x7fe7100fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe7184b5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100feb20 con 0x7fe710100630 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe7184b5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100fec90 con 0x7fe710102f60 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe716251640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710100630 0x7fe7100fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe716251640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710100630 0x7fe7100fff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51552/0 (socket says 172.21.15.31:51552) 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe716251640 1 -- 172.21.15.31:0/258643254 learned_addr learned my addr 172.21.15.31:0/258643254 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe715a50640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102f60 0x7fe7100fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe716251640 1 -- 172.21.15.31:0/258643254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102f60 msgr2=0x7fe7100fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe716251640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102f60 0x7fe7100fe5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe716251640 1 -- 172.21.15.31:0/258643254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe700009560 con 0x7fe710100630 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe716251640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710100630 0x7fe7100fff90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00c8f0 tx=0x7fe70c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.010+0000 7fe7077fe640 1 -- 172.21.15.31:0/258643254 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70c007b80 con 0x7fe710100630 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.011+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100fef70 con 0x7fe710100630 2024-09-16T09:03:13.014 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.011+0000 7fe7077fe640 1 -- 172.21.15.31:0/258643254 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe70c0054e0 con 0x7fe710100630 2024-09-16T09:03:13.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.011+0000 7fe7077fe640 1 -- 172.21.15.31:0/258643254 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe70c00f450 con 0x7fe710100630 2024-09-16T09:03:13.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.011+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71019d6d0 con 0x7fe710100630 2024-09-16T09:03:13.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.012+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe710100a10 con 0x7fe710100630 2024-09-16T09:03:13.015 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.012+0000 7fe7077fe640 1 -- 172.21.15.31:0/258643254 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fe70c004f60 con 0x7fe710100630 2024-09-16T09:03:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.013+0000 7fe7077fe640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe6fc07af30 0x7fe6fc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.013+0000 7fe7077fe640 1 -- 172.21.15.31:0/258643254 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6682+0+0 (secure 0 0 0) 0x7fe70c09dd70 con 0x7fe710100630 2024-09-16T09:03:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.013+0000 7fe715a50640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe6fc07af30 0x7fe6fc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.013+0000 7fe715a50640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe6fc07af30 0x7fe6fc07d3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe700009880 tx=0x7fe7000314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.018+0000 7fe7077fe640 1 -- 172.21.15.31:0/258643254 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fe70c060f20 con 0x7fe710100630 2024-09-16T09:03:13.226 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.224+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7101039c0 con 0x7fe6fc07af30 2024-09-16T09:03:13.227 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.225+0000 7fe7077fe640 1 -- 172.21.15.31:0/258643254 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe7101039c0 con 0x7fe6fc07af30 2024-09-16T09:03:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe6fc07af30 msgr2=0x7fe6fc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe6fc07af30 0x7fe6fc07d3f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe700009880 tx=0x7fe7000314e0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710100630 msgr2=0x7fe7100fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710100630 0x7fe7100fff90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00c8f0 tx=0x7fe70c00cdb0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 shutdown_connections 2024-09-16T09:03:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe6fc07af30 0x7fe6fc07d3f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe710102f60 0x7fe7100fe5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 --2- 172.21.15.31:0/258643254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe710100630 0x7fe7100fff90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 >> 172.21.15.31:0/258643254 conn(0x7fe7100fa250 msgr2=0x7fe7100fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 shutdown_connections 2024-09-16T09:03:13.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.227+0000 7fe7184b5640 1 -- 172.21.15.31:0/258643254 wait complete. 2024-09-16T09:03:13.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:03:12 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:03:12.828+0000 7faf17b08640 -1 osd.4 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:03:13.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/3881314038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f8100d90 msgr2=0x7fb5f8101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/3881314038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f8100d90 0x7fb5f8101210 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009880 tx=0x7fb5e802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/3881314038 shutdown_connections 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/3881314038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f8100d90 0x7fb5f8101210 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/3881314038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f80ffb90 0x7fb5f80fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/3881314038 >> 172.21.15.31:0/3881314038 conn(0x7fb5f80fb320 msgr2=0x7fb5f80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/3881314038 shutdown_connections 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.311+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/3881314038 wait complete. 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5ffd1b640 1 Processor -- start 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5ffd1b640 1 -- start start 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5ffd1b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f80ffb90 0x7fb5f8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5ffd1b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8100d90 0x7fb5f80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5ffd1b640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8072de0 con 0x7fb5f80ffb90 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5ffd1b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f8072f50 con 0x7fb5f8100d90 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5fdab7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f80ffb90 0x7fb5f8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5fdab7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f80ffb90 0x7fb5f8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51576/0 (socket says 172.21.15.31:51576) 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5fdab7640 1 -- 172.21.15.31:0/627480322 learned_addr learned my addr 172.21.15.31:0/627480322 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.312+0000 7fb5fd2b6640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8100d90 0x7fb5f80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5fdab7640 1 -- 172.21.15.31:0/627480322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8100d90 msgr2=0x7fb5f80728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5fdab7640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8100d90 0x7fb5f80728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5fdab7640 1 -- 172.21.15.31:0/627480322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e8009560 con 0x7fb5f80ffb90 2024-09-16T09:03:13.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5fdab7640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f80ffb90 0x7fb5f8074250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b3d0 tx=0x7fb5f400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5eeffd640 1 -- 172.21.15.31:0/627480322 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f4011800 con 0x7fb5f80ffb90 2024-09-16T09:03:13.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5eeffd640 1 -- 172.21.15.31:0/627480322 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5f4011e20 con 0x7fb5f80ffb90 2024-09-16T09:03:13.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f8073230 con 0x7fb5f80ffb90 2024-09-16T09:03:13.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5eeffd640 1 -- 172.21.15.31:0/627480322 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5f400f510 con 0x7fb5f80ffb90 2024-09-16T09:03:13.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.313+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f819d790 con 0x7fb5f80ffb90 2024-09-16T09:03:13.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.314+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f8108820 con 0x7fb5f80ffb90 2024-09-16T09:03:13.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.315+0000 7fb5eeffd640 1 -- 172.21.15.31:0/627480322 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fb5f4010380 con 0x7fb5f80ffb90 2024-09-16T09:03:13.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.316+0000 7fb5eeffd640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb5e407af30 0x7fb5e407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.316+0000 7fb5eeffd640 1 -- 172.21.15.31:0/627480322 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6682+0+0 (secure 0 0 0) 0x7fb5f409dbf0 con 0x7fb5f80ffb90 2024-09-16T09:03:13.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.316+0000 7fb5fd2b6640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb5e407af30 0x7fb5e407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.316+0000 7fb5fd2b6640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb5e407af30 0x7fb5e407d3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8000c00 tx=0x7fb5e8004400 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.320+0000 7fb5eeffd640 1 -- 172.21.15.31:0/627480322 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fb5f4060cf0 con 0x7fb5f80ffb90 2024-09-16T09:03:13.520 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.517+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5f81056e0 con 0x7fb5e407af30 2024-09-16T09:03:13.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.527+0000 7fb5eeffd640 1 -- 172.21.15.31:0/627480322 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fb5f81056e0 con 0x7fb5e407af30 2024-09-16T09:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:03:13.529 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (12m) 75s ago 14m 25.3M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (5m) 75s ago 14m 8476k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (5m) 5s ago 13m 8204k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (2m) 75s ago 14m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (2m) 5s ago 13m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (12m) 75s ago 14m 85.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (8m) 75s ago 8m 16.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (8m) 75s ago 8m 153M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (8m) 5s ago 8m 26.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (8m) 5s ago 8m 18.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (5m) 75s ago 16m 516M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (4m) 5s ago 13m 454M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:03:13.530 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (3m) 75s ago 16m 48.2M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (3m) 5s ago 13m 45.3M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (14m) 75s ago 14m 15.8M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (13m) 5s ago 13m 16.9M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (2m) 75s ago 12m 93.6M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (116s) 75s ago 11m 98.7M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (80s) 75s ago 11m 13.3M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (43s) 5s ago 10m 138M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (9s) 5s ago 10m 12.0M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (10m) 5s ago 10m 311M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T09:03:13.531 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (4m) 75s ago 14m 65.2M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:03:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb5e407af30 msgr2=0x7fb5e407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb5e407af30 0x7fb5e407d3f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8000c00 tx=0x7fb5e8004400 comp rx=0 tx=0).stop 2024-09-16T09:03:13.532 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f80ffb90 msgr2=0x7fb5f8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f80ffb90 0x7fb5f8074250 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400b3d0 tx=0x7fb5f400b890 comp rx=0 tx=0).stop 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 shutdown_connections 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb5e407af30 0x7fb5e407d3f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5f8100d90 0x7fb5f80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 --2- 172.21.15.31:0/627480322 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5f80ffb90 0x7fb5f8074250 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 >> 172.21.15.31:0/627480322 conn(0x7fb5f80fb320 msgr2=0x7fb5f8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 shutdown_connections 2024-09-16T09:03:13.533 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.529+0000 7fb5ffd1b640 1 -- 172.21.15.31:0/627480322 wait complete. 2024-09-16T09:03:13.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.612+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2663742915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 msgr2=0x7fd624105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.612+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2663742915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd624105350 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0098b0 tx=0x7fd60c02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.615 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.612+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2663742915 shutdown_connections 2024-09-16T09:03:13.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.612+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2663742915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd624105350 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.612+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2663742915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd624100630 0x7fd624102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.612+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2663742915 >> 172.21.15.31:0/2663742915 conn(0x7fd6240fa250 msgr2=0x7fd6240fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:13.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.613+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2663742915 shutdown_connections 2024-09-16T09:03:13.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.613+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2663742915 wait complete. 2024-09-16T09:03:13.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.613+0000 7fd62a1c5640 1 Processor -- start 2024-09-16T09:03:13.616 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.613+0000 7fd62a1c5640 1 -- start start 2024-09-16T09:03:13.617 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd62a1c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd624100630 0x7fd6240fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd62a1c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd6240fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd62a1c5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240feb20 con 0x7fd624102f60 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd62a1c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6240fec90 con 0x7fd624100630 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd61affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd6240fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd61affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd6240fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51596/0 (socket says 172.21.15.31:51596) 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd61affd640 1 -- 172.21.15.31:0/2730349439 learned_addr learned my addr 172.21.15.31:0/2730349439 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd6237fe640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd624100630 0x7fd6240fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd61affd640 1 -- 172.21.15.31:0/2730349439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd624100630 msgr2=0x7fd6240fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd61affd640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd624100630 0x7fd6240fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd61affd640 1 -- 172.21.15.31:0/2730349439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd60c009560 con 0x7fd624102f60 2024-09-16T09:03:13.618 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.614+0000 7fd61affd640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd6240fe5e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd60c02a9f0 tx=0x7fd60c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.615+0000 7fd6217fa640 1 -- 172.21.15.31:0/2730349439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c004aa0 con 0x7fd624102f60 2024-09-16T09:03:13.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.615+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6240fef10 con 0x7fd624102f60 2024-09-16T09:03:13.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.615+0000 7fd6217fa640 1 -- 172.21.15.31:0/2730349439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd60c004c00 con 0x7fd624102f60 2024-09-16T09:03:13.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.615+0000 7fd6217fa640 1 -- 172.21.15.31:0/2730349439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd60c003a80 con 0x7fd624102f60 2024-09-16T09:03:13.619 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.615+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62419d6d0 con 0x7fd624102f60 2024-09-16T09:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.616+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd624100a10 con 0x7fd624102f60 2024-09-16T09:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.617+0000 7fd6217fa640 1 -- 172.21.15.31:0/2730349439 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fd60c003d40 con 0x7fd624102f60 2024-09-16T09:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.617+0000 7fd6217fa640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd61007af30 0x7fd61007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.617+0000 7fd6237fe640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd61007af30 0x7fd61007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.617+0000 7fd6217fa640 1 -- 172.21.15.31:0/2730349439 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6682+0+0 (secure 0 0 0) 0x7fd60c0bf1c0 con 0x7fd624102f60 2024-09-16T09:03:13.620 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.618+0000 7fd6237fe640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd61007af30 0x7fd61007d3f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd614005fd0 tx=0x7fd61400b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.622+0000 7fd6217fa640 1 -- 172.21.15.31:0/2730349439 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fd60c082370 con 0x7fd624102f60 2024-09-16T09:03:13.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:13 smithi031 ceph-mon[91884]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T09:03:13.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:13 smithi031 ceph-mon[91884]: osdmap e83: 6 total, 5 up, 6 in 2024-09-16T09:03:13.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:13 smithi031 ceph-mon[91884]: from='osd.4 [v2:172.21.15.97:6808/54702930,v1:172.21.15.97:6809/54702930]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:03:13.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:13 smithi031 ceph-mon[91884]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:03:13.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:13 smithi031 ceph-mon[91884]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.891+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd6241087b0 con 0x7fd624102f60 2024-09-16T09:03:13.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.891+0000 7fd6217fa640 1 -- 172.21.15.31:0/2730349439 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7fd60c085490 con 0x7fd624102f60 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 1, 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 5 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:03:13.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 5, 2024-09-16T09:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 9 2024-09-16T09:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:03:13.895 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:03:13.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd61007af30 msgr2=0x7fd61007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd61007af30 0x7fd61007d3f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd614005fd0 tx=0x7fd61400b390 comp rx=0 tx=0).stop 2024-09-16T09:03:13.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 msgr2=0x7fd6240fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd6240fe5e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd60c02a9f0 tx=0x7fd60c0055a0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 shutdown_connections 2024-09-16T09:03:13.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd61007af30 0x7fd61007d3f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd624102f60 0x7fd6240fe5e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 --2- 172.21.15.31:0/2730349439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd624100630 0x7fd6240fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 >> 172.21.15.31:0/2730349439 conn(0x7fd6240fa250 msgr2=0x7fd6240fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:13.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 shutdown_connections 2024-09-16T09:03:13.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.894+0000 7fd62a1c5640 1 -- 172.21.15.31:0/2730349439 wait complete. 2024-09-16T09:03:13.979 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 -- 172.21.15.31:0/248099430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 msgr2=0x7f2ce40feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/248099430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce40feea0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009880 tx=0x7f2cd402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 -- 172.21.15.31:0/248099430 shutdown_connections 2024-09-16T09:03:13.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/248099430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40ffca0 0x7f2ce4100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/248099430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce40feea0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 -- 172.21.15.31:0/248099430 >> 172.21.15.31:0/248099430 conn(0x7f2ce40fa250 msgr2=0x7f2ce40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:13.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 -- 172.21.15.31:0/248099430 shutdown_connections 2024-09-16T09:03:13.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.977+0000 7f2cea5f9640 1 -- 172.21.15.31:0/248099430 wait complete. 2024-09-16T09:03:13.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2cea5f9640 1 Processor -- start 2024-09-16T09:03:13.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2cea5f9640 1 -- start start 2024-09-16T09:03:13.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2cea5f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce4106a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2cea5f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40ffca0 0x7f2ce41050a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2cea5f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce41055e0 con 0x7f2ce40feaa0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2cea5f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce4105750 con 0x7f2ce40ffca0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce4106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.978+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce4106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51612/0 (socket says 172.21.15.31:51612) 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce3fff640 1 -- 172.21.15.31:0/3884278535 learned_addr learned my addr 172.21.15.31:0/3884278535 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce37fe640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40ffca0 0x7f2ce41050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce3fff640 1 -- 172.21.15.31:0/3884278535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40ffca0 msgr2=0x7f2ce41050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce3fff640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40ffca0 0x7f2ce41050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce3fff640 1 -- 172.21.15.31:0/3884278535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4009560 con 0x7f2ce40feaa0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce37fe640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40ffca0 0x7f2ce41050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce3fff640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce4106a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402a9c0 tx=0x7f2cd4005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce17fa640 1 -- 172.21.15.31:0/3884278535 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd4004b20 con 0x7f2ce40feaa0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce41059d0 con 0x7f2ce40feaa0 2024-09-16T09:03:13.983 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce17fa640 1 -- 172.21.15.31:0/3884278535 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cd4004c80 con 0x7f2ce40feaa0 2024-09-16T09:03:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce419d760 con 0x7f2ce40feaa0 2024-09-16T09:03:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.979+0000 7f2ce17fa640 1 -- 172.21.15.31:0/3884278535 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd4003bc0 con 0x7f2ce40feaa0 2024-09-16T09:03:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.980+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc4005280 con 0x7f2ce40feaa0 2024-09-16T09:03:13.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.981+0000 7f2ce17fa640 1 -- 172.21.15.31:0/3884278535 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f2cd4031640 con 0x7f2ce40feaa0 2024-09-16T09:03:13.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.981+0000 7f2ce17fa640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2cd007af30 0x7f2cd007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:13.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.982+0000 7f2ce17fa640 1 -- 172.21.15.31:0/3884278535 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6682+0+0 (secure 0 0 0) 0x7f2cd40bf2e0 con 0x7f2ce40feaa0 2024-09-16T09:03:13.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.982+0000 7f2ce37fe640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2cd007af30 0x7f2cd007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:13.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.982+0000 7f2ce37fe640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2cd007af30 0x7f2cd007d3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc006fd0 tx=0x7f2ccc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:13.988 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:13.986+0000 7f2ce17fa640 1 -- 172.21.15.31:0/3884278535 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f2cd40823e0 con 0x7f2ce40feaa0 2024-09-16T09:03:14.211 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.209+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2cc4005d40 con 0x7f2ce40feaa0 2024-09-16T09:03:14.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.209+0000 7f2ce17fa640 1 -- 172.21.15.31:0/3884278535 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7f2cd4085500 con 0x7f2ce40feaa0 2024-09-16T09:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:03:14.212 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:03:14.213 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:03:14.214 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:14.215 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:14.216 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:03:14.216 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2cd007af30 msgr2=0x7f2cd007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2cd007af30 0x7f2cd007d3f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc006fd0 tx=0x7f2ccc008040 comp rx=0 tx=0).stop 2024-09-16T09:03:14.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 msgr2=0x7f2ce4106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce4106a50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2cd402a9c0 tx=0x7f2cd4005610 comp rx=0 tx=0).stop 2024-09-16T09:03:14.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 shutdown_connections 2024-09-16T09:03:14.217 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2cd007af30 0x7f2cd007d3f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ce40ffca0 0x7f2ce41050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 --2- 172.21.15.31:0/3884278535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ce40feaa0 0x7f2ce4106a50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 >> 172.21.15.31:0/3884278535 conn(0x7f2ce40fa250 msgr2=0x7f2ce4102ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:14.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 shutdown_connections 2024-09-16T09:03:14.218 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.212+0000 7f2cea5f9640 1 -- 172.21.15.31:0/3884278535 wait complete. 2024-09-16T09:03:14.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:13 smithi097 ceph-mon[79892]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T09:03:14.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:13 smithi097 ceph-mon[79892]: osdmap e83: 6 total, 5 up, 6 in 2024-09-16T09:03:14.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:13 smithi097 ceph-mon[79892]: from='osd.4 [v2:172.21.15.97:6808/54702930,v1:172.21.15.97:6809/54702930]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:03:14.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:13 smithi097 ceph-mon[79892]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:03:14.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:13 smithi097 ceph-mon[79892]: from='client.34342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:14.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 -- 172.21.15.31:0/3099789978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 msgr2=0x7fa9c40fee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/3099789978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c40fee80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009880 tx=0x7fa9ac02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 -- 172.21.15.31:0/3099789978 shutdown_connections 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/3099789978 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9c40ffc80 0x7fa9c4100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/3099789978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c40fee80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 -- 172.21.15.31:0/3099789978 >> 172.21.15.31:0/3099789978 conn(0x7fa9c40fa230 msgr2=0x7fa9c40fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 -- 172.21.15.31:0/3099789978 shutdown_connections 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.297+0000 7fa9c9d16640 1 -- 172.21.15.31:0/3099789978 wait complete. 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c9d16640 1 Processor -- start 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c9d16640 1 -- start start 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c9d16640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c41978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c9d16640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9c40ffc80 0x7fa9c4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:14.302 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c9d16640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c41983d0 con 0x7fa9c40fea80 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c9d16640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c4198540 con 0x7fa9c40ffc80 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c41978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c37fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c41978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51624/0 (socket says 172.21.15.31:51624) 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.298+0000 7fa9c37fe640 1 -- 172.21.15.31:0/1614835590 learned_addr learned my addr 172.21.15.31:0/1614835590 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c37fe640 1 -- 172.21.15.31:0/1614835590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9c40ffc80 msgr2=0x7fa9c4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c2ffd640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9c40ffc80 0x7fa9c4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c37fe640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9c40ffc80 0x7fa9c4197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c37fe640 1 -- 172.21.15.31:0/1614835590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac009560 con 0x7fa9c40fea80 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c37fe640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c41978c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac02a9c0 tx=0x7fa9ac0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c0ff9640 1 -- 172.21.15.31:0/1614835590 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9ac004aa0 con 0x7fa9c40fea80 2024-09-16T09:03:14.303 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c0ff9640 1 -- 172.21.15.31:0/1614835590 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9ac004c00 con 0x7fa9c40fea80 2024-09-16T09:03:14.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c419cf80 con 0x7fa9c40fea80 2024-09-16T09:03:14.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c0ff9640 1 -- 172.21.15.31:0/1614835590 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9ac0039b0 con 0x7fa9c40fea80 2024-09-16T09:03:14.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.299+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c419d470 con 0x7fa9c40fea80 2024-09-16T09:03:14.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.300+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c4107690 con 0x7fa9c40fea80 2024-09-16T09:03:14.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.301+0000 7fa9c0ff9640 1 -- 172.21.15.31:0/1614835590 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fa9ac031c00 con 0x7fa9c40fea80 2024-09-16T09:03:14.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.301+0000 7fa9c0ff9640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b007af30 0x7fa9b007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:14.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.302+0000 7fa9c0ff9640 1 -- 172.21.15.31:0/1614835590 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6682+0+0 (secure 0 0 0) 0x7fa9ac0bf0e0 con 0x7fa9c40fea80 2024-09-16T09:03:14.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.302+0000 7fa9c2ffd640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b007af30 0x7fa9b007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:14.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.302+0000 7fa9c2ffd640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b007af30 0x7fa9b007d3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005fd0 tx=0x7fa9b4005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:14.308 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.306+0000 7fa9c0ff9640 1 -- 172.21.15.31:0/1614835590 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fa9ac082260 con 0x7fa9c40fea80 2024-09-16T09:03:14.510 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.508+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9c4068bf0 con 0x7fa9b007af30 2024-09-16T09:03:14.512 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.510+0000 7fa9c0ff9640 1 -- 172.21.15.31:0/1614835590 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fa9c4068bf0 con 0x7fa9b007af30 2024-09-16T09:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:03:14.512 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "11/23 daemons upgraded", 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:03:14.513 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:03:14.514 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b007af30 msgr2=0x7fa9b007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b007af30 0x7fa9b007d3f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005fd0 tx=0x7fa9b4005850 comp rx=0 tx=0).stop 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 msgr2=0x7fa9c41978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c41978c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac02a9c0 tx=0x7fa9ac0055a0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 shutdown_connections 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9b007af30 0x7fa9b007d3f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9c40ffc80 0x7fa9c4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 --2- 172.21.15.31:0/1614835590 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c40fea80 0x7fa9c41978c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 >> 172.21.15.31:0/1614835590 conn(0x7fa9c40fa230 msgr2=0x7fa9c40684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 shutdown_connections 2024-09-16T09:03:14.515 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.512+0000 7fa9c9d16640 1 -- 172.21.15.31:0/1614835590 wait complete. 2024-09-16T09:03:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 -- 172.21.15.31:0/3252724934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 msgr2=0x7f3c34101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 --2- 172.21.15.31:0/3252724934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c34101210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009880 tx=0x7f3c2402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 -- 172.21.15.31:0/3252724934 shutdown_connections 2024-09-16T09:03:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 --2- 172.21.15.31:0/3252724934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c34101210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.600 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 --2- 172.21.15.31:0/3252724934 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c340ffb90 0x7f3c340fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 -- 172.21.15.31:0/3252724934 >> 172.21.15.31:0/3252724934 conn(0x7f3c340fb320 msgr2=0x7f3c340fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 -- 172.21.15.31:0/3252724934 shutdown_connections 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.596+0000 7f3c3c211640 1 -- 172.21.15.31:0/3252724934 wait complete. 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c3c211640 1 Processor -- start 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c3c211640 1 -- start start 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c3c211640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c340ffb90 0x7f3c34074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c3c211640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c340728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c3c211640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c34072de0 con 0x7f3c34100d90 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c3c211640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c34072f50 con 0x7f3c340ffb90 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c397ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c340728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c397ac640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c340728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51640/0 (socket says 172.21.15.31:51640) 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c397ac640 1 -- 172.21.15.31:0/929448876 learned_addr learned my addr 172.21.15.31:0/929448876 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:14.601 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c39fad640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c340ffb90 0x7f3c34074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c397ac640 1 -- 172.21.15.31:0/929448876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c340ffb90 msgr2=0x7f3c34074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c397ac640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c340ffb90 0x7f3c34074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.597+0000 7f3c397ac640 1 -- 172.21.15.31:0/929448876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24009560 con 0x7f3c34100d90 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.598+0000 7f3c397ac640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c340728a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005610 tx=0x7f3c24005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.598+0000 7f3c2affd640 1 -- 172.21.15.31:0/929448876 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c24004ba0 con 0x7f3c34100d90 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.598+0000 7f3c2affd640 1 -- 172.21.15.31:0/929448876 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c24004d00 con 0x7f3c34100d90 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.598+0000 7f3c2affd640 1 -- 172.21.15.31:0/929448876 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c24003bd0 con 0x7f3c34100d90 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.598+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c340731d0 con 0x7f3c34100d90 2024-09-16T09:03:14.602 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.598+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3419d640 con 0x7f3c34100d90 2024-09-16T09:03:14.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.599+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c340fff90 con 0x7f3c34100d90 2024-09-16T09:03:14.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.599+0000 7f3c2affd640 1 -- 172.21.15.31:0/929448876 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f3c24031640 con 0x7f3c34100d90 2024-09-16T09:03:14.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.600+0000 7f3c2affd640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3c2007af30 0x7f3c2007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:14.603 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.600+0000 7f3c39fad640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3c2007af30 0x7f3c2007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.600+0000 7f3c2affd640 1 -- 172.21.15.31:0/929448876 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6682+0+0 (secure 0 0 0) 0x7f3c240bf3b0 con 0x7f3c34100d90 2024-09-16T09:03:14.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.601+0000 7f3c39fad640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3c2007af30 0x7f3c2007d3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c30005db0 tx=0x7f3c30005d40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:14.606 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.605+0000 7f3c2affd640 1 -- 172.21.15.31:0/929448876 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f3c240824b0 con 0x7f3c34100d90 2024-09-16T09:03:14.887 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.885+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3c3419d8d0 con 0x7f3c34100d90 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.885+0000 7f3c2affd640 1 -- 172.21.15.31:0/929448876 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+961 (secure 0 0 0) 0x7f3c240855d0 con 0x7f3c34100d90 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN Degraded data redundancy: 46/282 objects degraded (16.312%), 14 pgs degraded 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 46/282 objects degraded (16.312%), 14 pgs degraded 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-09-16T09:03:14.888 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.f is active+undersized+degraded, acting [3,5] 2024-09-16T09:03:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-09-16T09:03:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-09-16T09:03:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-09-16T09:03:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-09-16T09:03:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-09-16T09:03:14.889 INFO:teuthology.orchestra.run.smithi031.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-09-16T09:03:14.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3c2007af30 msgr2=0x7f3c2007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3c2007af30 0x7f3c2007d3f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c30005db0 tx=0x7f3c30005d40 comp rx=0 tx=0).stop 2024-09-16T09:03:14.890 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 msgr2=0x7f3c340728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c340728a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c24005610 tx=0x7f3c24005640 comp rx=0 tx=0).stop 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 shutdown_connections 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3c2007af30 0x7f3c2007d3f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c34100d90 0x7f3c340728a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 --2- 172.21.15.31:0/929448876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3c340ffb90 0x7f3c34074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 >> 172.21.15.31:0/929448876 conn(0x7f3c340fb320 msgr2=0x7f3c340fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 shutdown_connections 2024-09-16T09:03:14.891 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:14.888+0000 7f3c3c211640 1 -- 172.21.15.31:0/929448876 wait complete. 2024-09-16T09:03:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: pgmap v151: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:15.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:03:15.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: osd.4 [v2:172.21.15.97:6808/54702930,v1:172.21.15.97:6809/54702930] boot 2024-09-16T09:03:15.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: osdmap e84: 6 total, 6 up, 6 in 2024-09-16T09:03:15.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:03:15.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2730349439' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:15.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3884278535' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:03:15.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:14 smithi031 ceph-mon[91884]: osdmap e85: 6 total, 6 up, 6 in 2024-09-16T09:03:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: from='client.34346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:15.302 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: pgmap v151: 65 pgs: 18 active+undersized, 14 active+undersized+degraded, 33 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 46/282 objects degraded (16.312%) 2024-09-16T09:03:15.302 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: from='client.34350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:15.302 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:03:15.302 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: osd.4 [v2:172.21.15.97:6808/54702930,v1:172.21.15.97:6809/54702930] boot 2024-09-16T09:03:15.302 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: osdmap e84: 6 total, 6 up, 6 in 2024-09-16T09:03:15.303 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:03:15.303 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2730349439' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:15.303 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3884278535' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:03:15.303 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:14 smithi097 ceph-mon[79892]: osdmap e85: 6 total, 6 up, 6 in 2024-09-16T09:03:16.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:15 smithi031 ceph-mon[91884]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:16.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:15 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/929448876' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:03:16.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:15 smithi097 ceph-mon[79892]: from='client.34362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:16.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:15 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/929448876' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:03:17.098 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:16 smithi097 ceph-mon[79892]: pgmap v154: 65 pgs: 2 peering, 17 active+undersized, 13 active+undersized+degraded, 33 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 42/282 objects degraded (14.894%) 2024-09-16T09:03:17.098 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:16 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 42/282 objects degraded (14.894%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:16 smithi031 ceph-mon[91884]: pgmap v154: 65 pgs: 2 peering, 17 active+undersized, 13 active+undersized+degraded, 33 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 42/282 objects degraded (14.894%) 2024-09-16T09:03:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:16 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 42/282 objects degraded (14.894%), 13 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:19.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:18 smithi031 ceph-mon[91884]: pgmap v155: 65 pgs: 2 peering, 13 active+undersized, 9 active+undersized+degraded, 41 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 33/282 objects degraded (11.702%) 2024-09-16T09:03:19.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:18 smithi097 ceph-mon[79892]: pgmap v155: 65 pgs: 2 peering, 13 active+undersized, 9 active+undersized+degraded, 41 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 33/282 objects degraded (11.702%) 2024-09-16T09:03:20.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:19 smithi097 ceph-mon[79892]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/282 objects degraded (11.702%), 9 pgs degraded) 2024-09-16T09:03:20.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:19 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:03:20.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:19 smithi031 ceph-mon[91884]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 33/282 objects degraded (11.702%), 9 pgs degraded) 2024-09-16T09:03:20.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:19 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:03:21.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:20 smithi097 ceph-mon[79892]: pgmap v156: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:21.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:20 smithi031 ceph-mon[91884]: pgmap v156: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:23.175 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:22 smithi031 ceph-mon[91884]: pgmap v157: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:22 smithi097 ceph-mon[79892]: pgmap v157: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:24 smithi031 ceph-mon[91884]: pgmap v158: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.1b"} : dispatch 2024-09-16T09:03:25.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:24 smithi097 ceph-mon[79892]: pgmap v158: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:25.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:25.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:25.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.1b"} : dispatch 2024-09-16T09:03:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.1b"}]': finished 2024-09-16T09:03:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:25 smithi031 ceph-mon[91884]: osdmap e86: 6 total, 6 up, 6 in 2024-09-16T09:03:26.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd rm-pg-upmap-items", "format": "json", "pgid": "2.1b"}]': finished 2024-09-16T09:03:26.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:25 smithi097 ceph-mon[79892]: osdmap e86: 6 total, 6 up, 6 in 2024-09-16T09:03:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:26 smithi031 ceph-mon[91884]: pgmap v160: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:26 smithi031 ceph-mon[91884]: osdmap e87: 6 total, 6 up, 6 in 2024-09-16T09:03:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:26 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-16T09:03:27.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:26 smithi097 ceph-mon[79892]: pgmap v160: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:27.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:26 smithi097 ceph-mon[79892]: osdmap e87: 6 total, 6 up, 6 in 2024-09-16T09:03:27.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:26 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-16T09:03:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:27 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T09:03:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:27 smithi031 ceph-mon[91884]: Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-09-16T09:03:28.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:27 smithi031 ceph-mon[91884]: osdmap e88: 6 total, 6 up, 6 in 2024-09-16T09:03:28.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:27 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T09:03:28.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:27 smithi097 ceph-mon[79892]: Upgrade: unsafe to stop osd(s) at this time (1 PGs are or would become offline) 2024-09-16T09:03:28.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:27 smithi097 ceph-mon[79892]: osdmap e88: 6 total, 6 up, 6 in 2024-09-16T09:03:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:28 smithi031 ceph-mon[91884]: pgmap v163: 65 pgs: 1 peering, 64 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:28 smithi031 ceph-mon[91884]: osdmap e89: 6 total, 6 up, 6 in 2024-09-16T09:03:29.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:28 smithi097 ceph-mon[79892]: pgmap v163: 65 pgs: 1 peering, 64 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:29.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:28 smithi097 ceph-mon[79892]: osdmap e89: 6 total, 6 up, 6 in 2024-09-16T09:03:30.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:29 smithi031 ceph-mon[91884]: osdmap e90: 6 total, 6 up, 6 in 2024-09-16T09:03:30.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:29 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:30.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:29 smithi097 ceph-mon[79892]: osdmap e90: 6 total, 6 up, 6 in 2024-09-16T09:03:30.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:29 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:31.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:30 smithi031 ceph-mon[91884]: pgmap v166: 65 pgs: 1 peering, 64 active+clean; 201 MiB data, 934 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:31.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:30 smithi097 ceph-mon[79892]: pgmap v166: 65 pgs: 1 peering, 64 active+clean; 201 MiB data, 934 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:32 smithi031 ceph-mon[91884]: pgmap v167: 65 pgs: 65 active+clean; 201 MiB data, 934 MiB used, 536 GiB / 536 GiB avail; 746 KiB/s, 4 objects/s recovering 2024-09-16T09:03:33.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:32 smithi097 ceph-mon[79892]: pgmap v167: 65 pgs: 65 active+clean; 201 MiB data, 934 MiB used, 536 GiB / 536 GiB avail; 746 KiB/s, 4 objects/s recovering 2024-09-16T09:03:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:34 smithi031 ceph-mon[91884]: pgmap v168: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 631 KiB/s, 4 objects/s recovering 2024-09-16T09:03:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:34 smithi097 ceph-mon[79892]: pgmap v168: 65 pgs: 65 active+clean; 201 MiB data, 930 MiB used, 536 GiB / 536 GiB avail; 631 KiB/s, 4 objects/s recovering 2024-09-16T09:03:37.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:36 smithi031 ceph-mon[91884]: pgmap v169: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 512 KiB/s, 3 objects/s recovering 2024-09-16T09:03:37.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:36 smithi097 ceph-mon[79892]: pgmap v169: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 512 KiB/s, 3 objects/s recovering 2024-09-16T09:03:39.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:38 smithi031 ceph-mon[91884]: pgmap v170: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 432 KiB/s, 2 objects/s recovering 2024-09-16T09:03:39.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:39.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:38 smithi097 ceph-mon[79892]: pgmap v170: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 432 KiB/s, 2 objects/s recovering 2024-09-16T09:03:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:41.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:40 smithi031 ceph-mon[91884]: pgmap v171: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 391 KiB/s, 2 objects/s recovering 2024-09-16T09:03:41.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:40 smithi097 ceph-mon[79892]: pgmap v171: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 391 KiB/s, 2 objects/s recovering 2024-09-16T09:03:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-16T09:03:42.167 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16} : dispatch 2024-09-16T09:03:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:42 smithi031 ceph-mon[91884]: pgmap v172: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s, 2 objects/s recovering 2024-09-16T09:03:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:42 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T09:03:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:42 smithi031 ceph-mon[91884]: Upgrade: osd.5 is safe to restart 2024-09-16T09:03:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:42 smithi097 ceph-mon[79892]: pgmap v172: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s, 2 objects/s recovering 2024-09-16T09:03:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:42 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-16T09:03:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:42 smithi097 ceph-mon[79892]: Upgrade: osd.5 is safe to restart 2024-09-16T09:03:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:43 smithi097 ceph-mon[79892]: Upgrade: Updating osd.5 2024-09-16T09:03:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:43 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:43 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T09:03:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:43 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:03:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:43 smithi097 ceph-mon[79892]: Deploying daemon osd.5 on smithi097 2024-09-16T09:03:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:43 smithi031 ceph-mon[91884]: Upgrade: Updating osd.5 2024-09-16T09:03:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:43 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:43 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T09:03:44.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:43 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:03:44.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:43 smithi031 ceph-mon[91884]: Deploying daemon osd.5 on smithi097 2024-09-16T09:03:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 -- 172.21.15.31:0/1509735583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 msgr2=0x7f61dc102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:44.989 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 --2- 172.21.15.31:0/1509735583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc102a70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0098b0 tx=0x7f61cc02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 -- 172.21.15.31:0/1509735583 shutdown_connections 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 --2- 172.21.15.31:0/1509735583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61dc102fb0 0x7f61dc1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 --2- 172.21.15.31:0/1509735583 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc102a70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 -- 172.21.15.31:0/1509735583 >> 172.21.15.31:0/1509735583 conn(0x7f61dc0fa280 msgr2=0x7f61dc0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 -- 172.21.15.31:0/1509735583 shutdown_connections 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.987+0000 7f61e3111640 1 -- 172.21.15.31:0/1509735583 wait complete. 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e3111640 1 Processor -- start 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e3111640 1 -- start start 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e3111640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e3111640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61dc102fb0 0x7f61dc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e3111640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc198430 con 0x7f61dc100680 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e3111640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc1985a0 con 0x7f61dc102fb0 2024-09-16T09:03:44.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e0ead640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:44.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e0ead640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc197920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43814/0 (socket says 172.21.15.31:43814) 2024-09-16T09:03:44.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e0ead640 1 -- 172.21.15.31:0/966557346 learned_addr learned my addr 172.21.15.31:0/966557346 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:44.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61dbfff640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61dc102fb0 0x7f61dc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:44.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e0ead640 1 -- 172.21.15.31:0/966557346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61dc102fb0 msgr2=0x7f61dc197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:44.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e0ead640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61dc102fb0 0x7f61dc197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:44.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.988+0000 7f61e0ead640 1 -- 172.21.15.31:0/966557346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc009560 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.989+0000 7f61e0ead640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc197920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f61cc004650 tx=0x7f61cc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.989+0000 7f61d9ffb640 1 -- 172.21.15.31:0/966557346 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61cc004aa0 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.989+0000 7f61d9ffb640 1 -- 172.21.15.31:0/966557346 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61cc005d50 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.989+0000 7f61d9ffb640 1 -- 172.21.15.31:0/966557346 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61cc0039e0 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.989+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61dc19cfe0 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.989+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc19d450 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.990+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc108830 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.991+0000 7f61d9ffb640 1 -- 172.21.15.31:0/966557346 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f61cc031c00 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.992+0000 7f61d9ffb640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f61c807af30 0x7f61c807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.992+0000 7f61d9ffb640 1 -- 172.21.15.31:0/966557346 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f61cc0bf0e0 con 0x7f61dc100680 2024-09-16T09:03:44.993 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.992+0000 7f61dbfff640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f61c807af30 0x7f61c807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:44.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.992+0000 7f61dbfff640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f61c807af30 0x7f61c807d3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61dc198e40 tx=0x7f61c4005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:44.997 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:44.996+0000 7f61d9ffb640 1 -- 172.21.15.31:0/966557346 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f61cc082250 con 0x7f61dc100680 2024-09-16T09:03:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.200+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61dc19d670 con 0x7f61c807af30 2024-09-16T09:03:45.201 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:44 smithi031 ceph-mon[91884]: pgmap v173: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:45.202 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.201+0000 7f61d9ffb640 1 -- 172.21.15.31:0/966557346 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f61dc19d670 con 0x7f61c807af30 2024-09-16T09:03:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f61c807af30 msgr2=0x7f61c807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f61c807af30 0x7f61c807d3f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f61dc198e40 tx=0x7f61c4005f90 comp rx=0 tx=0).stop 2024-09-16T09:03:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 msgr2=0x7f61dc197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc197920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f61cc004650 tx=0x7f61cc0055a0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 shutdown_connections 2024-09-16T09:03:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f61c807af30 0x7f61c807d3f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61dc102fb0 0x7f61dc197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 --2- 172.21.15.31:0/966557346 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61dc100680 0x7f61dc197920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 >> 172.21.15.31:0/966557346 conn(0x7f61dc0fa280 msgr2=0x7f61dc0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 shutdown_connections 2024-09-16T09:03:45.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.204+0000 7f61e3111640 1 -- 172.21.15.31:0/966557346 wait complete. 2024-09-16T09:03:45.217 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:03:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:44 smithi097 ceph-mon[79892]: pgmap v173: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 -- 172.21.15.31:0/471435547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 msgr2=0x7fe5881053b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 --2- 172.21.15.31:0/471435547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5881053b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009880 tx=0x7fe57c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 -- 172.21.15.31:0/471435547 shutdown_connections 2024-09-16T09:03:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 --2- 172.21.15.31:0/471435547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5881053b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 --2- 172.21.15.31:0/471435547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe588100690 0x7fe588102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 -- 172.21.15.31:0/471435547 >> 172.21.15.31:0/471435547 conn(0x7fe5880fa250 msgr2=0x7fe5880fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 -- 172.21.15.31:0/471435547 shutdown_connections 2024-09-16T09:03:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.289+0000 7fe58dbea640 1 -- 172.21.15.31:0/471435547 wait complete. 2024-09-16T09:03:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe58dbea640 1 Processor -- start 2024-09-16T09:03:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe58dbea640 1 -- start start 2024-09-16T09:03:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe58dbea640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe588100690 0x7fe5880fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe58dbea640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5880fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe58dbea640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880feb80 con 0x7fe588102fc0 2024-09-16T09:03:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe58dbea640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880fecf0 con 0x7fe588100690 2024-09-16T09:03:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe586ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5880fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe586ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5880fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43840/0 (socket says 172.21.15.31:43840) 2024-09-16T09:03:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe586ffd640 1 -- 172.21.15.31:0/4024204152 learned_addr learned my addr 172.21.15.31:0/4024204152 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.290+0000 7fe5877fe640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe588100690 0x7fe5880fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe586ffd640 1 -- 172.21.15.31:0/4024204152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe588100690 msgr2=0x7fe5880fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe586ffd640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe588100690 0x7fe5880fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe586ffd640 1 -- 172.21.15.31:0/4024204152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009560 con 0x7fe588102fc0 2024-09-16T09:03:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe586ffd640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5880fe640 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe57c02a9c0 tx=0x7fe57c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe584ff9640 1 -- 172.21.15.31:0/4024204152 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57c004aa0 con 0x7fe588102fc0 2024-09-16T09:03:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe584ff9640 1 -- 172.21.15.31:0/4024204152 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe57c005d50 con 0x7fe588102fc0 2024-09-16T09:03:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5880fefd0 con 0x7fe588102fc0 2024-09-16T09:03:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe584ff9640 1 -- 172.21.15.31:0/4024204152 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57c0039b0 con 0x7fe588102fc0 2024-09-16T09:03:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.291+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58819d690 con 0x7fe588102fc0 2024-09-16T09:03:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.292+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588108740 con 0x7fe588102fc0 2024-09-16T09:03:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.293+0000 7fe584ff9640 1 -- 172.21.15.31:0/4024204152 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fe57c003c70 con 0x7fe588102fc0 2024-09-16T09:03:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.293+0000 7fe584ff9640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe57007af30 0x7fe57007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.293+0000 7fe584ff9640 1 -- 172.21.15.31:0/4024204152 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fe57c0bf030 con 0x7fe588102fc0 2024-09-16T09:03:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.293+0000 7fe5877fe640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe57007af30 0x7fe57007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.294+0000 7fe5877fe640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe57007af30 0x7fe57007d3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe578009bd0 tx=0x7fe578005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.298+0000 7fe584ff9640 1 -- 172.21.15.31:0/4024204152 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fe57c082200 con 0x7fe588102fc0 2024-09-16T09:03:45.501 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.500+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe588103a00 con 0x7fe57007af30 2024-09-16T09:03:45.502 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.501+0000 7fe584ff9640 1 -- 172.21.15.31:0/4024204152 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe588103a00 con 0x7fe57007af30 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.503+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe57007af30 msgr2=0x7fe57007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe57007af30 0x7fe57007d3f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe578009bd0 tx=0x7fe578005f90 comp rx=0 tx=0).stop 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 msgr2=0x7fe5880fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5880fe640 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe57c02a9c0 tx=0x7fe57c0055a0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 shutdown_connections 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fe57007af30 0x7fe57007d3f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe588102fc0 0x7fe5880fe640 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 --2- 172.21.15.31:0/4024204152 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe588100690 0x7fe5880fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 >> 172.21.15.31:0/4024204152 conn(0x7fe5880fa250 msgr2=0x7fe5880fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 shutdown_connections 2024-09-16T09:03:45.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.504+0000 7fe58dbea640 1 -- 172.21.15.31:0/4024204152 wait complete. 2024-09-16T09:03:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 -- 172.21.15.31:0/2780224679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000fea20 msgr2=0x7f12000fee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2780224679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000fea20 0x7f12000fee20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009880 tx=0x7f11f002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 -- 172.21.15.31:0/2780224679 shutdown_connections 2024-09-16T09:03:45.589 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2780224679 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12000ffc20 0x7f12001000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2780224679 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000fea20 0x7f12000fee20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 -- 172.21.15.31:0/2780224679 >> 172.21.15.31:0/2780224679 conn(0x7f12000fa1f0 msgr2=0x7f12000fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 -- 172.21.15.31:0/2780224679 shutdown_connections 2024-09-16T09:03:45.590 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.588+0000 7f1204fa6640 1 -- 172.21.15.31:0/2780224679 wait complete. 2024-09-16T09:03:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f1204fa6640 1 Processor -- start 2024-09-16T09:03:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f1204fa6640 1 -- start start 2024-09-16T09:03:45.591 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f1204fa6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12000fea20 0x7f12001955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f1204fa6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000ffc20 0x7f1200195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f1204fa6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200196100 con 0x7f12000ffc20 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f1204fa6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200196270 con 0x7f12000fea20 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000ffc20 0x7f1200195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fdd9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000ffc20 0x7f1200195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43846/0 (socket says 172.21.15.31:43846) 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fdd9b640 1 -- 172.21.15.31:0/2278782033 learned_addr learned my addr 172.21.15.31:0/2278782033 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fe59c640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12000fea20 0x7f12001955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fdd9b640 1 -- 172.21.15.31:0/2278782033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12000fea20 msgr2=0x7f12001955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fdd9b640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12000fea20 0x7f12001955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fdd9b640 1 -- 172.21.15.31:0/2278782033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0009560 con 0x7f12000ffc20 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.589+0000 7f11fe59c640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12000fea20 0x7f12001955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:03:45.592 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.590+0000 7f11fdd9b640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000ffc20 0x7f1200195b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d900 tx=0x7f11f400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.590+0000 7f11ef7fe640 1 -- 172.21.15.31:0/2278782033 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f4009c00 con 0x7f12000ffc20 2024-09-16T09:03:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.590+0000 7f11ef7fe640 1 -- 172.21.15.31:0/2278782033 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11f4009d60 con 0x7f12000ffc20 2024-09-16T09:03:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.590+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120019acc0 con 0x7f12000ffc20 2024-09-16T09:03:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.590+0000 7f11ef7fe640 1 -- 172.21.15.31:0/2278782033 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11f40049d0 con 0x7f12000ffc20 2024-09-16T09:03:45.593 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.590+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120019b210 con 0x7f12000ffc20 2024-09-16T09:03:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.591+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1200107610 con 0x7f12000ffc20 2024-09-16T09:03:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.592+0000 7f11ef7fe640 1 -- 172.21.15.31:0/2278782033 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f11f4004020 con 0x7f12000ffc20 2024-09-16T09:03:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.592+0000 7f11ef7fe640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f11e807af30 0x7f11e807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.592+0000 7f11ef7fe640 1 -- 172.21.15.31:0/2278782033 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f11f409dd60 con 0x7f12000ffc20 2024-09-16T09:03:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.592+0000 7f11fe59c640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f11e807af30 0x7f11e807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.594 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.593+0000 7f11fe59c640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f11e807af30 0x7f11e807d3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009230 tx=0x7f11f00314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:45.597 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.597+0000 7f11ef7fe640 1 -- 172.21.15.31:0/2278782033 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f11f4060e80 con 0x7f12000ffc20 2024-09-16T09:03:45.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.794+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f120019b460 con 0x7f11e807af30 2024-09-16T09:03:45.806 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.805+0000 7f11ef7fe640 1 -- 172.21.15.31:0/2278782033 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f120019b460 con 0x7f11e807af30 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (13m) 107s ago 15m 25.3M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (6m) 107s ago 15m 8476k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (5m) 38s ago 14m 8204k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (3m) 107s ago 15m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (3m) 38s ago 14m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (13m) 107s ago 14m 85.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (9m) 107s ago 9m 16.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (8m) 107s ago 8m 153M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (9m) 38s ago 9m 26.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (8m) 38s ago 8m 18.5M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:03:45.807 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (5m) 107s ago 16m 516M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (4m) 38s ago 14m 454M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (4m) 107s ago 16m 48.2M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (4m) 38s ago 14m 45.3M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (15m) 107s ago 15m 15.8M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (14m) 38s ago 14m 16.9M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 107s ago 12m 93.6M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (2m) 107s ago 12m 98.7M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (113s) 107s ago 11m 13.3M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (76s) 38s ago 11m 138M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (41s) 38s ago 11m 12.0M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (10m) 38s ago 10m 311M 2240M 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a40b1bdcc426 2024-09-16T09:03:45.808 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (4m) 107s ago 14m 65.2M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:03:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f11e807af30 msgr2=0x7f11e807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f11e807af30 0x7f11e807d3f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f11f0009230 tx=0x7f11f00314e0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000ffc20 msgr2=0x7f1200195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000ffc20 0x7f1200195b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f11f400d900 tx=0x7f11f400ddc0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 shutdown_connections 2024-09-16T09:03:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f11e807af30 0x7f11e807d3f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12000ffc20 0x7f1200195b30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 --2- 172.21.15.31:0/2278782033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f12000fea20 0x7f12001955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 >> 172.21.15.31:0/2278782033 conn(0x7f12000fa1f0 msgr2=0x7f1200068490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 shutdown_connections 2024-09-16T09:03:45.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.808+0000 7f1204fa6640 1 -- 172.21.15.31:0/2278782033 wait complete. 2024-09-16T09:03:45.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2943186117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e8100d20 msgr2=0x7f81e81011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2943186117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e8100d20 0x7f81e81011a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009880 tx=0x7f81dc02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2943186117 shutdown_connections 2024-09-16T09:03:45.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2943186117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e8100d20 0x7f81e81011a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2943186117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e80ffb20 0x7f81e80fff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2943186117 >> 172.21.15.31:0/2943186117 conn(0x7f81e80fb2f0 msgr2=0x7f81e80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:45.893 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2943186117 shutdown_connections 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2943186117 wait complete. 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 Processor -- start 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.891+0000 7f81ee2cf640 1 -- start start 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81ee2cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e80ffb20 0x7f81e8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81ee2cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8100d20 0x7f81e80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81ee2cf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8072de0 con 0x7f81e80ffb20 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81ee2cf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8072f50 con 0x7f81e8100d20 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e80ffb20 0x7f81e8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e7fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e80ffb20 0x7f81e8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43864/0 (socket says 172.21.15.31:43864) 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e7fff640 1 -- 172.21.15.31:0/2037785452 learned_addr learned my addr 172.21.15.31:0/2037785452 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e77fe640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8100d20 0x7f81e80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e7fff640 1 -- 172.21.15.31:0/2037785452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8100d20 msgr2=0x7f81e80728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e7fff640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8100d20 0x7f81e80728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:45.894 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e7fff640 1 -- 172.21.15.31:0/2037785452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc009560 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.892+0000 7f81e7fff640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e80ffb20 0x7f81e8074250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f81d800d8d0 tx=0x7f81d800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.893+0000 7f81e57fa640 1 -- 172.21.15.31:0/2037785452 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d800bea0 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.893+0000 7f81e57fa640 1 -- 172.21.15.31:0/2037785452 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81d8009c00 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.893+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e8073230 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.893+0000 7f81e57fa640 1 -- 172.21.15.31:0/2037785452 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d800f790 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.893+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e819d5d0 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.894+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e8109070 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.894+0000 7f81e57fa640 1 -- 172.21.15.31:0/2037785452 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f81d800fa50 con 0x7f81e80ffb20 2024-09-16T09:03:45.896 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.895+0000 7f81e57fa640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f81d007af30 0x7f81d007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:45.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.895+0000 7f81e57fa640 1 -- 172.21.15.31:0/2037785452 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f81d809df30 con 0x7f81e80ffb20 2024-09-16T09:03:45.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.895+0000 7f81e77fe640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f81d007af30 0x7f81d007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:45.897 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.895+0000 7f81e77fe640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f81d007af30 0x7f81d007d3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f81dc000c00 tx=0x7f81dc0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:45.900 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:45.899+0000 7f81e57fa640 1 -- 172.21.15.31:0/2037785452 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f81d80d5190 con 0x7f81e80ffb20 2024-09-16T09:03:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.157+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81e818f110 con 0x7f81e80ffb20 2024-09-16T09:03:46.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.157+0000 7f81e57fa640 1 -- 172.21.15.31:0/2037785452 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7f81d8061050 con 0x7f81e80ffb20 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 1, 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 5 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:03:46.159 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:03:46.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:03:46.160 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:03:46.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 5, 2024-09-16T09:03:46.160 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 9 2024-09-16T09:03:46.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:03:46.160 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f81d007af30 msgr2=0x7f81d007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f81d007af30 0x7f81d007d3f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f81dc000c00 tx=0x7f81dc0314e0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e80ffb20 msgr2=0x7f81e8074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e80ffb20 0x7f81e8074250 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f81d800d8d0 tx=0x7f81d800dd90 comp rx=0 tx=0).stop 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 shutdown_connections 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f81d007af30 0x7f81d007d3f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81e8100d20 0x7f81e80728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 --2- 172.21.15.31:0/2037785452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f81e80ffb20 0x7f81e8074250 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.161 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 >> 172.21.15.31:0/2037785452 conn(0x7f81e80fb2f0 msgr2=0x7f81e81043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:46.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 shutdown_connections 2024-09-16T09:03:46.162 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.160+0000 7f81ee2cf640 1 -- 172.21.15.31:0/2037785452 wait complete. 2024-09-16T09:03:46.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 -- 172.21.15.31:0/2383245561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae640fde70 msgr2=0x7fae640fe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 --2- 172.21.15.31:0/2383245561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae640fde70 0x7fae640fe270 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fae54009880 tx=0x7fae5402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 -- 172.21.15.31:0/2383245561 shutdown_connections 2024-09-16T09:03:46.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 --2- 172.21.15.31:0/2383245561 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae64100ef0 0x7fae64101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 --2- 172.21.15.31:0/2383245561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae640fde70 0x7fae640fe270 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.246 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 -- 172.21.15.31:0/2383245561 >> 172.21.15.31:0/2383245561 conn(0x7fae640f99c0 msgr2=0x7fae640fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 -- 172.21.15.31:0/2383245561 shutdown_connections 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.243+0000 7fae69f8d640 1 -- 172.21.15.31:0/2383245561 wait complete. 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.244+0000 7fae69f8d640 1 Processor -- start 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.244+0000 7fae69f8d640 1 -- start start 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae69f8d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae640fde70 0x7fae64100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae69f8d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae64100ef0 0x7fae640feea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae69f8d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae640ff3e0 con 0x7fae64100ef0 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae69f8d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae640ff550 con 0x7fae640fde70 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae64100ef0 0x7fae640feea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae62ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae64100ef0 0x7fae640feea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43894/0 (socket says 172.21.15.31:43894) 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae62ffd640 1 -- 172.21.15.31:0/3473075627 learned_addr learned my addr 172.21.15.31:0/3473075627 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:46.247 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae637fe640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae640fde70 0x7fae64100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae62ffd640 1 -- 172.21.15.31:0/3473075627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae640fde70 msgr2=0x7fae64100850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae62ffd640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae640fde70 0x7fae64100850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae62ffd640 1 -- 172.21.15.31:0/3473075627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54009560 con 0x7fae64100ef0 2024-09-16T09:03:46.248 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae62ffd640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae64100ef0 0x7fae640feea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fae5800e970 tx=0x7fae5800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae60ff9640 1 -- 172.21.15.31:0/3473075627 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae5800cc90 con 0x7fae64100ef0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae640ff830 con 0x7fae64100ef0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae60ff9640 1 -- 172.21.15.31:0/3473075627 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae5800cdf0 con 0x7fae64100ef0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.245+0000 7fae60ff9640 1 -- 172.21.15.31:0/3473075627 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae58010430 con 0x7fae64100ef0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.246+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae6419d6a0 con 0x7fae64100ef0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.246+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae640fe270 con 0x7fae64100ef0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.247+0000 7fae60ff9640 1 -- 172.21.15.31:0/3473075627 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fae580046c0 con 0x7fae64100ef0 2024-09-16T09:03:46.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.248+0000 7fae60ff9640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae4c07af30 0x7fae4c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.248+0000 7fae60ff9640 1 -- 172.21.15.31:0/3473075627 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fae58014070 con 0x7fae64100ef0 2024-09-16T09:03:46.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.248+0000 7fae637fe640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae4c07af30 0x7fae4c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.250 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.248+0000 7fae637fe640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae4c07af30 0x7fae4c07d3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae54009230 tx=0x7fae540314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:46.253 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.252+0000 7fae60ff9640 1 -- 172.21.15.31:0/3473075627 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fae580616c0 con 0x7fae64100ef0 2024-09-16T09:03:46.262 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:46 smithi097 systemd[1]: Stopping Ceph osd.5 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:03:46.475 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.474+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fae6418f730 con 0x7fae64100ef0 2024-09-16T09:03:46.476 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.475+0000 7fae60ff9640 1 -- 172.21.15.31:0/3473075627 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 17 v17) v1 ==== 76+0+1918 (secure 0 0 0) 0x7fae580647e0 con 0x7fae64100ef0 2024-09-16T09:03:46.476 INFO:teuthology.orchestra.run.smithi031.stdout:e17 2024-09-16T09:03:46.476 INFO:teuthology.orchestra.run.smithi031.stdout:btime 1970-01-01T00:00:00:000000+0000 2024-09-16T09:03:46.476 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:03:46.476 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:03:46.476 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T08:54:56.899545+0000 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:03:46.477 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:03:46.478 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:46.479 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 17 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae4c07af30 msgr2=0x7fae4c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae4c07af30 0x7fae4c07d3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fae54009230 tx=0x7fae540314e0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae64100ef0 msgr2=0x7fae640feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae64100ef0 0x7fae640feea0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fae5800e970 tx=0x7fae5800ed70 comp rx=0 tx=0).stop 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 shutdown_connections 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae4c07af30 0x7fae4c07d3f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.480 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae64100ef0 0x7fae640feea0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 --2- 172.21.15.31:0/3473075627 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae640fde70 0x7fae64100850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 >> 172.21.15.31:0/3473075627 conn(0x7fae640f99c0 msgr2=0x7fae641045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 shutdown_connections 2024-09-16T09:03:46.481 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.478+0000 7fae69f8d640 1 -- 172.21.15.31:0/3473075627 wait complete. 2024-09-16T09:03:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1193905512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 msgr2=0x7fd6ac101570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1193905512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac101570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd6a8009880 tx=0x7fd6a802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1193905512 shutdown_connections 2024-09-16T09:03:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1193905512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ac101b40 0x7fd6ac0fdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1193905512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac101570 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1193905512 >> 172.21.15.31:0/1193905512 conn(0x7fd6ac0f99c0 msgr2=0x7fd6ac0fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1193905512 shutdown_connections 2024-09-16T09:03:46.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.562+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1193905512 wait complete. 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b38b8640 1 Processor -- start 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b38b8640 1 -- start start 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b38b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac100870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b38b8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ac101b40 0x7fd6ac0feec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b38b8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0ff400 con 0x7fd6ac101170 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b38b8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0ff570 con 0x7fd6ac101b40 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b1654640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac100870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b1654640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac100870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43920/0 (socket says 172.21.15.31:43920) 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b1654640 1 -- 172.21.15.31:0/1570038779 learned_addr learned my addr 172.21.15.31:0/1570038779 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.563+0000 7fd6b0e53640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ac101b40 0x7fd6ac0feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.566 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6b1654640 1 -- 172.21.15.31:0/1570038779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ac101b40 msgr2=0x7fd6ac0feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6b1654640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ac101b40 0x7fd6ac0feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6b1654640 1 -- 172.21.15.31:0/1570038779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a8009560 con 0x7fd6ac101170 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6b1654640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac100870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6a802a9c0 tx=0x7fd6a80055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6a27fc640 1 -- 172.21.15.31:0/1570038779 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a8004aa0 con 0x7fd6ac101170 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6a27fc640 1 -- 172.21.15.31:0/1570038779 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6a8005d50 con 0x7fd6ac101170 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6a27fc640 1 -- 172.21.15.31:0/1570038779 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6a80039e0 con 0x7fd6ac101170 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac0ff7f0 con 0x7fd6ac101170 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.564+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac19d6e0 con 0x7fd6ac101170 2024-09-16T09:03:46.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.565+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac101570 con 0x7fd6ac101170 2024-09-16T09:03:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.566+0000 7fd6a27fc640 1 -- 172.21.15.31:0/1570038779 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fd6a8031c00 con 0x7fd6ac101170 2024-09-16T09:03:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.566+0000 7fd6a27fc640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd69807af30 0x7fd69807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.567+0000 7fd6b0e53640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd69807af30 0x7fd69807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.567+0000 7fd6a27fc640 1 -- 172.21.15.31:0/1570038779 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fd6a80bf0e0 con 0x7fd6ac101170 2024-09-16T09:03:46.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.567+0000 7fd6b0e53640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd69807af30 0x7fd69807d3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac1005f0 tx=0x7fd69c005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:46.572 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.571+0000 7fd6a27fc640 1 -- 172.21.15.31:0/1570038779 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fd6a8082210 con 0x7fd6ac101170 2024-09-16T09:03:46.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[54831]: 2024-09-16T09:03:46.305+0000 7f02e5888640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:03:46.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[54831]: 2024-09-16T09:03:46.305+0000 7f02e5888640 -1 osd.5 90 *** Got signal Terminated *** 2024-09-16T09:03:46.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[54831]: 2024-09-16T09:03:46.305+0000 7f02e5888640 -1 osd.5 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:03:46.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.776+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6ac105860 con 0x7fd69807af30 2024-09-16T09:03:46.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.777+0000 7fd6a27fc640 1 -- 172.21.15.31:0/1570038779 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd6ac105860 con 0x7fd69807af30 2024-09-16T09:03:46.778 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:03:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:03:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:03:46.778 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr" 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "11/23 daemons upgraded", 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:03:46.779 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:03:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.779+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd69807af30 msgr2=0x7fd69807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.779+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd69807af30 0x7fd69807d3f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac1005f0 tx=0x7fd69c005f90 comp rx=0 tx=0).stop 2024-09-16T09:03:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.779+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 msgr2=0x7fd6ac100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.779+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac100870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd6a802a9c0 tx=0x7fd6a80055a0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.780+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 shutdown_connections 2024-09-16T09:03:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.780+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd69807af30 0x7fd69807d3f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.780+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6ac101b40 0x7fd6ac0feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.780+0000 7fd6b38b8640 1 --2- 172.21.15.31:0/1570038779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6ac101170 0x7fd6ac100870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.780+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 >> 172.21.15.31:0/1570038779 conn(0x7fd6ac0f99c0 msgr2=0x7fd6ac104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.780+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 shutdown_connections 2024-09-16T09:03:46.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.780+0000 7fd6b38b8640 1 -- 172.21.15.31:0/1570038779 wait complete. 2024-09-16T09:03:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 -- 172.21.15.31:0/1426408971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 msgr2=0x7f623c105350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 --2- 172.21.15.31:0/1426408971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c105350 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f62380098b0 tx=0x7f623802a4d0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 -- 172.21.15.31:0/1426408971 shutdown_connections 2024-09-16T09:03:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 --2- 172.21.15.31:0/1426408971 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c105350 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 --2- 172.21.15.31:0/1426408971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f623c100630 0x7f623c102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 -- 172.21.15.31:0/1426408971 >> 172.21.15.31:0/1426408971 conn(0x7f623c0fa250 msgr2=0x7f623c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:46.867 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 -- 172.21.15.31:0/1426408971 shutdown_connections 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.865+0000 7f6244503640 1 -- 172.21.15.31:0/1426408971 wait complete. 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6244503640 1 Processor -- start 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6244503640 1 -- start start 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6244503640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f623c100630 0x7f623c0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6244503640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c0fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6244503640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0feb20 con 0x7f623c102f60 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6244503640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0fec90 con 0x7f623c100630 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6241a9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c0fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6241a9e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c0fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:43926/0 (socket says 172.21.15.31:43926) 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6241a9e640 1 -- 172.21.15.31:0/3823081339 learned_addr learned my addr 172.21.15.31:0/3823081339 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:03:46.868 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f624229f640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f623c100630 0x7f623c0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.869 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6241a9e640 1 -- 172.21.15.31:0/3823081339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f623c100630 msgr2=0x7f623c0fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6241a9e640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f623c100630 0x7f623c0fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f6241a9e640 1 -- 172.21.15.31:0/3823081339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6238009560 con 0x7f623c102f60 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.866+0000 7f624229f640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f623c100630 0x7f623c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.867+0000 7f6241a9e640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c0fe5e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f623802a9f0 tx=0x7f62380054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.867+0000 7f622f7fe640 1 -- 172.21.15.31:0/3823081339 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6238004aa0 con 0x7f623c102f60 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.867+0000 7f622f7fe640 1 -- 172.21.15.31:0/3823081339 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6238004c00 con 0x7f623c102f60 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.867+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c0fef10 con 0x7f623c102f60 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.867+0000 7f622f7fe640 1 -- 172.21.15.31:0/3823081339 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6238003a60 con 0x7f623c102f60 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.867+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c19d6d0 con 0x7f623c102f60 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.868+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c1087b0 con 0x7f623c102f60 2024-09-16T09:03:46.870 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.869+0000 7f622f7fe640 1 -- 172.21.15.31:0/3823081339 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f6238003cd0 con 0x7f623c102f60 2024-09-16T09:03:46.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.869+0000 7f622f7fe640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622807af30 0x7f622807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:03:46.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.869+0000 7f622f7fe640 1 -- 172.21.15.31:0/3823081339 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f62380bf1d0 con 0x7f623c102f60 2024-09-16T09:03:46.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.869+0000 7f624229f640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622807af30 0x7f622807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:03:46.871 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.870+0000 7f624229f640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622807af30 0x7f622807d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62300058c0 tx=0x7f6230005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:03:46.874 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:46.874+0000 7f622f7fe640 1 -- 172.21.15.31:0/3823081339 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6238082370 con 0x7f623c102f60 2024-09-16T09:03:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:46 smithi031 ceph-mon[91884]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:46 smithi031 ceph-mon[91884]: pgmap v174: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:46 smithi031 ceph-mon[91884]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:46 smithi031 ceph-mon[91884]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:46 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2037785452' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:47.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:46 smithi031 ceph-mon[91884]: osd.5 marked itself down and dead 2024-09-16T09:03:47.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:46 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3473075627' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:03:47.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.138+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f623c19d930 con 0x7f623c102f60 2024-09-16T09:03:47.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.139+0000 7f622f7fe640 1 -- 172.21.15.31:0/3823081339 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+337 (secure 0 0 0) 0x7f6238085490 con 0x7f623c102f60 2024-09-16T09:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 osds down; all OSDs are running squid or later but require_osd_release < squid 2024-09-16T09:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-16T09:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: osd.5 (root=default,host=smithi097) is down 2024-09-16T09:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-09-16T09:03:47.140 INFO:teuthology.orchestra.run.smithi031.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622807af30 msgr2=0x7f622807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622807af30 0x7f622807d3f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f62300058c0 tx=0x7f6230005850 comp rx=0 tx=0).stop 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 msgr2=0x7f623c0fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c0fe5e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f623802a9f0 tx=0x7f62380054f0 comp rx=0 tx=0).stop 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 shutdown_connections 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622807af30 0x7f622807d3f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f623c102f60 0x7f623c0fe5e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 --2- 172.21.15.31:0/3823081339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f623c100630 0x7f623c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:03:47.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 >> 172.21.15.31:0/3823081339 conn(0x7f623c0fa250 msgr2=0x7f623c0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:03:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 shutdown_connections 2024-09-16T09:03:47.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:03:47.141+0000 7f6244503640 1 -- 172.21.15.31:0/3823081339 wait complete. 2024-09-16T09:03:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-mon[79892]: from='client.34370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-mon[79892]: pgmap v174: 65 pgs: 65 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-mon[79892]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-mon[79892]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2037785452' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:03:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-mon[79892]: osd.5 marked itself down and dead 2024-09-16T09:03:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:46 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3473075627' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:03:47.598 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:47 smithi097 podman[98451]: 2024-09-16 09:03:47.337670236 +0000 UTC m=+1.175980223 container died a40b1bdcc42650f7818aa10402058b661784d214a6ce14568f71a5bfd0a705fa (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-16T09:03:47.937 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:47 smithi097 podman[98451]: 2024-09-16 09:03:47.780891673 +0000 UTC m=+1.619201636 container cleanup a40b1bdcc42650f7818aa10402058b661784d214a6ce14568f71a5bfd0a705fa (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=reef-ae7f373, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:03:47.937 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:47 smithi097 bash[98451]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5 2024-09-16T09:03:48.261 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:48 smithi097 podman[98470]: 2024-09-16 09:03:48.09898102 +0000 UTC m=+0.763377389 container remove a40b1bdcc42650f7818aa10402058b661784d214a6ce14568f71a5bfd0a705fa (image=quay.ceph.io/ceph-ci/ceph@sha256:b3b6674281196ab754723b03bfca741560573b75b455f42d9498f03e14e89d87, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=reef-ae7f373, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-16T09:03:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:47 smithi097 ceph-mon[79892]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:47 smithi097 ceph-mon[79892]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:03:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:47 smithi097 ceph-mon[79892]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-16T09:03:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:47 smithi097 ceph-mon[79892]: osdmap e91: 6 total, 5 up, 6 in 2024-09-16T09:03:48.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:47 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3823081339' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:03:48.353 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:47 smithi031 ceph-mon[91884]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:03:48.353 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:47 smithi031 ceph-mon[91884]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:03:48.353 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:47 smithi031 ceph-mon[91884]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-16T09:03:48.354 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:47 smithi031 ceph-mon[91884]: osdmap e91: 6 total, 5 up, 6 in 2024-09-16T09:03:48.354 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:47 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3823081339' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:03:48.712 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:48 smithi097 podman[98559]: 2024-09-16 09:03:48.350941058 +0000 UTC m=+0.015686815 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:03:48.712 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:48 smithi097 podman[98559]: 2024-09-16 09:03:48.474885382 +0000 UTC m=+0.139631137 container create 9ec13ee9ffedfc37e52b34f7b8e02357b15b5a5c32664cdcae19c53043c23b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T09:03:48.712 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:48 smithi097 podman[98559]: 2024-09-16 09:03:48.711354993 +0000 UTC m=+0.376100751 container init 9ec13ee9ffedfc37e52b34f7b8e02357b15b5a5c32664cdcae19c53043c23b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T09:03:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:48 smithi097 ceph-mon[79892]: pgmap v176: 65 pgs: 13 stale+active+clean, 52 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:48 smithi097 ceph-mon[79892]: osdmap e92: 6 total, 5 up, 6 in 2024-09-16T09:03:49.013 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:48 smithi097 podman[98559]: 2024-09-16 09:03:48.715044978 +0000 UTC m=+0.379790733 container start 9ec13ee9ffedfc37e52b34f7b8e02357b15b5a5c32664cdcae19c53043c23b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T09:03:49.013 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:48 smithi097 podman[98559]: 2024-09-16 09:03:48.794084566 +0000 UTC m=+0.458830330 container attach 9ec13ee9ffedfc37e52b34f7b8e02357b15b5a5c32664cdcae19c53043c23b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:03:49.013 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:48 smithi097 podman[98559]: 2024-09-16 09:03:48.902426465 +0000 UTC m=+0.567172221 container died 9ec13ee9ffedfc37e52b34f7b8e02357b15b5a5c32664cdcae19c53043c23b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T09:03:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:48 smithi031 ceph-mon[91884]: pgmap v176: 65 pgs: 13 stale+active+clean, 52 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:48 smithi031 ceph-mon[91884]: osdmap e92: 6 total, 5 up, 6 in 2024-09-16T09:03:49.999 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:49 smithi097 podman[98583]: 2024-09-16 09:03:49.572874791 +0000 UTC m=+0.664274675 container remove 9ec13ee9ffedfc37e52b34f7b8e02357b15b5a5c32664cdcae19c53043c23b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T09:03:49.999 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:49 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-16T09:03:49.999 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:49 smithi097 systemd[1]: Stopped Ceph osd.5 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:03:49.999 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:49 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.5.service: Consumed 1min 3.919s CPU time. 2024-09-16T09:03:50.000 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:49 smithi097 systemd[1]: Starting Ceph osd.5 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:03:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:50 smithi097 ceph-mon[79892]: Health check failed: Reduced data availability: 2 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-16T09:03:50.262 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:49 smithi097 podman[98684]: 2024-09-16 09:03:49.901269238 +0000 UTC m=+0.016276275 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:03:50.263 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 podman[98684]: 2024-09-16 09:03:50.040622813 +0000 UTC m=+0.155629832 container create 170e9c5551e163bc89cdd0a96be09521d841a71a0326346bf0074e370116d7d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T09:03:50.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:50 smithi031 ceph-mon[91884]: Health check failed: Reduced data availability: 2 pgs inactive, 7 pgs peering (PG_AVAILABILITY) 2024-09-16T09:03:50.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 podman[98684]: 2024-09-16 09:03:50.276939837 +0000 UTC m=+0.391946856 container init 170e9c5551e163bc89cdd0a96be09521d841a71a0326346bf0074e370116d7d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T09:03:50.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 podman[98684]: 2024-09-16 09:03:50.280522226 +0000 UTC m=+0.395529254 container start 170e9c5551e163bc89cdd0a96be09521d841a71a0326346bf0074e370116d7d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:03:50.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 podman[98684]: 2024-09-16 09:03:50.402026617 +0000 UTC m=+0.517033640 container attach 170e9c5551e163bc89cdd0a96be09521d841a71a0326346bf0074e370116d7d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:03:50.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:50.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 bash[98684]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:50.763 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:50.763 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:50 smithi097 bash[98684]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:03:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:51 smithi097 ceph-mon[79892]: pgmap v178: 65 pgs: 3 active+undersized, 17 peering, 4 stale+active+clean, 41 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:51 smithi031 ceph-mon[91884]: pgmap v178: 65 pgs: 3 active+undersized, 17 peering, 4 stale+active+clean, 41 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:03:52.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:52 smithi031 ceph-mon[91884]: Health check failed: Degraded data redundancy: 12/282 objects degraded (4.255%), 5 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:52.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:52 smithi097 ceph-mon[79892]: Health check failed: Degraded data redundancy: 12/282 objects degraded (4.255%), 5 pgs degraded (PG_DEGRADED) 2024-09-16T09:03:53.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:52 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T09:03:53.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:52 smithi097 bash[98684]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T09:03:53.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:52 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T09:03:53.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:52 smithi097 bash[98684]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T09:03:53.355 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:53 smithi097 ceph-mon[79892]: pgmap v179: 65 pgs: 7 active+undersized, 17 peering, 5 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 12/282 objects degraded (4.255%) 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 bash[98684]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 bash[98684]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 bash[98684]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T09:03:53.356 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 bash[98684]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-16T09:03:53.357 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate[98699]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-16T09:03:53.357 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 bash[98684]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-16T09:03:53.357 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 podman[98684]: 2024-09-16 09:03:53.060142705 +0000 UTC m=+3.175149727 container died 170e9c5551e163bc89cdd0a96be09521d841a71a0326346bf0074e370116d7d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-16T09:03:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:53 smithi031 ceph-mon[91884]: pgmap v179: 65 pgs: 7 active+undersized, 17 peering, 5 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 12/282 objects degraded (4.255%) 2024-09-16T09:03:53.931 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 podman[98850]: 2024-09-16 09:03:53.680500554 +0000 UTC m=+0.609398053 container remove 170e9c5551e163bc89cdd0a96be09521d841a71a0326346bf0074e370116d7d9 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T09:03:53.931 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 podman[98912]: 2024-09-16 09:03:53.788157584 +0000 UTC m=+0.015818974 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:03:53.932 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:53 smithi097 podman[98912]: 2024-09-16 09:03:53.931418639 +0000 UTC m=+0.159080014 container create 857ff6eb4b2a677cc124bfc3eea5783dc6b5bfc19d3cc128c786177f5ae11ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T09:03:54.192 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:54 smithi097 podman[98912]: 2024-09-16 09:03:54.167164099 +0000 UTC m=+0.394825475 container init 857ff6eb4b2a677cc124bfc3eea5783dc6b5bfc19d3cc128c786177f5ae11ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T09:03:54.192 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:54 smithi097 podman[98912]: 2024-09-16 09:03:54.17062891 +0000 UTC m=+0.398290297 container start 857ff6eb4b2a677cc124bfc3eea5783dc6b5bfc19d3cc128c786177f5ae11ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-16T09:03:54.512 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-osd[98931]: -- 172.21.15.97:0/2106076885 <== mon.1 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc4b21c1e0 con 0x55fc4b1ac000 2024-09-16T09:03:54.512 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:54 smithi097 bash[98912]: 857ff6eb4b2a677cc124bfc3eea5783dc6b5bfc19d3cc128c786177f5ae11ff8 2024-09-16T09:03:54.512 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:54 smithi097 systemd[1]: Started Ceph osd.5 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:03:55.095 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-mon[79892]: pgmap v180: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:03:55.096 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:55.096 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:55.096 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-mon[79892]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 7 pgs peering) 2024-09-16T09:03:55.096 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:55.096 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:55.096 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:54 smithi031 ceph-mon[91884]: pgmap v180: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:03:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:03:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:54 smithi031 ceph-mon[91884]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 7 pgs peering) 2024-09-16T09:03:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:55.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:03:56.011 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:03:55 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:03:55.799+0000 7f077041f640 -1 Falling back to public interface 2024-09-16T09:03:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:56 smithi031 ceph-mon[91884]: pgmap v181: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:03:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:56 smithi097 ceph-mon[79892]: pgmap v181: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:03:58.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:58.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:58.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:58.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:58.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:03:59.211 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:03:58 smithi097 ceph-mon[79892]: pgmap v182: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:03:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:03:58 smithi031 ceph-mon[91884]: pgmap v182: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:00 smithi097 ceph-mon[79892]: pgmap v183: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:01.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:00 smithi031 ceph-mon[91884]: pgmap v183: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:01.378 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:01.378 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:01.378 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:01.378 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:01.378 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:01.970 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:04:01.530+0000 7f077041f640 -1 osd.5 90 log_to_monitors true 2024-09-16T09:04:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all osd 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-16T09:04:02.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-16T09:04:02.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: Upgrade: Setting require_osd_release to 19 squid 2024-09-16T09:04:02.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd require-osd-release", "release": "squid"} : dispatch 2024-09-16T09:04:02.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='osd.5 [v2:172.21.15.97:6816/1603582518,v1:172.21.15.97:6817/1603582518]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T09:04:02.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T09:04:02.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:01 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 41/282 objects degraded (14.539%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T09:04:02.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:02.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all osd 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.0"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.1"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.2"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.3"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.4"} : dispatch 2024-09-16T09:04:02.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-16T09:04:02.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd.5"} : dispatch 2024-09-16T09:04:02.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-16T09:04:02.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: Upgrade: Setting require_osd_release to 19 squid 2024-09-16T09:04:02.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd require-osd-release", "release": "squid"} : dispatch 2024-09-16T09:04:02.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='osd.5 [v2:172.21.15.97:6816/1603582518,v1:172.21.15.97:6817/1603582518]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T09:04:02.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T09:04:02.375 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:01 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 41/282 objects degraded (14.539%), 14 pgs degraded (PG_DEGRADED) 2024-09-16T09:04:03.262 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:04:02.922+0000 7f07641c1640 -1 osd.5 90 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:04:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: pgmap v184: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-16T09:04:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-16T09:04:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T09:04:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: osdmap e93: 6 total, 5 up, 6 in 2024-09-16T09:04:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: from='osd.5 [v2:172.21.15.97:6816/1603582518,v1:172.21.15.97:6817/1603582518]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:04:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:04:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: Upgrade: Scaling down filesystem cephfs 2024-09-16T09:04:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} : dispatch 2024-09-16T09:04:03.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:02 smithi097 ceph-mon[79892]: from='osd.5 ' entity='osd.5' 2024-09-16T09:04:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: pgmap v184: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 931 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-16T09:04:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: osdmap e93: 6 total, 5 up, 6 in 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: from='osd.5 [v2:172.21.15.97:6816/1603582518,v1:172.21.15.97:6817/1603582518]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi097", "root=default"]} : dispatch 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: Upgrade: Scaling down filesystem cephfs 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} : dispatch 2024-09-16T09:04:03.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:02 smithi031 ceph-mon[91884]: from='osd.5 ' entity='osd.5' 2024-09-16T09:04:04.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-09-16T09:04:04.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: stopping daemon mds.cephfs.smithi097.cagmus 2024-09-16T09:04:04.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T09:04:04.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:stopping} 2 up:standby 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: osd.5 [v2:172.21.15.97:6816/1603582518,v1:172.21.15.97:6817/1603582518] boot 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: osdmap e94: 6 total, 6 up, 6 in 2024-09-16T09:04:04.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:04 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:04:04.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-09-16T09:04:04.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: stopping daemon mds.cephfs.smithi097.cagmus 2024-09-16T09:04:04.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:active} 2 up:standby 2024-09-16T09:04:04.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: fsmap cephfs:2 {0=cephfs.smithi031.mjhxgi=up:active,1=cephfs.smithi097.cagmus=up:stopping} 2 up:standby 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: osd.5 [v2:172.21.15.97:6816/1603582518,v1:172.21.15.97:6817/1603582518] boot 2024-09-16T09:04:04.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: osdmap e94: 6 total, 6 up, 6 in 2024-09-16T09:04:04.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:04 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:04:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:05 smithi031 ceph-mon[91884]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-09-16T09:04:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:05 smithi031 ceph-mon[91884]: pgmap v186: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 868 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:05 smithi031 ceph-mon[91884]: osdmap e95: 6 total, 6 up, 6 in 2024-09-16T09:04:05.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:05 smithi097 ceph-mon[79892]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-09-16T09:04:05.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:05 smithi097 ceph-mon[79892]: pgmap v186: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 868 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:05.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:05 smithi097 ceph-mon[79892]: osdmap e95: 6 total, 6 up, 6 in 2024-09-16T09:04:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:07 smithi031 ceph-mon[91884]: pgmap v189: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 868 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:07.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:07 smithi097 ceph-mon[79892]: pgmap v189: 65 pgs: 15 active+undersized, 14 active+undersized+degraded, 36 active+clean; 201 MiB data, 868 MiB used, 536 GiB / 536 GiB avail; 41/282 objects degraded (14.539%) 2024-09-16T09:04:08.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:08 smithi097 ceph-mon[79892]: Health check update: Degraded data redundancy: 21/282 objects degraded (7.447%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T09:04:08.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:08 smithi031 ceph-mon[91884]: Health check update: Degraded data redundancy: 21/282 objects degraded (7.447%), 7 pgs degraded (PG_DEGRADED) 2024-09-16T09:04:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:09 smithi031 ceph-mon[91884]: pgmap v190: 65 pgs: 4 active+undersized, 7 active+undersized+degraded, 54 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s; 21/282 objects degraded (7.447%) 2024-09-16T09:04:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:09 smithi097 ceph-mon[79892]: pgmap v190: 65 pgs: 4 active+undersized, 7 active+undersized+degraded, 54 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s; 21/282 objects degraded (7.447%) 2024-09-16T09:04:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:10.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:10 smithi031 ceph-mon[91884]: pgmap v191: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 280 B/s wr, 0 op/s 2024-09-16T09:04:10.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:10 smithi097 ceph-mon[79892]: pgmap v191: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 280 B/s wr, 0 op/s 2024-09-16T09:04:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:11 smithi031 ceph-mon[91884]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/282 objects degraded (7.447%), 7 pgs degraded) 2024-09-16T09:04:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:11 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:04:11.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:11 smithi097 ceph-mon[79892]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 21/282 objects degraded (7.447%), 7 pgs degraded) 2024-09-16T09:04:11.413 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:11 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:04:12.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:12 smithi031 ceph-mon[91884]: pgmap v192: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:12.432 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:12 smithi097 ceph-mon[79892]: pgmap v192: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:13.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:13.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: pgmap v193: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 217 B/s wr, 0 op/s 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:14 smithi097 ceph-mon[79892]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-09-16T09:04:14.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: pgmap v193: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 217 B/s wr, 0 op/s 2024-09-16T09:04:14.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:14.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:14.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:14.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:14.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:14.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:14 smithi031 ceph-mon[91884]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2024-09-16T09:04:16.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:16 smithi097 ceph-mon[79892]: pgmap v194: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 196 B/s wr, 0 op/s 2024-09-16T09:04:16.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:16 smithi031 ceph-mon[91884]: pgmap v194: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 196 B/s wr, 0 op/s 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.237+0000 7f07873a0640 1 -- 172.21.15.31:0/4083946486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780100430 msgr2=0x7f0780102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.237+0000 7f07873a0640 1 --2- 172.21.15.31:0/4083946486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780100430 0x7f0780102820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f077c0098b0 tx=0x7f077c02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.237+0000 7f07873a0640 1 -- 172.21.15.31:0/4083946486 shutdown_connections 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.237+0000 7f07873a0640 1 --2- 172.21.15.31:0/4083946486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780102d60 0x7f0780105150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.237+0000 7f07873a0640 1 --2- 172.21.15.31:0/4083946486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780100430 0x7f0780102820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.237+0000 7f07873a0640 1 -- 172.21.15.31:0/4083946486 >> 172.21.15.31:0/4083946486 conn(0x7f07800fa010 msgr2=0x7f07800fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.238+0000 7f07873a0640 1 -- 172.21.15.31:0/4083946486 shutdown_connections 2024-09-16T09:04:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.238+0000 7f07873a0640 1 -- 172.21.15.31:0/4083946486 wait complete. 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.238+0000 7f07873a0640 1 Processor -- start 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.238+0000 7f07873a0640 1 -- start start 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f07873a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780100430 0x7f07801954a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f07873a0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780102d60 0x7f07801959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f07873a0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780195fb0 con 0x7f0780102d60 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f07873a0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0780196120 con 0x7f0780100430 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f0785b9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780102d60 0x7f07801959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f0785b9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780102d60 0x7f07801959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36228/0 (socket says 172.21.15.31:36228) 2024-09-16T09:04:17.242 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f0785b9d640 1 -- 172.21.15.31:0/674395527 learned_addr learned my addr 172.21.15.31:0/674395527 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f078639e640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780100430 0x7f07801954a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f0785b9d640 1 -- 172.21.15.31:0/674395527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780100430 msgr2=0x7f07801954a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f0785b9d640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780100430 0x7f07801954a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f0785b9d640 1 -- 172.21.15.31:0/674395527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c009560 con 0x7f0780102d60 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.239+0000 7f0785b9d640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780102d60 0x7f07801959e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f077000d6f0 tx=0x7f077000dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.240+0000 7f07777fe640 1 -- 172.21.15.31:0/674395527 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07700051f0 con 0x7f0780102d60 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.240+0000 7f07777fe640 1 -- 172.21.15.31:0/674395527 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0770009ce0 con 0x7f0780102d60 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.240+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078019abc0 con 0x7f0780102d60 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.240+0000 7f07777fe640 1 -- 172.21.15.31:0/674395527 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0770004e80 con 0x7f0780102d60 2024-09-16T09:04:17.243 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.240+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078019b190 con 0x7f0780102d60 2024-09-16T09:04:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.241+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07801085e0 con 0x7f0780102d60 2024-09-16T09:04:17.244 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.242+0000 7f07777fe640 1 -- 172.21.15.31:0/674395527 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f07700046c0 con 0x7f0780102d60 2024-09-16T09:04:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.242+0000 7f07777fe640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f076c07aee0 0x7f076c07d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.242+0000 7f07777fe640 1 -- 172.21.15.31:0/674395527 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f077009d9b0 con 0x7f0780102d60 2024-09-16T09:04:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.242+0000 7f078639e640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f076c07aee0 0x7f076c07d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.245 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.243+0000 7f078639e640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f076c07aee0 0x7f076c07d3a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f077c006010 tx=0x7f077c0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:17.249 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.247+0000 7f07777fe640 1 -- 172.21.15.31:0/674395527 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f0770060b80 con 0x7f0780102d60 2024-09-16T09:04:17.455 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.453+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f078019b3d0 con 0x7f076c07aee0 2024-09-16T09:04:17.456 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.454+0000 7f07777fe640 1 -- 172.21.15.31:0/674395527 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f078019b3d0 con 0x7f076c07aee0 2024-09-16T09:04:17.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f076c07aee0 msgr2=0x7f076c07d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f076c07aee0 0x7f076c07d3a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f077c006010 tx=0x7f077c0314e0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780102d60 msgr2=0x7f07801959e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780102d60 0x7f07801959e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f077000d6f0 tx=0x7f077000dbb0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 shutdown_connections 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f076c07aee0 0x7f076c07d3a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0780102d60 0x7f07801959e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 --2- 172.21.15.31:0/674395527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0780100430 0x7f07801954a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.456+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 >> 172.21.15.31:0/674395527 conn(0x7f07800fa010 msgr2=0x7f07800fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.457+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 shutdown_connections 2024-09-16T09:04:17.459 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.457+0000 7f07873a0640 1 -- 172.21.15.31:0/674395527 wait complete. 2024-09-16T09:04:17.467 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:04:17.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.541+0000 7f102a399640 1 -- 172.21.15.31:0/2377394119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1024100d90 msgr2=0x7f1024101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.541+0000 7f102a399640 1 --2- 172.21.15.31:0/2377394119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1024100d90 0x7f1024101210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1018009880 tx=0x7f101802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.541+0000 7f102a399640 1 -- 172.21.15.31:0/2377394119 shutdown_connections 2024-09-16T09:04:17.544 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.541+0000 7f102a399640 1 --2- 172.21.15.31:0/2377394119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1024100d90 0x7f1024101210 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.541+0000 7f102a399640 1 --2- 172.21.15.31:0/2377394119 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f10240ffb90 0x7f10240fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.541+0000 7f102a399640 1 -- 172.21.15.31:0/2377394119 >> 172.21.15.31:0/2377394119 conn(0x7f10240fb320 msgr2=0x7f10240fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.542+0000 7f102a399640 1 -- 172.21.15.31:0/2377394119 shutdown_connections 2024-09-16T09:04:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.542+0000 7f102a399640 1 -- 172.21.15.31:0/2377394119 wait complete. 2024-09-16T09:04:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.542+0000 7f102a399640 1 Processor -- start 2024-09-16T09:04:17.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.542+0000 7f102a399640 1 -- start start 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f102a399640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10240ffb90 0x7f1024195630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f102a399640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1024100d90 0x7f1024195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f102a399640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024196140 con 0x7f10240ffb90 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f102a399640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10241962b0 con 0x7f1024100d90 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f1023fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10240ffb90 0x7f1024195630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f1023fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10240ffb90 0x7f1024195630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36242/0 (socket says 172.21.15.31:36242) 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f1023fff640 1 -- 172.21.15.31:0/2489209020 learned_addr learned my addr 172.21.15.31:0/2489209020 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:17.549 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f10237fe640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1024100d90 0x7f1024195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f1023fff640 1 -- 172.21.15.31:0/2489209020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1024100d90 msgr2=0x7f1024195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f1023fff640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1024100d90 0x7f1024195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f1023fff640 1 -- 172.21.15.31:0/2489209020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1018009560 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f1023fff640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10240ffb90 0x7f1024195630 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f101400d900 tx=0x7f101400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.543+0000 7f10217fa640 1 -- 172.21.15.31:0/2489209020 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1014009c00 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.544+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102419ad50 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.544+0000 7f10217fa640 1 -- 172.21.15.31:0/2489209020 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1014009d60 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.544+0000 7f10217fa640 1 -- 172.21.15.31:0/2489209020 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f101400f660 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.544+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102419b290 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.545+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102418f730 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.545+0000 7f10217fa640 1 -- 172.21.15.31:0/2489209020 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f1014010460 con 0x7f10240ffb90 2024-09-16T09:04:17.550 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.546+0000 7f10217fa640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f100c07af30 0x7f100c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.546+0000 7f10217fa640 1 -- 172.21.15.31:0/2489209020 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f101409dce0 con 0x7f10240ffb90 2024-09-16T09:04:17.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.546+0000 7f10237fe640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f100c07af30 0x7f100c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.551 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.547+0000 7f10237fe640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f100c07af30 0x7f100c07d3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1018000c00 tx=0x7f10180314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:17.553 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.551+0000 7f10217fa640 1 -- 172.21.15.31:0/2489209020 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f1014060f30 con 0x7f10240ffb90 2024-09-16T09:04:17.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.751+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f102402c560 con 0x7f100c07af30 2024-09-16T09:04:17.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.752+0000 7f10217fa640 1 -- 172.21.15.31:0/2489209020 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f102402c560 con 0x7f100c07af30 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f100c07af30 msgr2=0x7f100c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f100c07af30 0x7f100c07d3f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1018000c00 tx=0x7f10180314e0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10240ffb90 msgr2=0x7f1024195630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10240ffb90 0x7f1024195630 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f101400d900 tx=0x7f101400ddc0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 shutdown_connections 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f100c07af30 0x7f100c07d3f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1024100d90 0x7f1024195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 --2- 172.21.15.31:0/2489209020 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10240ffb90 0x7f1024195630 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 >> 172.21.15.31:0/2489209020 conn(0x7f10240fb320 msgr2=0x7f1024104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 shutdown_connections 2024-09-16T09:04:17.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.755+0000 7f102a399640 1 -- 172.21.15.31:0/2489209020 wait complete. 2024-09-16T09:04:17.841 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 -- 172.21.15.31:0/994500127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 msgr2=0x7fba1c0feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 --2- 172.21.15.31:0/994500127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c0feef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fba18009880 tx=0x7fba1802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 -- 172.21.15.31:0/994500127 shutdown_connections 2024-09-16T09:04:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 --2- 172.21.15.31:0/994500127 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba1c0ffcf0 0x7fba1c100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 --2- 172.21.15.31:0/994500127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c0feef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 -- 172.21.15.31:0/994500127 >> 172.21.15.31:0/994500127 conn(0x7fba1c0fa280 msgr2=0x7fba1c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 -- 172.21.15.31:0/994500127 shutdown_connections 2024-09-16T09:04:17.842 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.839+0000 7fba24273640 1 -- 172.21.15.31:0/994500127 wait complete. 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba24273640 1 Processor -- start 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba24273640 1 -- start start 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba24273640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba24273640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba1c0ffcf0 0x7fba1c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba24273640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c19a9f0 con 0x7fba1c0feaf0 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba24273640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1c19ab30 con 0x7fba1c0ffcf0 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba2200f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba2200f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c068790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36264/0 (socket says 172.21.15.31:36264) 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.840+0000 7fba2200f640 1 -- 172.21.15.31:0/2328706713 learned_addr learned my addr 172.21.15.31:0/2328706713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:17.844 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba2180e640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba1c0ffcf0 0x7fba1c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba2200f640 1 -- 172.21.15.31:0/2328706713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba1c0ffcf0 msgr2=0x7fba1c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba2200f640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba1c0ffcf0 0x7fba1c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba2200f640 1 -- 172.21.15.31:0/2328706713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18009560 con 0x7fba1c0feaf0 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba2180e640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba1c0ffcf0 0x7fba1c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba2200f640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c068790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fba1802a9c0 tx=0x7fba180054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba137fe640 1 -- 172.21.15.31:0/2328706713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba18004aa0 con 0x7fba1c0feaf0 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1c19ad80 con 0x7fba1c0feaf0 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba137fe640 1 -- 172.21.15.31:0/2328706713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba18004c00 con 0x7fba1c0feaf0 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba137fe640 1 -- 172.21.15.31:0/2328706713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba18003a80 con 0x7fba1c0feaf0 2024-09-16T09:04:17.845 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.841+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1c19b270 con 0x7fba1c0feaf0 2024-09-16T09:04:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.842+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1c18e690 con 0x7fba1c0feaf0 2024-09-16T09:04:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.843+0000 7fba137fe640 1 -- 172.21.15.31:0/2328706713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fba18003cf0 con 0x7fba1c0feaf0 2024-09-16T09:04:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.843+0000 7fba137fe640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba0807af30 0x7fba0807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.843+0000 7fba137fe640 1 -- 172.21.15.31:0/2328706713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fba180bf120 con 0x7fba1c0feaf0 2024-09-16T09:04:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.843+0000 7fba2180e640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba0807af30 0x7fba0807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:17.846 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.844+0000 7fba2180e640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba0807af30 0x7fba0807d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba1c068d10 tx=0x7fba0c005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:17.850 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:17.848+0000 7fba137fe640 1 -- 172.21.15.31:0/2328706713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fba180822c0 con 0x7fba1c0feaf0 2024-09-16T09:04:18.042 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.040+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fba1c02c550 con 0x7fba0807af30 2024-09-16T09:04:18.051 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.049+0000 7fba137fe640 1 -- 172.21.15.31:0/2328706713 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fba1c02c550 con 0x7fba0807af30 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (13m) 2m ago 15m 25.3M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (6m) 2m ago 16m 8476k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (6m) 21s ago 14m 8674k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (4m) 2m ago 15m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (3m) 21s ago 14m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (13m) 2m ago 15m 85.7M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (9m) 2m ago 9m 16.3M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 265cf7079daa 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (9m) 2m ago 9m 153M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 46a823308f93 2024-09-16T09:04:18.052 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (9m) 21s ago 9m 27.0M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (9m) 21s ago 9m 18.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (6m) 2m ago 17m 516M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (5m) 21s ago 14m 455M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (5m) 2m ago 17m 48.2M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (4m) 21s ago 14m 38.9M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (15m) 2m ago 15m 15.8M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (14m) 21s ago 14m 16.7M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (3m) 2m ago 13m 93.6M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (3m) 2m ago 12m 98.7M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 2m ago 12m 13.3M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (108s) 21s ago 11m 140M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (74s) 21s ago 11m 94.6M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (24s) 21s ago 11m 14.3M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:04:18.053 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 2m ago 15m 65.2M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:04:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba0807af30 msgr2=0x7fba0807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba0807af30 0x7fba0807d3f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fba1c068d10 tx=0x7fba0c005e50 comp rx=0 tx=0).stop 2024-09-16T09:04:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 msgr2=0x7fba1c068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c068790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fba1802a9c0 tx=0x7fba180054f0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 shutdown_connections 2024-09-16T09:04:18.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba0807af30 0x7fba0807d3f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba1c0ffcf0 0x7fba1c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 --2- 172.21.15.31:0/2328706713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba1c0feaf0 0x7fba1c068790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.051+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 >> 172.21.15.31:0/2328706713 conn(0x7fba1c0fa280 msgr2=0x7fba1c1033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.052+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 shutdown_connections 2024-09-16T09:04:18.055 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.052+0000 7fba24273640 1 -- 172.21.15.31:0/2328706713 wait complete. 2024-09-16T09:04:18.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 -- 172.21.15.31:0/3534677997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 msgr2=0x7f5860102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 --2- 172.21.15.31:0/3534677997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f5860102a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f585c0098b0 tx=0x7f585c02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 -- 172.21.15.31:0/3534677997 shutdown_connections 2024-09-16T09:04:18.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 --2- 172.21.15.31:0/3534677997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5860102fb0 0x7f58601053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 --2- 172.21.15.31:0/3534677997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f5860102a70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.136 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 -- 172.21.15.31:0/3534677997 >> 172.21.15.31:0/3534677997 conn(0x7f58600fa280 msgr2=0x7f58600fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:18.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 -- 172.21.15.31:0/3534677997 shutdown_connections 2024-09-16T09:04:18.137 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 -- 172.21.15.31:0/3534677997 wait complete. 2024-09-16T09:04:18.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 Processor -- start 2024-09-16T09:04:18.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.134+0000 7f58672a5640 1 -- start start 2024-09-16T09:04:18.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f58672a5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f58601956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f58672a5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5860102fb0 0x7f5860195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f58672a5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5860196200 con 0x7f5860100680 2024-09-16T09:04:18.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f58672a5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5860196370 con 0x7f5860102fb0 2024-09-16T09:04:18.138 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5865041640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f58601956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5865041640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f58601956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36286/0 (socket says 172.21.15.31:36286) 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5865041640 1 -- 172.21.15.31:0/478516205 learned_addr learned my addr 172.21.15.31:0/478516205 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5864840640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5860102fb0 0x7f5860195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5865041640 1 -- 172.21.15.31:0/478516205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5860102fb0 msgr2=0x7f5860195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5865041640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5860102fb0 0x7f5860195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5865041640 1 -- 172.21.15.31:0/478516205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c009560 con 0x7f5860100680 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.135+0000 7f5865041640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f58601956f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f585c0054f0 tx=0x7f585c005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.136+0000 7f58567fc640 1 -- 172.21.15.31:0/478516205 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c004aa0 con 0x7f5860100680 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.136+0000 7f58567fc640 1 -- 172.21.15.31:0/478516205 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f585c004c00 con 0x7f5860100680 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.136+0000 7f58567fc640 1 -- 172.21.15.31:0/478516205 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585c003b10 con 0x7f5860100680 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.136+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586019adb0 con 0x7f5860100680 2024-09-16T09:04:18.139 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.136+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586019b240 con 0x7f5860100680 2024-09-16T09:04:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.137+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5860108810 con 0x7f5860100680 2024-09-16T09:04:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.137+0000 7f58567fc640 1 -- 172.21.15.31:0/478516205 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f585c003d80 con 0x7f5860100680 2024-09-16T09:04:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.138+0000 7f58567fc640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f584c07af30 0x7f584c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.140 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.138+0000 7f58567fc640 1 -- 172.21.15.31:0/478516205 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f585c0bf0d0 con 0x7f5860100680 2024-09-16T09:04:18.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.138+0000 7f5864840640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f584c07af30 0x7f584c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.141 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.138+0000 7f5864840640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f584c07af30 0x7f584c07d3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5860196c10 tx=0x7f585000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:18.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.142+0000 7f58567fc640 1 -- 172.21.15.31:0/478516205 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f585c082320 con 0x7f5860100680 2024-09-16T09:04:18.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.402+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f586019b4b0 con 0x7f5860100680 2024-09-16T09:04:18.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:18 smithi031 ceph-mon[91884]: pgmap v195: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-16T09:04:18.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:18 smithi031 ceph-mon[91884]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:18.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:18 smithi031 ceph-mon[91884]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:18.405 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:18 smithi031 ceph-mon[91884]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:18.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.402+0000 7f58567fc640 1 -- 172.21.15.31:0/478516205 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+729 (secure 0 0 0) 0x7f585c085440 con 0x7f5860100680 2024-09-16T09:04:18.405 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:18.406 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:04:18.407 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 4, 2024-09-16T09:04:18.407 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 10 2024-09-16T09:04:18.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:04:18.407 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:04:18.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.404+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f584c07af30 msgr2=0x7f584c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.404+0000 7f58672a5640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f584c07af30 0x7f584c07d3f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5860196c10 tx=0x7f585000b410 comp rx=0 tx=0).stop 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.404+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 msgr2=0x7f58601956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f58601956f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f585c0054f0 tx=0x7f585c005520 comp rx=0 tx=0).stop 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 shutdown_connections 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f584c07af30 0x7f584c07d3f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5860102fb0 0x7f5860195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 --2- 172.21.15.31:0/478516205 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5860100680 0x7f58601956f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 >> 172.21.15.31:0/478516205 conn(0x7f58600fa280 msgr2=0x7f58600fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 shutdown_connections 2024-09-16T09:04:18.408 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.405+0000 7f58672a5640 1 -- 172.21.15.31:0/478516205 wait complete. 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 -- 172.21.15.31:0/647255925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c0ffbc0 msgr2=0x7fc07c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 --2- 172.21.15.31:0/647255925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c0ffbc0 0x7fc07c0fffc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009880 tx=0x7fc06c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 -- 172.21.15.31:0/647255925 shutdown_connections 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 --2- 172.21.15.31:0/647255925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc07c100dc0 0x7fc07c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 --2- 172.21.15.31:0/647255925 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c0ffbc0 0x7fc07c0fffc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 -- 172.21.15.31:0/647255925 >> 172.21.15.31:0/647255925 conn(0x7fc07c0fb350 msgr2=0x7fc07c0fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 -- 172.21.15.31:0/647255925 shutdown_connections 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.488+0000 7fc08182c640 1 -- 172.21.15.31:0/647255925 wait complete. 2024-09-16T09:04:18.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc08182c640 1 Processor -- start 2024-09-16T09:04:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc08182c640 1 -- start start 2024-09-16T09:04:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc08182c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc07c0ffbc0 0x7fc07c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc08182c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c100dc0 0x7fc07c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc08182c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c198450 con 0x7fc07c100dc0 2024-09-16T09:04:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc08182c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c1985c0 con 0x7fc07c0ffbc0 2024-09-16T09:04:18.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc07a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c100dc0 0x7fc07c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc07a7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c100dc0 0x7fc07c197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36300/0 (socket says 172.21.15.31:36300) 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.489+0000 7fc07a7fc640 1 -- 172.21.15.31:0/852705651 learned_addr learned my addr 172.21.15.31:0/852705651 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc07affd640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc07c0ffbc0 0x7fc07c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc07a7fc640 1 -- 172.21.15.31:0/852705651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc07c0ffbc0 msgr2=0x7fc07c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc07a7fc640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc07c0ffbc0 0x7fc07c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc07a7fc640 1 -- 172.21.15.31:0/852705651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c009560 con 0x7fc07c100dc0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc07a7fc640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c100dc0 0x7fc07c197e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc07000b450 tx=0x7fc07000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc08082a640 1 -- 172.21.15.31:0/852705651 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc070011860 con 0x7fc07c100dc0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc08082a640 1 -- 172.21.15.31:0/852705651 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc070011e80 con 0x7fc07c100dc0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c19d060 con 0x7fc07c100dc0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc08082a640 1 -- 172.21.15.31:0/852705651 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc07000f570 con 0x7fc07c100dc0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.490+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c19d630 con 0x7fc07c100dc0 2024-09-16T09:04:18.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.491+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c105700 con 0x7fc07c100dc0 2024-09-16T09:04:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.492+0000 7fc08082a640 1 -- 172.21.15.31:0/852705651 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fc0700103e0 con 0x7fc07c100dc0 2024-09-16T09:04:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.492+0000 7fc08082a640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc06407af30 0x7fc06407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.493+0000 7fc07affd640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc06407af30 0x7fc06407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.493+0000 7fc08082a640 1 -- 172.21.15.31:0/852705651 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fc07009dc70 con 0x7fc07c100dc0 2024-09-16T09:04:18.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.493+0000 7fc07affd640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc06407af30 0x7fc06407d3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009230 tx=0x7fc06c0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:18.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.497+0000 7fc08082a640 1 -- 172.21.15.31:0/852705651 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fc070060e10 con 0x7fc07c100dc0 2024-09-16T09:04:18.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.721+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc07c02c480 con 0x7fc07c100dc0 2024-09-16T09:04:18.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.722+0000 7fc08082a640 1 -- 172.21.15.31:0/852705651 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1929 (secure 0 0 0) 0x7fc070063f30 con 0x7fc07c100dc0 2024-09-16T09:04:18.724 INFO:teuthology.orchestra.run.smithi031.stdout:e19 2024-09-16T09:04:18.724 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T09:04:03:030615+0000 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 19 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T09:04:03.030612+0000 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:04:18.725 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 47 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=14548,1=24325} 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:04:18.726 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 14548 members: 14548 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{0:14548} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.31:6828/1804644990,v1:172.21.15.31:6829/1804644990] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{1:24325} state up:stopping seq 4 join_fscid=1 addr [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:14560} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/2549073061,v1:172.21.15.31:6827/2549073061] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:04:18.727 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:24337} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:04:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-16T09:04:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc06407af30 msgr2=0x7fc06407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc06407af30 0x7fc06407d3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009230 tx=0x7fc06c0314e0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c100dc0 msgr2=0x7fc07c197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c100dc0 0x7fc07c197e80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc07000b450 tx=0x7fc07000b910 comp rx=0 tx=0).stop 2024-09-16T09:04:18.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 shutdown_connections 2024-09-16T09:04:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc06407af30 0x7fc06407d3f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07c100dc0 0x7fc07c197e80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 --2- 172.21.15.31:0/852705651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc07c0ffbc0 0x7fc07c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 >> 172.21.15.31:0/852705651 conn(0x7fc07c0fb350 msgr2=0x7fc07c072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 shutdown_connections 2024-09-16T09:04:18.729 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.724+0000 7fc08182c640 1 -- 172.21.15.31:0/852705651 wait complete. 2024-09-16T09:04:18.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:18 smithi097 ceph-mon[79892]: pgmap v195: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-09-16T09:04:18.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:18 smithi097 ceph-mon[79892]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:18.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:18 smithi097 ceph-mon[79892]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:18.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:18 smithi097 ceph-mon[79892]: from='client.34406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 -- 172.21.15.31:0/1956884243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0ff410 msgr2=0x7f6d1c0ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 --2- 172.21.15.31:0/1956884243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0ff410 0x7f6d1c0ff890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009c20 tx=0x7f6d1002a970 comp rx=0 tx=0).stop 2024-09-16T09:04:18.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 -- 172.21.15.31:0/1956884243 shutdown_connections 2024-09-16T09:04:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 --2- 172.21.15.31:0/1956884243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0ff410 0x7f6d1c0ff890 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 --2- 172.21.15.31:0/1956884243 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1c0fe210 0x7f6d1c0fe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 -- 172.21.15.31:0/1956884243 >> 172.21.15.31:0/1956884243 conn(0x7f6d1c0f99c0 msgr2=0x7f6d1c0fbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 -- 172.21.15.31:0/1956884243 shutdown_connections 2024-09-16T09:04:18.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.810+0000 7f6d21987640 1 -- 172.21.15.31:0/1956884243 wait complete. 2024-09-16T09:04:18.814 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d21987640 1 Processor -- start 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d21987640 1 -- start start 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d21987640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0fe210 0x7f6d1c195620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d21987640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1c0ff410 0x7f6d1c195b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d21987640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c196130 con 0x7f6d1c0fe210 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d21987640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c1962a0 con 0x7f6d1c0ff410 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0fe210 0x7f6d1c195620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d1affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0fe210 0x7f6d1c195620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36318/0 (socket says 172.21.15.31:36318) 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.811+0000 7f6d1affd640 1 -- 172.21.15.31:0/1328734359 learned_addr learned my addr 172.21.15.31:0/1328734359 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d1a7fc640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1c0ff410 0x7f6d1c195b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d1affd640 1 -- 172.21.15.31:0/1328734359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1c0ff410 msgr2=0x7f6d1c195b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d1affd640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1c0ff410 0x7f6d1c195b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:18.815 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d1affd640 1 -- 172.21.15.31:0/1328734359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d10009900 con 0x7f6d1c0fe210 2024-09-16T09:04:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d1a7fc640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1c0ff410 0x7f6d1c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:04:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d1affd640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0fe210 0x7f6d1c195620 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00e970 tx=0x7f6d0c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d20985640 1 -- 172.21.15.31:0/1328734359 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d0c0096e0 con 0x7f6d1c0fe210 2024-09-16T09:04:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c19ad40 con 0x7f6d1c0fe210 2024-09-16T09:04:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d20985640 1 -- 172.21.15.31:0/1328734359 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d0c0044f0 con 0x7f6d1c0fe210 2024-09-16T09:04:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d20985640 1 -- 172.21.15.31:0/1328734359 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d0c010430 con 0x7f6d1c0fe210 2024-09-16T09:04:18.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.812+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c19b2b0 con 0x7f6d1c0fe210 2024-09-16T09:04:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.813+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c18f530 con 0x7f6d1c0fe210 2024-09-16T09:04:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.814+0000 7f6d20985640 1 -- 172.21.15.31:0/1328734359 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f6d0c00cc90 con 0x7f6d1c0fe210 2024-09-16T09:04:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.814+0000 7f6d20985640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d0407af40 0x7f6d0407d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.814+0000 7f6d20985640 1 -- 172.21.15.31:0/1328734359 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6653+0+0 (secure 0 0 0) 0x7f6d0c014070 con 0x7f6d1c0fe210 2024-09-16T09:04:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.814+0000 7f6d1a7fc640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d0407af40 0x7f6d0407d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:18.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.815+0000 7f6d1a7fc640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d0407af40 0x7f6d0407d400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c196b40 tx=0x7f6d10005ef0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:18.821 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:18.819+0000 7f6d20985640 1 -- 172.21.15.31:0/1328734359 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6d0c061670 con 0x7f6d1c0fe210 2024-09-16T09:04:19.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.033+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d1c02c550 con 0x7f6d0407af40 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.035+0000 7f6d20985640 1 -- 172.21.15.31:0/1328734359 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6d1c02c550 con 0x7f6d0407af40 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:04:19.037 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-09-16T09:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "osd" 2024-09-16T09:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "12/23 daemons upgraded", 2024-09-16T09:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading osd daemons", 2024-09-16T09:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:04:19.038 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:04:19.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d0407af40 msgr2=0x7f6d0407d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:19.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d0407af40 0x7f6d0407d400 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c196b40 tx=0x7f6d10005ef0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0fe210 msgr2=0x7f6d1c195620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:19.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0fe210 0x7f6d1c195620 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00e970 tx=0x7f6d0c00ee30 comp rx=0 tx=0).stop 2024-09-16T09:04:19.039 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 shutdown_connections 2024-09-16T09:04:19.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d0407af40 0x7f6d0407d400 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d1c0ff410 0x7f6d1c195b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 --2- 172.21.15.31:0/1328734359 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d1c0fe210 0x7f6d1c195620 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 >> 172.21.15.31:0/1328734359 conn(0x7f6d1c0f99c0 msgr2=0x7f6d1c102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:19.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 shutdown_connections 2024-09-16T09:04:19.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.037+0000 7f6d21987640 1 -- 172.21.15.31:0/1328734359 wait complete. 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.121+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/2713138449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 msgr2=0x7fdf94101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.121+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/2713138449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf94101210 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009880 tx=0x7fdf8402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.121+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/2713138449 shutdown_connections 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.121+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/2713138449 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf94101210 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.121+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/2713138449 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf940ffb90 0x7fdf940fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.121+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/2713138449 >> 172.21.15.31:0/2713138449 conn(0x7fdf940fb320 msgr2=0x7fdf940fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/2713138449 shutdown_connections 2024-09-16T09:04:19.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/2713138449 wait complete. 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 Processor -- start 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 -- start start 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf940ffb90 0x7fdf94074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf940728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94072de0 con 0x7fdf94100d90 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf9b8fa640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94072f50 con 0x7fdf940ffb90 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.122+0000 7fdf98e95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf940728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf98e95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf940728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:36340/0 (socket says 172.21.15.31:36340) 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf98e95640 1 -- 172.21.15.31:0/3970941782 learned_addr learned my addr 172.21.15.31:0/3970941782 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf99696640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf940ffb90 0x7fdf94074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf98e95640 1 -- 172.21.15.31:0/3970941782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf940ffb90 msgr2=0x7fdf94074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf98e95640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf940ffb90 0x7fdf94074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.126 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf98e95640 1 -- 172.21.15.31:0/3970941782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84009560 con 0x7fdf94100d90 2024-09-16T09:04:19.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf98e95640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf940728a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006010 tx=0x7fdf840055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf8a7fc640 1 -- 172.21.15.31:0/3970941782 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf84004b20 con 0x7fdf94100d90 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf940731d0 con 0x7fdf94100d90 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf8a7fc640 1 -- 172.21.15.31:0/3970941782 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf84004c80 con 0x7fdf94100d90 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf8a7fc640 1 -- 172.21.15.31:0/3970941782 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf84003b80 con 0x7fdf94100d90 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.123+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9419d690 con 0x7fdf94100d90 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.124+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf940fff90 con 0x7fdf94100d90 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.125+0000 7fdf8a7fc640 1 -- 172.21.15.31:0/3970941782 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fdf84031640 con 0x7fdf94100d90 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.126+0000 7fdf8a7fc640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf8007af30 0x7fdf8007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.126+0000 7fdf99696640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf8007af30 0x7fdf8007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:19.128 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.126+0000 7fdf8a7fc640 1 -- 172.21.15.31:0/3970941782 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fdf840bf260 con 0x7fdf94100d90 2024-09-16T09:04:19.129 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.126+0000 7fdf99696640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf8007af30 0x7fdf8007d3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf900058c0 tx=0x7fdf9000a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:19.132 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.130+0000 7fdf8a7fc640 1 -- 172.21.15.31:0/3970941782 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fdf84082430 con 0x7fdf94100d90 2024-09-16T09:04:19.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.389+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdf9402c3e0 con 0x7fdf94100d90 2024-09-16T09:04:19.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.389+0000 7fdf8a7fc640 1 -- 172.21.15.31:0/3970941782 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fdf84085550 con 0x7fdf94100d90 2024-09-16T09:04:19.392 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.391+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf8007af30 msgr2=0x7fdf8007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.391+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf8007af30 0x7fdf8007d3f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdf900058c0 tx=0x7fdf9000a2d0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.391+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 msgr2=0x7fdf940728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.391+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf940728a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdf84006010 tx=0x7fdf840055f0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.392+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 shutdown_connections 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.392+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf8007af30 0x7fdf8007d3f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.392+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf94100d90 0x7fdf940728a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.392+0000 7fdf9b8fa640 1 --2- 172.21.15.31:0/3970941782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf940ffb90 0x7fdf94074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:19.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.392+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 >> 172.21.15.31:0/3970941782 conn(0x7fdf940fb320 msgr2=0x7fdf94104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:19.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.392+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 shutdown_connections 2024-09-16T09:04:19.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:19.392+0000 7fdf9b8fa640 1 -- 172.21.15.31:0/3970941782 wait complete. 2024-09-16T09:04:19.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:19 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/478516205' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:19.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:19 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/852705651' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:04:19.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:19.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:19 smithi097 ceph-mon[79892]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:19.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:19 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/478516205' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:19.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:19 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/852705651' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:04:19.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:19.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:19 smithi031 ceph-mon[91884]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:20.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:20 smithi097 ceph-mon[79892]: pgmap v196: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:20.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:20 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3970941782' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:04:20.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:20 smithi097 ceph-mon[79892]: daemon mds.cephfs.smithi097.cagmus finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-09-16T09:04:20.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:20 smithi031 ceph-mon[91884]: pgmap v196: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:20.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:20 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3970941782' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:04:20.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:20 smithi031 ceph-mon[91884]: daemon mds.cephfs.smithi097.cagmus finished stopping rank 1 in filesystem cephfs (now has 1 ranks) 2024-09-16T09:04:21.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:21 smithi097 ceph-mon[79892]: mds.1 [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] down:stopped 2024-09-16T09:04:21.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:21 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 2 up:standby 2024-09-16T09:04:21.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:21 smithi031 ceph-mon[91884]: mds.1 [v2:172.21.15.97:6824/17261087,v1:172.21.15.97:6825/17261087] down:stopped 2024-09-16T09:04:21.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:21 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 2 up:standby 2024-09-16T09:04:22.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:22 smithi097 ceph-mon[79892]: pgmap v197: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:22.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:22 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.97:6824/1475349315,v1:172.21.15.97:6825/1475349315] up:boot 2024-09-16T09:04:22.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:22 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 3 up:standby 2024-09-16T09:04:22.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:22 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"} : dispatch 2024-09-16T09:04:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:22 smithi031 ceph-mon[91884]: pgmap v197: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:22 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.97:6824/1475349315,v1:172.21.15.97:6825/1475349315] up:boot 2024-09-16T09:04:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:22 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 3 up:standby 2024-09-16T09:04:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:22 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"} : dispatch 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: pgmap v198: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.567 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi031.lcfutm"]} : dispatch 2024-09-16T09:04:24.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: Upgrade: It appears safe to stop mds.cephfs.smithi031.lcfutm 2024-09-16T09:04:24.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:24.568 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: pgmap v198: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:24.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi031.lcfutm"]} : dispatch 2024-09-16T09:04:24.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: Upgrade: It appears safe to stop mds.cephfs.smithi031.lcfutm 2024-09-16T09:04:24.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:24.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:25.720 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:25 smithi031 ceph-mon[91884]: Upgrade: Updating mds.cephfs.smithi031.lcfutm 2024-09-16T09:04:25.720 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:25.721 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:04:25.721 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:25.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:25 smithi097 ceph-mon[79892]: Upgrade: Updating mds.cephfs.smithi031.lcfutm 2024-09-16T09:04:25.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:25.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.lcfutm", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:04:25.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:26.465 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:26 smithi031 ceph-mon[91884]: Deploying daemon mds.cephfs.smithi031.lcfutm on smithi031 2024-09-16T09:04:26.466 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:26 smithi031 ceph-mon[91884]: pgmap v199: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:26.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:26 smithi097 ceph-mon[79892]: Deploying daemon mds.cephfs.smithi031.lcfutm on smithi031 2024-09-16T09:04:26.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:26 smithi097 ceph-mon[79892]: pgmap v199: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:28.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:28 smithi097 ceph-mon[79892]: pgmap v200: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:28.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:28 smithi097 ceph-mon[79892]: osdmap e96: 6 total, 6 up, 6 in 2024-09-16T09:04:28.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:28 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 2 up:standby 2024-09-16T09:04:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:28 smithi031 ceph-mon[91884]: pgmap v200: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-09-16T09:04:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:28 smithi031 ceph-mon[91884]: osdmap e96: 6 total, 6 up, 6 in 2024-09-16T09:04:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:28 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 2 up:standby 2024-09-16T09:04:30.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:30 smithi097 ceph-mon[79892]: pgmap v202: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:30.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:30 smithi031 ceph-mon[91884]: pgmap v202: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:31 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:31 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:31 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:31 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:boot 2024-09-16T09:04:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:31 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 3 up:standby 2024-09-16T09:04:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:31 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"} : dispatch 2024-09-16T09:04:32.048 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:32.048 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:32.048 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:32.048 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:31 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:boot 2024-09-16T09:04:32.048 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:31 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi031.mjhxgi=up:active} 3 up:standby 2024-09-16T09:04:32.049 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:31 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.lcfutm"} : dispatch 2024-09-16T09:04:33.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:32 smithi097 ceph-mon[79892]: pgmap v203: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:33.042 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:32 smithi031 ceph-mon[91884]: pgmap v203: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:34 smithi097 ceph-mon[79892]: pgmap v204: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:35.070 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:34 smithi031 ceph-mon[91884]: pgmap v204: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:36.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:36 smithi031 ceph-mon[91884]: pgmap v205: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:36.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:36.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:36.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:36.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:36 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:36 smithi097 ceph-mon[79892]: pgmap v205: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:36 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:38.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:38 smithi097 ceph-mon[79892]: pgmap v206: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:38.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:38 smithi031 ceph-mon[91884]: pgmap v206: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:39.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:39.889 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:39.890 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi031.mjhxgi"]} : dispatch 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:40.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:40.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi031.mjhxgi"]} : dispatch 2024-09-16T09:04:40.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:40 smithi031 ceph-mon[91884]: pgmap v207: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:40.893 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:40 smithi031 ceph-mon[91884]: Upgrade: It appears safe to stop mds.cephfs.smithi031.mjhxgi 2024-09-16T09:04:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:40 smithi097 ceph-mon[79892]: pgmap v207: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:40 smithi097 ceph-mon[79892]: Upgrade: It appears safe to stop mds.cephfs.smithi031.mjhxgi 2024-09-16T09:04:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:41 smithi031 ceph-mon[91884]: Upgrade: Updating mds.cephfs.smithi031.mjhxgi 2024-09-16T09:04:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:04:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:41 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:42.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:41 smithi031 ceph-mon[91884]: Deploying daemon mds.cephfs.smithi031.mjhxgi on smithi031 2024-09-16T09:04:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:41 smithi097 ceph-mon[79892]: Upgrade: Updating mds.cephfs.smithi031.mjhxgi 2024-09-16T09:04:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi031.mjhxgi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:04:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:41 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:42.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:41 smithi097 ceph-mon[79892]: Deploying daemon mds.cephfs.smithi031.mjhxgi on smithi031 2024-09-16T09:04:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[91880]: 2024-09-16T09:04:42.709+0000 7faf88e84640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T09:04:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: pgmap v208: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T09:04:43.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T09:04:43.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: osdmap e97: 6 total, 6 up, 6 in 2024-09-16T09:04:43.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: Standby daemon mds.cephfs.smithi097.sbwmfq assigned to filesystem cephfs as rank 0 2024-09-16T09:04:43.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T09:04:43.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-16T09:04:43.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:42 smithi031 ceph-mon[91884]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:replay} 2 up:standby 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: pgmap v208: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: osdmap e97: 6 total, 6 up, 6 in 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: Standby daemon mds.cephfs.smithi097.sbwmfq assigned to filesystem cephfs as rank 0 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-16T09:04:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:42 smithi097 ceph-mon[79892]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:replay} 2 up:standby 2024-09-16T09:04:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:44 smithi031 ceph-mon[91884]: pgmap v210: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-09-16T09:04:45.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:44 smithi097 ceph-mon[79892]: pgmap v210: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 1 op/s 2024-09-16T09:04:47.117 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:46 smithi031 ceph-mon[91884]: pgmap v211: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 1 op/s 2024-09-16T09:04:47.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:46 smithi097 ceph-mon[79892]: pgmap v211: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s rd, 1 op/s 2024-09-16T09:04:48.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:48 smithi031 ceph-mon[91884]: pgmap v212: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-09-16T09:04:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:48 smithi097 ceph-mon[79892]: pgmap v212: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 11 MiB/s rd, 4 op/s 2024-09-16T09:04:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 -- 172.21.15.31:0/456198307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fe4d0 msgr2=0x7f67980fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 --2- 172.21.15.31:0/456198307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fe4d0 0x7f67980fe8d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6794009880 tx=0x7f679402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 -- 172.21.15.31:0/456198307 shutdown_connections 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 --2- 172.21.15.31:0/456198307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67980fee10 0x7f67980ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 --2- 172.21.15.31:0/456198307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fe4d0 0x7f67980fe8d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 -- 172.21.15.31:0/456198307 >> 172.21.15.31:0/456198307 conn(0x7f67980fa280 msgr2=0x7f67980fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 -- 172.21.15.31:0/456198307 shutdown_connections 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.482+0000 7f679f935640 1 -- 172.21.15.31:0/456198307 wait complete. 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.483+0000 7f679f935640 1 Processor -- start 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.483+0000 7f679f935640 1 -- start start 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.483+0000 7f679f935640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67980fe4d0 0x7f6798197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.483+0000 7f679f935640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fee10 0x7f6798197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.483+0000 7f679f935640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6798198350 con 0x7f67980fee10 2024-09-16T09:04:49.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.483+0000 7f679f935640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67981984c0 con 0x7f67980fe4d0 2024-09-16T09:04:49.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679ced0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fee10 0x7f6798197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679d6d1640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67980fe4d0 0x7f6798197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679ced0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fee10 0x7f6798197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51898/0 (socket says 172.21.15.31:51898) 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679ced0640 1 -- 172.21.15.31:0/1731998338 learned_addr learned my addr 172.21.15.31:0/1731998338 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679ced0640 1 -- 172.21.15.31:0/1731998338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67980fe4d0 msgr2=0x7f6798197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679ced0640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67980fe4d0 0x7f6798197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679ced0640 1 -- 172.21.15.31:0/1731998338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794009560 con 0x7f67980fee10 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679ced0640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fee10 0x7f6798197d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f678800d900 tx=0x7f678800ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f678e7fc640 1 -- 172.21.15.31:0/1731998338 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6788009c00 con 0x7f67980fee10 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679819cf60 con 0x7f67980fee10 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f678e7fc640 1 -- 172.21.15.31:0/1731998338 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f678800b660 con 0x7f67980fee10 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f678e7fc640 1 -- 172.21.15.31:0/1731998338 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f678800f680 con 0x7f67980fee10 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.484+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679819d480 con 0x7f67980fee10 2024-09-16T09:04:49.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.486+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6798191950 con 0x7f67980fee10 2024-09-16T09:04:49.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.486+0000 7f678e7fc640 1 -- 172.21.15.31:0/1731998338 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f6788010ab0 con 0x7f67980fee10 2024-09-16T09:04:49.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.487+0000 7f678e7fc640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f678407af30 0x7f678407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:49.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.487+0000 7f678e7fc640 1 -- 172.21.15.31:0/1731998338 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6739+0+0 (secure 0 0 0) 0x7f678809dd60 con 0x7f67980fee10 2024-09-16T09:04:49.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.487+0000 7f679d6d1640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f678407af30 0x7f678407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:49.489 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.488+0000 7f679d6d1640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f678407af30 0x7f678407d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6794000c00 tx=0x7f67940314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:49.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.491+0000 7f678e7fc640 1 -- 172.21.15.31:0/1731998338 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6788060ed0 con 0x7f67980fee10 2024-09-16T09:04:49.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.696+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f679802c590 con 0x7f678407af30 2024-09-16T09:04:49.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.698+0000 7f678e7fc640 1 -- 172.21.15.31:0/1731998338 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f679802c590 con 0x7f678407af30 2024-09-16T09:04:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.699+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f678407af30 msgr2=0x7f678407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f678407af30 0x7f678407d3f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6794000c00 tx=0x7f67940314e0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fee10 msgr2=0x7f6798197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fee10 0x7f6798197d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f678800d900 tx=0x7f678800ddc0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 shutdown_connections 2024-09-16T09:04:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f678407af30 0x7f678407d3f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67980fee10 0x7f6798197d80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 --2- 172.21.15.31:0/1731998338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67980fe4d0 0x7f6798197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 >> 172.21.15.31:0/1731998338 conn(0x7f67980fa280 msgr2=0x7f6798107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:49.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 shutdown_connections 2024-09-16T09:04:49.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.700+0000 7f679f935640 1 -- 172.21.15.31:0/1731998338 wait complete. 2024-09-16T09:04:49.710 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:04:49.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 -- 172.21.15.31:0/1718606035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 msgr2=0x7fa630102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:49.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 --2- 172.21.15.31:0/1718606035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630102a80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa620009880 tx=0x7fa62002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.790 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 -- 172.21.15.31:0/1718606035 shutdown_connections 2024-09-16T09:04:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 --2- 172.21.15.31:0/1718606035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa630102fc0 0x7fa6301053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 --2- 172.21.15.31:0/1718606035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630102a80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 -- 172.21.15.31:0/1718606035 >> 172.21.15.31:0/1718606035 conn(0x7fa6300fa230 msgr2=0x7fa6300fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 -- 172.21.15.31:0/1718606035 shutdown_connections 2024-09-16T09:04:49.791 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.789+0000 7fa634e1e640 1 -- 172.21.15.31:0/1718606035 wait complete. 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa634e1e640 1 Processor -- start 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa634e1e640 1 -- start start 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa634e1e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa634e1e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa630102fc0 0x7fa630195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa634e1e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630196160 con 0x7fa630100690 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa634e1e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6301962d0 con 0x7fa630102fc0 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa62e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51918/0 (socket says 172.21.15.31:51918) 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa62e59c640 1 -- 172.21.15.31:0/2676731714 learned_addr learned my addr 172.21.15.31:0/2676731714 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:49.793 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa627fff640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa630102fc0 0x7fa630195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa62e59c640 1 -- 172.21.15.31:0/2676731714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa630102fc0 msgr2=0x7fa630195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa62e59c640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa630102fc0 0x7fa630195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.790+0000 7fa62e59c640 1 -- 172.21.15.31:0/2676731714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa620009560 con 0x7fa630100690 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.791+0000 7fa62e59c640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630195650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa620004650 tx=0x7fa6200055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.791+0000 7fa6277fe640 1 -- 172.21.15.31:0/2676731714 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa620004aa0 con 0x7fa630100690 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.791+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63019ad20 con 0x7fa630100690 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.791+0000 7fa6277fe640 1 -- 172.21.15.31:0/2676731714 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa620005d50 con 0x7fa630100690 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.791+0000 7fa6277fe640 1 -- 172.21.15.31:0/2676731714 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa620003a20 con 0x7fa630100690 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.791+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63019b290 con 0x7fa630100690 2024-09-16T09:04:49.794 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.792+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa630108770 con 0x7fa630100690 2024-09-16T09:04:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.793+0000 7fa6277fe640 1 -- 172.21.15.31:0/2676731714 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fa620004c00 con 0x7fa630100690 2024-09-16T09:04:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.793+0000 7fa6277fe640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa61c07af30 0x7fa61c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.793+0000 7fa6277fe640 1 -- 172.21.15.31:0/2676731714 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6739+0+0 (secure 0 0 0) 0x7fa6200bf0c0 con 0x7fa630100690 2024-09-16T09:04:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.793+0000 7fa627fff640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa61c07af30 0x7fa61c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:49.795 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.794+0000 7fa627fff640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa61c07af30 0x7fa61c07d3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa630196b20 tx=0x7fa618005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:49.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:49.797+0000 7fa6277fe640 1 -- 172.21.15.31:0/2676731714 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fa620082230 con 0x7fa630100690 2024-09-16T09:04:50.012 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.011+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6301039e0 con 0x7fa61c07af30 2024-09-16T09:04:50.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:49 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] up:boot 2024-09-16T09:04:50.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:49 smithi031 ceph-mon[91884]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:replay} 3 up:standby 2024-09-16T09:04:50.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"} : dispatch 2024-09-16T09:04:50.012 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:50.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:50.013 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:49 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:50.013 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.012+0000 7fa6277fe640 1 -- 172.21.15.31:0/2676731714 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa6301039e0 con 0x7fa61c07af30 2024-09-16T09:04:50.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa61c07af30 msgr2=0x7fa61c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa61c07af30 0x7fa61c07d3f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa630196b20 tx=0x7fa618005f90 comp rx=0 tx=0).stop 2024-09-16T09:04:50.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 msgr2=0x7fa630195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630195650 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa620004650 tx=0x7fa6200055a0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.016 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 shutdown_connections 2024-09-16T09:04:50.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa61c07af30 0x7fa61c07d3f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa630102fc0 0x7fa630195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 --2- 172.21.15.31:0/2676731714 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa630100690 0x7fa630195650 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 >> 172.21.15.31:0/2676731714 conn(0x7fa6300fa230 msgr2=0x7fa6300fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:50.017 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 shutdown_connections 2024-09-16T09:04:50.018 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.014+0000 7fa634e1e640 1 -- 172.21.15.31:0/2676731714 wait complete. 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 -- 172.21.15.31:0/1896843929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770100610 msgr2=0x7f1770102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 --2- 172.21.15.31:0/1896843929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770100610 0x7f1770102a00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f17600098b0 tx=0x7f176002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 -- 172.21.15.31:0/1896843929 shutdown_connections 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 --2- 172.21.15.31:0/1896843929 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1770102f40 0x7f1770105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 --2- 172.21.15.31:0/1896843929 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770100610 0x7f1770102a00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 -- 172.21.15.31:0/1896843929 >> 172.21.15.31:0/1896843929 conn(0x7f17700fa230 msgr2=0x7f17700fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 -- 172.21.15.31:0/1896843929 shutdown_connections 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.099+0000 7f1776937640 1 -- 172.21.15.31:0/1896843929 wait complete. 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f1776937640 1 Processor -- start 2024-09-16T09:04:50.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f1776937640 1 -- start start 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f1776937640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1770100610 0x7f17700fff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f1776937640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770102f40 0x7f17700fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f1776937640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17700feb00 con 0x7f1770102f40 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f1776937640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17700fec70 con 0x7f1770100610 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f176f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770102f40 0x7f17700fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f176f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770102f40 0x7f17700fe5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51926/0 (socket says 172.21.15.31:51926) 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f176f7fe640 1 -- 172.21.15.31:0/3843652954 learned_addr learned my addr 172.21.15.31:0/3843652954 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f176ffff640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1770100610 0x7f17700fff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f176f7fe640 1 -- 172.21.15.31:0/3843652954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1770100610 msgr2=0x7f17700fff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f176f7fe640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1770100610 0x7f17700fff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.100+0000 7f176f7fe640 1 -- 172.21.15.31:0/3843652954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1760009560 con 0x7f1770102f40 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.101+0000 7f176f7fe640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770102f40 0x7f17700fe5c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f176400c8f0 tx=0x7f176400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.101+0000 7f176d7fa640 1 -- 172.21.15.31:0/3843652954 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1764007b80 con 0x7f1770102f40 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.101+0000 7f176d7fa640 1 -- 172.21.15.31:0/3843652954 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1764007ce0 con 0x7f1770102f40 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.101+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17700fef50 con 0x7f1770102f40 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.101+0000 7f176d7fa640 1 -- 172.21.15.31:0/3843652954 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17640187b0 con 0x7f1770102f40 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.101+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177019d700 con 0x7f1770102f40 2024-09-16T09:04:50.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.102+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17701087c0 con 0x7f1770102f40 2024-09-16T09:04:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.103+0000 7f176d7fa640 1 -- 172.21.15.31:0/3843652954 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f176400f450 con 0x7f1770102f40 2024-09-16T09:04:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.103+0000 7f176d7fa640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f175807af30 0x7f175807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.103+0000 7f176d7fa640 1 -- 172.21.15.31:0/3843652954 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6739+0+0 (secure 0 0 0) 0x7f176409e0c0 con 0x7f1770102f40 2024-09-16T09:04:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.103+0000 7f176ffff640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f175807af30 0x7f175807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.104+0000 7f176ffff640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f175807af30 0x7f175807d3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1760006010 tx=0x7f1760031500 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:50.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.108+0000 7f176d7fa640 1 -- 172.21.15.31:0/3843652954 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f1764061200 con 0x7f1770102f40 2024-09-16T09:04:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:49 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] up:boot 2024-09-16T09:04:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:49 smithi097 ceph-mon[79892]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:replay} 3 up:standby 2024-09-16T09:04:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi031.mjhxgi"} : dispatch 2024-09-16T09:04:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:50.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:49 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:50.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.305+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1770103960 con 0x7f175807af30 2024-09-16T09:04:50.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.314+0000 7f176d7fa640 1 -- 172.21.15.31:0/3843652954 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1770103960 con 0x7f175807af30 2024-09-16T09:04:50.317 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:04:50.318 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (14m) 14s ago 16m 25.7M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:04:50.318 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (7m) 14s ago 16m 8887k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:04:50.318 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (7m) 53s ago 15m 8674k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:04:50.318 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (4m) 14s ago 16m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:04:50.318 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (4m) 53s ago 15m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:04:50.319 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (14m) 14s ago 16m 87.0M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:04:50.319 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (20s) 14s ago 10m 10.9M - 19.3.0-4917-g8293d73f 425e316ba8eb 7e4ca0c031e9 2024-09-16T09:04:50.319 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 starting - - - - 2024-09-16T09:04:50.319 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (10m) 53s ago 10m 27.0M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 3280071f054c 2024-09-16T09:04:50.319 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (9m) 53s ago 9m 18.9M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:04:50.320 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (6m) 14s ago 17m 520M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:04:50.320 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (5m) 53s ago 15m 455M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:04:50.320 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (5m) 14s ago 17m 50.7M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:04:50.320 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (5m) 53s ago 15m 38.9M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:04:50.320 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (16m) 14s ago 16m 15.6M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:04:50.321 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (15m) 53s ago 15m 16.7M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:04:50.321 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 14s ago 13m 114M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:04:50.321 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (3m) 14s ago 13m 105M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:04:50.321 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (2m) 14s ago 12m 123M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:04:50.321 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (2m) 53s ago 12m 140M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:04:50.322 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (106s) 53s ago 12m 94.6M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:04:50.322 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (56s) 53s ago 11m 14.3M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:04:50.322 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (5m) 14s ago 15m 65.4M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:04:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f175807af30 msgr2=0x7f175807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f175807af30 0x7f175807d3f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1760006010 tx=0x7f1760031500 comp rx=0 tx=0).stop 2024-09-16T09:04:50.323 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770102f40 msgr2=0x7f17700fe5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770102f40 0x7f17700fe5c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f176400c8f0 tx=0x7f176400cdb0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 shutdown_connections 2024-09-16T09:04:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f175807af30 0x7f175807d3f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1770102f40 0x7f17700fe5c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 --2- 172.21.15.31:0/3843652954 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1770100610 0x7f17700fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.316+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 >> 172.21.15.31:0/3843652954 conn(0x7f17700fa230 msgr2=0x7f17700fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.317+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 shutdown_connections 2024-09-16T09:04:50.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.317+0000 7f1776937640 1 -- 172.21.15.31:0/3843652954 wait complete. 2024-09-16T09:04:50.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 -- 172.21.15.31:0/3898637702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 msgr2=0x7ff5c80fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/3898637702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c80fff90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009c20 tx=0x7ff5c402a970 comp rx=0 tx=0).stop 2024-09-16T09:04:50.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 -- 172.21.15.31:0/3898637702 shutdown_connections 2024-09-16T09:04:50.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/3898637702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c8100d90 0x7ff5c8101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/3898637702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c80fff90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 -- 172.21.15.31:0/3898637702 >> 172.21.15.31:0/3898637702 conn(0x7ff5c80fb320 msgr2=0x7ff5c80fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:50.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 -- 172.21.15.31:0/3898637702 shutdown_connections 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.400+0000 7ff5cfde9640 1 -- 172.21.15.31:0/3898637702 wait complete. 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cfde9640 1 Processor -- start 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cfde9640 1 -- start start 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cfde9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c8195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cfde9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c8100d90 0x7ff5c8195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cfde9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8196220 con 0x7ff5c80ffb90 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cfde9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c8196390 con 0x7ff5c8100d90 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cdb85640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c8195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cdb85640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c8195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51936/0 (socket says 172.21.15.31:51936) 2024-09-16T09:04:50.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cdb85640 1 -- 172.21.15.31:0/1973521696 learned_addr learned my addr 172.21.15.31:0/1973521696 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:50.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cd384640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c8100d90 0x7ff5c8195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cdb85640 1 -- 172.21.15.31:0/1973521696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c8100d90 msgr2=0x7ff5c8195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cdb85640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c8100d90 0x7ff5c8195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.401+0000 7ff5cdb85640 1 -- 172.21.15.31:0/1973521696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c4009900 con 0x7ff5c80ffb90 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.402+0000 7ff5cdb85640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c8195710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5c402f600 tx=0x7ff5c402f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.402+0000 7ff5beffd640 1 -- 172.21.15.31:0/1973521696 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c4030840 con 0x7ff5c80ffb90 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.402+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c819add0 con 0x7ff5c80ffb90 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.402+0000 7ff5beffd640 1 -- 172.21.15.31:0/1973521696 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5c4004040 con 0x7ff5c80ffb90 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.402+0000 7ff5beffd640 1 -- 172.21.15.31:0/1973521696 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5c403a6b0 con 0x7ff5c80ffb90 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.402+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c819b2e0 con 0x7ff5c80ffb90 2024-09-16T09:04:50.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.403+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8108800 con 0x7ff5c80ffb90 2024-09-16T09:04:50.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.403+0000 7ff5beffd640 1 -- 172.21.15.31:0/1973521696 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7ff5c400a040 con 0x7ff5c80ffb90 2024-09-16T09:04:50.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.404+0000 7ff5beffd640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff5b407af30 0x7ff5b407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.404+0000 7ff5cd384640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff5b407af30 0x7ff5b407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.404+0000 7ff5beffd640 1 -- 172.21.15.31:0/1973521696 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6739+0+0 (secure 0 0 0) 0x7ff5c402c040 con 0x7ff5c80ffb90 2024-09-16T09:04:50.407 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.405+0000 7ff5cd384640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff5b407af30 0x7ff5b407d3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8196c30 tx=0x7ff5b8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:50.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.408+0000 7ff5beffd640 1 -- 172.21.15.31:0/1973521696 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ff5c40829c0 con 0x7ff5c80ffb90 2024-09-16T09:04:50.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.684+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff5c8108990 con 0x7ff5c80ffb90 2024-09-16T09:04:50.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.685+0000 7ff5beffd640 1 -- 172.21.15.31:0/1973521696 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+833 (secure 0 0 0) 0x7ff5c4085ae0 con 0x7ff5c80ffb90 2024-09-16T09:04:50.686 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:04:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:04:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:04:50.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2, 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 18.2.4-853-gae7f3733 (ae7f37330c1a72493684eb1fcaed6f2a01195bb6) reef (stable)": 2, 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 12 2024-09-16T09:04:50.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:04:50.688 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.687+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff5b407af30 msgr2=0x7ff5b407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff5b407af30 0x7ff5b407d3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8196c30 tx=0x7ff5b8008040 comp rx=0 tx=0).stop 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 msgr2=0x7ff5c8195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c8195710 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5c402f600 tx=0x7ff5c402f630 comp rx=0 tx=0).stop 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 shutdown_connections 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff5b407af30 0x7ff5b407d3f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff5c8100d90 0x7ff5c8195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 --2- 172.21.15.31:0/1973521696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5c80ffb90 0x7ff5c8195710 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 >> 172.21.15.31:0/1973521696 conn(0x7ff5c80fb320 msgr2=0x7ff5c8103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 shutdown_connections 2024-09-16T09:04:50.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.688+0000 7ff5cfde9640 1 -- 172.21.15.31:0/1973521696 wait complete. 2024-09-16T09:04:50.777 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 -- 172.21.15.31:0/669683390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 msgr2=0x7f720c105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 --2- 172.21.15.31:0/669683390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c105330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f72000098b0 tx=0x7f720002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 -- 172.21.15.31:0/669683390 shutdown_connections 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 --2- 172.21.15.31:0/669683390 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c105330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 --2- 172.21.15.31:0/669683390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c100610 0x7f720c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 -- 172.21.15.31:0/669683390 >> 172.21.15.31:0/669683390 conn(0x7f720c0fa230 msgr2=0x7f720c0fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 -- 172.21.15.31:0/669683390 shutdown_connections 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 -- 172.21.15.31:0/669683390 wait complete. 2024-09-16T09:04:50.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 Processor -- start 2024-09-16T09:04:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.776+0000 7f7212552640 1 -- start start 2024-09-16T09:04:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f7212552640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c100610 0x7f720c1956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f7212552640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f7212552640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c1961e0 con 0x7f720c102f40 2024-09-16T09:04:50.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f7212552640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c196350 con 0x7f720c100610 2024-09-16T09:04:50.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c195c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51956/0 (socket says 172.21.15.31:51956) 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720b7fe640 1 -- 172.21.15.31:0/3430218320 learned_addr learned my addr 172.21.15.31:0/3430218320 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720bfff640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c100610 0x7f720c1956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720b7fe640 1 -- 172.21.15.31:0/3430218320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c100610 msgr2=0x7f720c1956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720b7fe640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c100610 0x7f720c1956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720b7fe640 1 -- 172.21.15.31:0/3430218320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7200009560 con 0x7f720c102f40 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.777+0000 7f720b7fe640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c195c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7200009230 tx=0x7f72000054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.778+0000 7f72097fa640 1 -- 172.21.15.31:0/3430218320 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7200004aa0 con 0x7f720c102f40 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.778+0000 7f72097fa640 1 -- 172.21.15.31:0/3430218320 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7200004c00 con 0x7f720c102f40 2024-09-16T09:04:50.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.778+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720c19ad40 con 0x7f720c102f40 2024-09-16T09:04:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.778+0000 7f72097fa640 1 -- 172.21.15.31:0/3430218320 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7200003a80 con 0x7f720c102f40 2024-09-16T09:04:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.778+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720c19b2b0 con 0x7f720c102f40 2024-09-16T09:04:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.779+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720c108790 con 0x7f720c102f40 2024-09-16T09:04:50.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.780+0000 7f72097fa640 1 -- 172.21.15.31:0/3430218320 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f7200031c90 con 0x7f720c102f40 2024-09-16T09:04:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.780+0000 7f72097fa640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f71f407af30 0x7f71f407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.780+0000 7f72097fa640 1 -- 172.21.15.31:0/3430218320 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6739+0+0 (secure 0 0 0) 0x7f72000bf130 con 0x7f720c102f40 2024-09-16T09:04:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.780+0000 7f720bfff640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f71f407af30 0x7f71f407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:50.783 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.781+0000 7f720bfff640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f71f407af30 0x7f71f407d3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005fd0 tx=0x7f71fc00b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:50.785 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:50.785+0000 7f72097fa640 1 -- 172.21.15.31:0/3430218320 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f7200082320 con 0x7f720c102f40 2024-09-16T09:04:51.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:50 smithi031 ceph-mon[91884]: pgmap v213: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-09-16T09:04:51.015 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:50 smithi031 ceph-mon[91884]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:51.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:50 smithi031 ceph-mon[91884]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:51.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:50 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1973521696' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:51.019 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.018+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f720c108920 con 0x7f720c102f40 2024-09-16T09:04:51.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.019+0000 7f72097fa640 1 -- 172.21.15.31:0/3430218320 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1922 (secure 0 0 0) 0x7f7200085440 con 0x7f720c102f40 2024-09-16T09:04:51.020 INFO:teuthology.orchestra.run.smithi031.stdout:e27 2024-09-16T09:04:51.020 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T09:04:50:834523+0000 2024-09-16T09:04:51.020 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:04:51.020 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:04:51.020 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:04:51.020 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 27 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T09:04:49.876319+0000 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:04:51.021 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:04:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 97 2024-09-16T09:04:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:04:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-09-16T09:04:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-09-16T09:04:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=24337} 2024-09-16T09:04:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:04:51.022 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 1 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{0:24337} state up:reconnect seq 151 join_fscid=1 addr [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:04:51.023 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:04:51.024 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{-1:34434} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:04:51.024 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{-1:44323} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6824/1475349315,v1:172.21.15.97:6825/1475349315] compat {c=[1],r=[1],i=[7ff]}] 2024-09-16T09:04:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-16T09:04:51.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f71f407af30 msgr2=0x7f71f407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f71f407af30 0x7f71f407d3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f71fc005fd0 tx=0x7f71fc00b390 comp rx=0 tx=0).stop 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 msgr2=0x7f720c195c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c195c10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7200009230 tx=0x7f72000054f0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 shutdown_connections 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f71f407af30 0x7f71f407d3f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f720c102f40 0x7f720c195c10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 --2- 172.21.15.31:0/3430218320 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f720c100610 0x7f720c1956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 >> 172.21.15.31:0/3430218320 conn(0x7f720c0fa230 msgr2=0x7f720c0fc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 shutdown_connections 2024-09-16T09:04:51.025 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.021+0000 7f7212552640 1 -- 172.21.15.31:0/3430218320 wait complete. 2024-09-16T09:04:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.106+0000 7f06baf1d640 1 -- 172.21.15.31:0/1800957135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40ffcf0 msgr2=0x7f06b4100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.106+0000 7f06baf1d640 1 --2- 172.21.15.31:0/1800957135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40ffcf0 0x7f06b4100170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009880 tx=0x7f06a802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 -- 172.21.15.31:0/1800957135 shutdown_connections 2024-09-16T09:04:51.110 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 --2- 172.21.15.31:0/1800957135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40ffcf0 0x7f06b4100170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 --2- 172.21.15.31:0/1800957135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06b40feaf0 0x7f06b40feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 -- 172.21.15.31:0/1800957135 >> 172.21.15.31:0/1800957135 conn(0x7f06b40fa280 msgr2=0x7f06b40fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:51.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 -- 172.21.15.31:0/1800957135 shutdown_connections 2024-09-16T09:04:51.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 -- 172.21.15.31:0/1800957135 wait complete. 2024-09-16T09:04:51.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 Processor -- start 2024-09-16T09:04:51.111 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 -- start start 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40feaf0 0x7f06b4068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06b40ffcf0 0x7f06b419a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419a910 con 0x7f06b40feaf0 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.107+0000 7f06baf1d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b419aa50 con 0x7f06b40ffcf0 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b8cb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40feaf0 0x7f06b4068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b8cb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40feaf0 0x7f06b4068790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:51974/0 (socket says 172.21.15.31:51974) 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b8cb9640 1 -- 172.21.15.31:0/4116810170 learned_addr learned my addr 172.21.15.31:0/4116810170 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b3fff640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06b40ffcf0 0x7f06b419a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b8cb9640 1 -- 172.21.15.31:0/4116810170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06b40ffcf0 msgr2=0x7f06b419a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b8cb9640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06b40ffcf0 0x7f06b419a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.112 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b8cb9640 1 -- 172.21.15.31:0/4116810170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8009560 con 0x7f06b40feaf0 2024-09-16T09:04:51.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b8cb9640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40feaf0 0x7f06b4068790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06a400c8f0 tx=0x7f06a400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:51.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b1ffb640 1 -- 172.21.15.31:0/4116810170 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a40041f0 con 0x7f06b40feaf0 2024-09-16T09:04:51.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b1ffb640 1 -- 172.21.15.31:0/4116810170 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06a4007e10 con 0x7f06b40feaf0 2024-09-16T09:04:51.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06b1ffb640 1 -- 172.21.15.31:0/4116810170 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a400f450 con 0x7f06b40feaf0 2024-09-16T09:04:51.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.108+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b419ad00 con 0x7f06b40feaf0 2024-09-16T09:04:51.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.109+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b419b220 con 0x7f06b40feaf0 2024-09-16T09:04:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.110+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b40feef0 con 0x7f06b40feaf0 2024-09-16T09:04:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.110+0000 7f06b1ffb640 1 -- 172.21.15.31:0/4116810170 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f06a4007950 con 0x7f06b40feaf0 2024-09-16T09:04:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.111+0000 7f06b1ffb640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f069c07af30 0x7f069c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.111+0000 7f06b1ffb640 1 -- 172.21.15.31:0/4116810170 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6739+0+0 (secure 0 0 0) 0x7f06a4013070 con 0x7f06b40feaf0 2024-09-16T09:04:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.111+0000 7f06b3fff640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f069c07af30 0x7f069c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:51.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.111+0000 7f06b3fff640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f069c07af30 0x7f069c07d3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06a8000c00 tx=0x7f06a80314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:51.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.115+0000 7f06b1ffb640 1 -- 172.21.15.31:0/4116810170 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f06a4061140 con 0x7f06b40feaf0 2024-09-16T09:04:51.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:50 smithi097 ceph-mon[79892]: pgmap v213: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 5 op/s 2024-09-16T09:04:51.164 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:50 smithi097 ceph-mon[79892]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:51.164 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:50 smithi097 ceph-mon[79892]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:51.164 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:50 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1973521696' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:51.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.318+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06b402c540 con 0x7f069c07af30 2024-09-16T09:04:51.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.320+0000 7f06b1ffb640 1 -- 172.21.15.31:0/4116810170 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f06b402c540 con 0x7f069c07af30 2024-09-16T09:04:51.321 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:04:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:04:51.321 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:04:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:04:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:04:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:04:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:04:51.322 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-09-16T09:04:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "osd" 2024-09-16T09:04:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:04:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "13/23 daemons upgraded", 2024-09-16T09:04:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T09:04:51.323 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:04:51.323 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:04:51.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f069c07af30 msgr2=0x7f069c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f069c07af30 0x7f069c07d3f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f06a8000c00 tx=0x7f06a80314e0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40feaf0 msgr2=0x7f06b4068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40feaf0 0x7f06b4068790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06a400c8f0 tx=0x7f06a400cdb0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 shutdown_connections 2024-09-16T09:04:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f069c07af30 0x7f069c07d3f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f06b40ffcf0 0x7f06b419a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 --2- 172.21.15.31:0/4116810170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f06b40feaf0 0x7f06b4068790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 >> 172.21.15.31:0/4116810170 conn(0x7f06b40fa280 msgr2=0x7f06b41033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 shutdown_connections 2024-09-16T09:04:51.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.322+0000 7f06baf1d640 1 -- 172.21.15.31:0/4116810170 wait complete. 2024-09-16T09:04:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 -- 172.21.15.31:0/1243762284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234100630 msgr2=0x7f6234102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 --2- 172.21.15.31:0/1243762284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234100630 0x7f6234102a20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62300098b0 tx=0x7f623002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 -- 172.21.15.31:0/1243762284 shutdown_connections 2024-09-16T09:04:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 --2- 172.21.15.31:0/1243762284 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6234102f60 0x7f6234105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 --2- 172.21.15.31:0/1243762284 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234100630 0x7f6234102a20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 -- 172.21.15.31:0/1243762284 >> 172.21.15.31:0/1243762284 conn(0x7f62340fa230 msgr2=0x7f62340fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 -- 172.21.15.31:0/1243762284 shutdown_connections 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 -- 172.21.15.31:0/1243762284 wait complete. 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.412+0000 7f623c2d8640 1 Processor -- start 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f623c2d8640 1 -- start start 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f623c2d8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6234100630 0x7f62340fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f623c2d8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234102f60 0x7f62340fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f623c2d8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340feb20 con 0x7f6234102f60 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f623c2d8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340fec90 con 0x7f6234100630 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f6239873640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234102f60 0x7f62340fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:51.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f6239873640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234102f60 0x7f62340fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:60038/0 (socket says 172.21.15.31:60038) 2024-09-16T09:04:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f6239873640 1 -- 172.21.15.31:0/3773936039 learned_addr learned my addr 172.21.15.31:0/3773936039 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:04:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f623a074640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6234100630 0x7f62340fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f6239873640 1 -- 172.21.15.31:0/3773936039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6234100630 msgr2=0x7f62340fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f6239873640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6234100630 0x7f62340fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.413+0000 7f6239873640 1 -- 172.21.15.31:0/3773936039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230009560 con 0x7f6234102f60 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.414+0000 7f6239873640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234102f60 0x7f62340fe5e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f622400c8f0 tx=0x7f622400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.414+0000 7f622b7fe640 1 -- 172.21.15.31:0/3773936039 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6224007b80 con 0x7f6234102f60 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.414+0000 7f622b7fe640 1 -- 172.21.15.31:0/3773936039 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6224007ce0 con 0x7f6234102f60 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.414+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62340fef70 con 0x7f6234102f60 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.414+0000 7f622b7fe640 1 -- 172.21.15.31:0/3773936039 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62240187b0 con 0x7f6234102f60 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.414+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623419d720 con 0x7f6234102f60 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.415+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62341087e0 con 0x7f6234102f60 2024-09-16T09:04:51.417 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.416+0000 7f622b7fe640 1 -- 172.21.15.31:0/3773936039 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f622400f450 con 0x7f6234102f60 2024-09-16T09:04:51.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.416+0000 7f622b7fe640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622007af30 0x7f622007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:04:51.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.417+0000 7f622b7fe640 1 -- 172.21.15.31:0/3773936039 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6739+0+0 (secure 0 0 0) 0x7f622409e390 con 0x7f6234102f60 2024-09-16T09:04:51.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.417+0000 7f623a074640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622007af30 0x7f622007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:04:51.418 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.417+0000 7f623a074640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622007af30 0x7f622007d3f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6230006010 tx=0x7f62300314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:04:51.422 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.421+0000 7f622b7fe640 1 -- 172.21.15.31:0/3773936039 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6224061570 con 0x7f6234102f60 2024-09-16T09:04:51.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.684+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f62341089a0 con 0x7f6234102f60 2024-09-16T09:04:51.685 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.684+0000 7f622b7fe640 1 -- 172.21.15.31:0/3773936039 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f6224064690 con 0x7f6234102f60 2024-09-16T09:04:51.685 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem is degraded 2024-09-16T09:04:51.686 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T09:04:51.686 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs is degraded 2024-09-16T09:04:51.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622007af30 msgr2=0x7f622007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.687 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622007af30 0x7f622007d3f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6230006010 tx=0x7f62300314e0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234102f60 msgr2=0x7f62340fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234102f60 0x7f62340fe5e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f622400c8f0 tx=0x7f622400cdb0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 shutdown_connections 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f622007af30 0x7f622007d3f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6234102f60 0x7f62340fe5e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 --2- 172.21.15.31:0/3773936039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6234100630 0x7f62340fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 >> 172.21.15.31:0/3773936039 conn(0x7f62340fa230 msgr2=0x7f62340fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 shutdown_connections 2024-09-16T09:04:51.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:04:51.686+0000 7f623c2d8640 1 -- 172.21.15.31:0/3773936039 wait complete. 2024-09-16T09:04:51.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:51 smithi031 ceph-mon[91884]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:51.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:51 smithi031 ceph-mon[91884]: reconnect by client.24369 v1:192.168.0.1:0/2254071746 after 0 2024-09-16T09:04:51.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:51 smithi031 ceph-mon[91884]: reconnect by client.24365 v1:172.21.15.31:0/3242977461 after 0 2024-09-16T09:04:51.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:51 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:reconnect 2024-09-16T09:04:51.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:51 smithi031 ceph-mon[91884]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:reconnect} 3 up:standby 2024-09-16T09:04:51.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:51 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3430218320' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:04:51.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:51 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3773936039' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:04:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:51 smithi097 ceph-mon[79892]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:51 smithi097 ceph-mon[79892]: reconnect by client.24369 v1:192.168.0.1:0/2254071746 after 0 2024-09-16T09:04:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:51 smithi097 ceph-mon[79892]: reconnect by client.24365 v1:172.21.15.31:0/3242977461 after 0 2024-09-16T09:04:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:51 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:reconnect 2024-09-16T09:04:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:51 smithi097 ceph-mon[79892]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:reconnect} 3 up:standby 2024-09-16T09:04:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:51 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3430218320' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:04:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:51 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3773936039' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:04:53.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:52 smithi031 ceph-mon[91884]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:53.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:52 smithi031 ceph-mon[91884]: pgmap v214: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-09-16T09:04:53.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:52 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:rejoin 2024-09-16T09:04:53.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:52 smithi031 ceph-mon[91884]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:rejoin} 3 up:standby 2024-09-16T09:04:53.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:52 smithi031 ceph-mon[91884]: daemon mds.cephfs.smithi097.sbwmfq is now active in filesystem cephfs as rank 0 2024-09-16T09:04:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:52 smithi097 ceph-mon[79892]: from='client.34458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:04:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:52 smithi097 ceph-mon[79892]: pgmap v214: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 6 op/s 2024-09-16T09:04:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:52 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:rejoin 2024-09-16T09:04:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:52 smithi097 ceph-mon[79892]: fsmap cephfs:1/1 {0=cephfs.smithi097.sbwmfq=up:rejoin} 3 up:standby 2024-09-16T09:04:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:52 smithi097 ceph-mon[79892]: daemon mds.cephfs.smithi097.sbwmfq is now active in filesystem cephfs as rank 0 2024-09-16T09:04:54.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:53 smithi097 ceph-mon[79892]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T09:04:54.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:53 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:04:54.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:53 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:active 2024-09-16T09:04:54.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:53 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi097.sbwmfq=up:active} 3 up:standby 2024-09-16T09:04:54.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:54.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:53 smithi031 ceph-mon[91884]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T09:04:54.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:53 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:04:54.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:53 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.97:6826/859148422,v1:172.21.15.97:6827/859148422] up:active 2024-09-16T09:04:54.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:53 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi097.sbwmfq=up:active} 3 up:standby 2024-09-16T09:04:54.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:04:54.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:54.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:55.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:54 smithi031 ceph-mon[91884]: pgmap v215: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 96 B/s wr, 6 op/s 2024-09-16T09:04:55.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:55.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:55.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:54 smithi097 ceph-mon[79892]: pgmap v215: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 96 B/s wr, 6 op/s 2024-09-16T09:04:55.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:55.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:56 smithi031 ceph-mon[91884]: pgmap v216: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T09:04:57.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:56 smithi097 ceph-mon[79892]: pgmap v216: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T09:04:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: pgmap v217: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T09:04:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:59.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi097.cagmus"]} : dispatch 2024-09-16T09:04:59.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:04:58 smithi097 ceph-mon[79892]: Upgrade: It appears safe to stop mds.cephfs.smithi097.cagmus 2024-09-16T09:04:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: pgmap v217: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 14 MiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T09:04:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:04:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi097.cagmus"]} : dispatch 2024-09-16T09:04:59.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:04:58 smithi031 ceph-mon[91884]: Upgrade: It appears safe to stop mds.cephfs.smithi097.cagmus 2024-09-16T09:05:00.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:00 smithi097 ceph-mon[79892]: pgmap v218: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T09:05:00.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:00 smithi097 ceph-mon[79892]: Upgrade: Updating mds.cephfs.smithi097.cagmus 2024-09-16T09:05:00.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:00.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:05:00.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:00 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:00.740 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:00 smithi097 ceph-mon[79892]: Deploying daemon mds.cephfs.smithi097.cagmus on smithi097 2024-09-16T09:05:00.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:00 smithi031 ceph-mon[91884]: pgmap v218: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 7.1 MiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T09:05:00.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:00 smithi031 ceph-mon[91884]: Upgrade: Updating mds.cephfs.smithi097.cagmus 2024-09-16T09:05:00.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:00.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.cagmus", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:05:00.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:00 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:00.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:00 smithi031 ceph-mon[91884]: Deploying daemon mds.cephfs.smithi097.cagmus on smithi097 2024-09-16T09:05:02.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:02 smithi097 ceph-mon[79892]: pgmap v219: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-16T09:05:02.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:02 smithi097 ceph-mon[79892]: osdmap e98: 6 total, 6 up, 6 in 2024-09-16T09:05:02.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:02 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi097.sbwmfq=up:active} 2 up:standby 2024-09-16T09:05:02.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:02 smithi031 ceph-mon[91884]: pgmap v219: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 3.1 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-16T09:05:02.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:02 smithi031 ceph-mon[91884]: osdmap e98: 6 total, 6 up, 6 in 2024-09-16T09:05:02.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:02 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi097.sbwmfq=up:active} 2 up:standby 2024-09-16T09:05:04.722 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:04 smithi031 ceph-mon[91884]: pgmap v221: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-09-16T09:05:04.753 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:04 smithi097 ceph-mon[79892]: pgmap v221: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 3.3 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-09-16T09:05:06.729 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:06 smithi097 ceph-mon[79892]: pgmap v222: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-09-16T09:05:06.729 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:06.730 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:06.730 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:06 smithi031 ceph-mon[91884]: pgmap v222: 65 pgs: 65 active+clean; 201 MiB data, 872 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 5.5 KiB/s wr, 5 op/s 2024-09-16T09:05:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:06.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:07.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:07 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.97:6824/257407185,v1:172.21.15.97:6825/257407185] up:boot 2024-09-16T09:05:07.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:07 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi097.sbwmfq=up:active} 3 up:standby 2024-09-16T09:05:07.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"} : dispatch 2024-09-16T09:05:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:07 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.97:6824/257407185,v1:172.21.15.97:6825/257407185] up:boot 2024-09-16T09:05:07.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:07 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi097.sbwmfq=up:active} 3 up:standby 2024-09-16T09:05:07.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.cagmus"} : dispatch 2024-09-16T09:05:08.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:08 smithi097 ceph-mon[79892]: pgmap v223: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 5.1 KiB/s wr, 2 op/s 2024-09-16T09:05:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:08 smithi031 ceph-mon[91884]: pgmap v223: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 5.1 KiB/s wr, 2 op/s 2024-09-16T09:05:09.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:09.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:09.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:09.803 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:09.803 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:09.803 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:10.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:10 smithi097 ceph-mon[79892]: pgmap v224: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 5.1 KiB/s wr, 2 op/s 2024-09-16T09:05:10.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:10.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:10 smithi031 ceph-mon[91884]: pgmap v224: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 5.1 KiB/s wr, 2 op/s 2024-09-16T09:05:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:12.682 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:12 smithi097 ceph-mon[79892]: pgmap v225: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:12.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:12 smithi031 ceph-mon[91884]: pgmap v225: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:13.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:13.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi097.sbwmfq"]} : dispatch 2024-09-16T09:05:13.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:13 smithi097 ceph-mon[79892]: Upgrade: It appears safe to stop mds.cephfs.smithi097.sbwmfq 2024-09-16T09:05:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi097.sbwmfq"]} : dispatch 2024-09-16T09:05:14.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:13 smithi031 ceph-mon[91884]: Upgrade: It appears safe to stop mds.cephfs.smithi097.sbwmfq 2024-09-16T09:05:14.833 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:14 smithi097 ceph-mon[79892]: pgmap v226: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:14.833 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:14.833 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:05:14.833 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:14 smithi031 ceph-mon[91884]: pgmap v226: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi097.sbwmfq", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-16T09:05:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:15.881 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:15 smithi097 ceph-mon[79892]: Upgrade: Updating mds.cephfs.smithi097.sbwmfq 2024-09-16T09:05:15.881 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:15 smithi097 ceph-mon[79892]: Deploying daemon mds.cephfs.smithi097.sbwmfq on smithi097 2024-09-16T09:05:16.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:15 smithi031 ceph-mon[91884]: Upgrade: Updating mds.cephfs.smithi097.sbwmfq 2024-09-16T09:05:16.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:15 smithi031 ceph-mon[91884]: Deploying daemon mds.cephfs.smithi097.sbwmfq on smithi097 2024-09-16T09:05:16.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:05:16.161+0000 7f22b0c50640 -1 bdev(0x55760d9b6c00 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 3 2024-09-16T09:05:16.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:05:16.163+0000 7f22ab445640 -1 bdev(0x55760d9b6c00 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-09-16T09:05:16.512 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:05:16.163+0000 7f22b2c54640 -1 bdev(0x55760d9b6c00 /var/lib/ceph/osd/ceph-3/block) aio_submit retries 4 2024-09-16T09:05:16.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[91880]: 2024-09-16T09:05:16.132+0000 7faf88e84640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T09:05:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: pgmap v227: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T09:05:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T09:05:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: osdmap e99: 6 total, 6 up, 6 in 2024-09-16T09:05:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: Standby daemon mds.cephfs.smithi031.lcfutm assigned to filesystem cephfs as rank 0 2024-09-16T09:05:17.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T09:05:17.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-16T09:05:17.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:16 smithi097 ceph-mon[79892]: fsmap cephfs:1/1 {0=cephfs.smithi031.lcfutm=up:replay} 2 up:standby 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: pgmap v227: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: osdmap e99: 6 total, 6 up, 6 in 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: Standby daemon mds.cephfs.smithi031.lcfutm assigned to filesystem cephfs as rank 0 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-09-16T09:05:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:16 smithi031 ceph-mon[91884]: fsmap cephfs:1/1 {0=cephfs.smithi031.lcfutm=up:replay} 2 up:standby 2024-09-16T09:05:19.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:18 smithi097 ceph-mon[79892]: pgmap v229: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-09-16T09:05:19.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:18 smithi031 ceph-mon[91884]: pgmap v229: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 819 KiB/s rd, 0 op/s 2024-09-16T09:05:20.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T09:05:20.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T09:05:21.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:20 smithi097 ceph-mon[79892]: pgmap v230: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 4.6 MiB/s rd, 1 op/s 2024-09-16T09:05:21.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:20 smithi031 ceph-mon[91884]: pgmap v230: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 4.6 MiB/s rd, 1 op/s 2024-09-16T09:05:21.778 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 -- 172.21.15.31:0/3677355287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 msgr2=0x7f2d78101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 --2- 172.21.15.31:0/3677355287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d78101210 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009880 tx=0x7f2d6802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 -- 172.21.15.31:0/3677355287 shutdown_connections 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 --2- 172.21.15.31:0/3677355287 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d78101210 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 --2- 172.21.15.31:0/3677355287 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d780ffb90 0x7f2d780fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 -- 172.21.15.31:0/3677355287 >> 172.21.15.31:0/3677355287 conn(0x7f2d780fb360 msgr2=0x7f2d780fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 -- 172.21.15.31:0/3677355287 shutdown_connections 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.775+0000 7f2d8061c640 1 -- 172.21.15.31:0/3677355287 wait complete. 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d8061c640 1 Processor -- start 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d8061c640 1 -- start start 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d8061c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d780ffb90 0x7f2d78074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:21.779 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d8061c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d780728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d8061c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78074790 con 0x7f2d78100d90 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d8061c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d78072e10 con 0x7f2d780ffb90 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d7dbb7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d780728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d7dbb7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d780728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41914/0 (socket says 172.21.15.31:41914) 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d7dbb7640 1 -- 172.21.15.31:0/413235809 learned_addr learned my addr 172.21.15.31:0/413235809 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d7e3b8640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d780ffb90 0x7f2d78074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d7dbb7640 1 -- 172.21.15.31:0/413235809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d780ffb90 msgr2=0x7f2d78074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d7dbb7640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d780ffb90 0x7f2d78074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.776+0000 7f2d7dbb7640 1 -- 172.21.15.31:0/413235809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d68009560 con 0x7f2d78100d90 2024-09-16T09:05:21.780 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.777+0000 7f2d7dbb7640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d780728a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d68006010 tx=0x7f2d680055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.777+0000 7f2d6f7fe640 1 -- 172.21.15.31:0/413235809 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d68004b20 con 0x7f2d78100d90 2024-09-16T09:05:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.777+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d78073090 con 0x7f2d78100d90 2024-09-16T09:05:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.777+0000 7f2d6f7fe640 1 -- 172.21.15.31:0/413235809 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d68004c80 con 0x7f2d78100d90 2024-09-16T09:05:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.777+0000 7f2d6f7fe640 1 -- 172.21.15.31:0/413235809 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d68003b80 con 0x7f2d78100d90 2024-09-16T09:05:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.777+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d78073600 con 0x7f2d78100d90 2024-09-16T09:05:21.781 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.778+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7818f710 con 0x7f2d78100d90 2024-09-16T09:05:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.779+0000 7f2d6f7fe640 1 -- 172.21.15.31:0/413235809 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f2d68031640 con 0x7f2d78100d90 2024-09-16T09:05:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.780+0000 7f2d6f7fe640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2d6407af30 0x7f2d6407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.780+0000 7f2d6f7fe640 1 -- 172.21.15.31:0/413235809 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f2d680bf1f0 con 0x7f2d78100d90 2024-09-16T09:05:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.780+0000 7f2d7e3b8640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2d6407af30 0x7f2d6407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:21.782 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.780+0000 7f2d7e3b8640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2d6407af30 0x7f2d6407d3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d78100bf0 tx=0x7f2d74005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:21.786 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.784+0000 7f2d6f7fe640 1 -- 172.21.15.31:0/413235809 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f2d68082390 con 0x7f2d78100d90 2024-09-16T09:05:21.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.988+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d7802c450 con 0x7f2d6407af30 2024-09-16T09:05:21.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.989+0000 7f2d6f7fe640 1 -- 172.21.15.31:0/413235809 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2d7802c450 con 0x7f2d6407af30 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2d6407af30 msgr2=0x7f2d6407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2d6407af30 0x7f2d6407d3f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2d78100bf0 tx=0x7f2d74005f90 comp rx=0 tx=0).stop 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 msgr2=0x7f2d780728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d780728a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2d68006010 tx=0x7f2d680055f0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 shutdown_connections 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f2d6407af30 0x7f2d6407d3f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d78100d90 0x7f2d780728a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 --2- 172.21.15.31:0/413235809 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d780ffb90 0x7f2d78074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.991+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 >> 172.21.15.31:0/413235809 conn(0x7f2d780fb360 msgr2=0x7f2d78104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.992+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 shutdown_connections 2024-09-16T09:05:21.995 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:21.992+0000 7f2d8061c640 1 -- 172.21.15.31:0/413235809 wait complete. 2024-09-16T09:05:22.006 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3892036181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fee10 msgr2=0x7f6d700ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3892036181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fee10 0x7f6d700ff270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009880 tx=0x7f6d6402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3892036181 shutdown_connections 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3892036181 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fee10 0x7f6d700ff270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3892036181 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d700fe4d0 0x7f6d700fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3892036181 >> 172.21.15.31:0/3892036181 conn(0x7f6d700fa280 msgr2=0x7f6d700fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3892036181 shutdown_connections 2024-09-16T09:05:22.100 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.097+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3892036181 wait complete. 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.098+0000 7f6d74b2a640 1 Processor -- start 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.098+0000 7f6d74b2a640 1 -- start start 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.098+0000 7f6d74b2a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fe4d0 0x7f6d701978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.098+0000 7f6d74b2a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d700fee10 0x7f6d70197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.098+0000 7f6d74b2a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d701983e0 con 0x7f6d700fe4d0 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.098+0000 7f6d74b2a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70198550 con 0x7f6d700fee10 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fe4d0 0x7f6d701978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fe4d0 0x7f6d701978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41926/0 (socket says 172.21.15.31:41926) 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6e59c640 1 -- 172.21.15.31:0/3112877428 learned_addr learned my addr 172.21.15.31:0/3112877428 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6dd9b640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d700fee10 0x7f6d70197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.102 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6e59c640 1 -- 172.21.15.31:0/3112877428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d700fee10 msgr2=0x7f6d70197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6e59c640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d700fee10 0x7f6d70197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6e59c640 1 -- 172.21.15.31:0/3112877428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d64009560 con 0x7f6d700fe4d0 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d6e59c640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fe4d0 0x7f6d701978d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c920 tx=0x7f6d6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d5f7fe640 1 -- 172.21.15.31:0/3112877428 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d60007b80 con 0x7f6d700fe4d0 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d7019cfa0 con 0x7f6d700fe4d0 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.099+0000 7f6d5f7fe640 1 -- 172.21.15.31:0/3112877428 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d600054e0 con 0x7f6d700fe4d0 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.100+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d7019d460 con 0x7f6d700fe4d0 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.100+0000 7f6d5f7fe640 1 -- 172.21.15.31:0/3112877428 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6000f450 con 0x7f6d700fe4d0 2024-09-16T09:05:22.103 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.101+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d70191970 con 0x7f6d700fe4d0 2024-09-16T09:05:22.104 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.102+0000 7f6d5f7fe640 1 -- 172.21.15.31:0/3112877428 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f6d60005020 con 0x7f6d700fe4d0 2024-09-16T09:05:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.102+0000 7f6d5f7fe640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d5807af30 0x7f6d5807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.102+0000 7f6d5f7fe640 1 -- 172.21.15.31:0/3112877428 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f6d6009e170 con 0x7f6d700fe4d0 2024-09-16T09:05:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.102+0000 7f6d6dd9b640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d5807af30 0x7f6d5807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.103+0000 7f6d6dd9b640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d5807af30 0x7f6d5807d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d64000c00 tx=0x7f6d640314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:22.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.106+0000 7f6d5f7fe640 1 -- 172.21.15.31:0/3112877428 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6d600613b0 con 0x7f6d700fe4d0 2024-09-16T09:05:22.312 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.310+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d7002c560 con 0x7f6d5807af30 2024-09-16T09:05:22.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.311+0000 7f6d5f7fe640 1 -- 172.21.15.31:0/3112877428 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6d7002c560 con 0x7f6d5807af30 2024-09-16T09:05:22.315 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d5807af30 msgr2=0x7f6d5807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d5807af30 0x7f6d5807d3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d64000c00 tx=0x7f6d640314e0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fe4d0 msgr2=0x7f6d701978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fe4d0 0x7f6d701978d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000c920 tx=0x7f6d6000cde0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 shutdown_connections 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6d5807af30 0x7f6d5807d3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6d700fee10 0x7f6d70197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 --2- 172.21.15.31:0/3112877428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d700fe4d0 0x7f6d701978d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 >> 172.21.15.31:0/3112877428 conn(0x7f6d700fa280 msgr2=0x7f6d70106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 shutdown_connections 2024-09-16T09:05:22.316 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.313+0000 7f6d74b2a640 1 -- 172.21.15.31:0/3112877428 wait complete. 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.397+0000 7fbdfda0c640 1 -- 172.21.15.31:0/4178175003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 msgr2=0x7fbdf81053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.397+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/4178175003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf81053a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0098b0 tx=0x7fbdec02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 -- 172.21.15.31:0/4178175003 shutdown_connections 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/4178175003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf81053a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/4178175003 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8100680 0x7fbdf8102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 -- 172.21.15.31:0/4178175003 >> 172.21.15.31:0/4178175003 conn(0x7fbdf80fa280 msgr2=0x7fbdf80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 -- 172.21.15.31:0/4178175003 shutdown_connections 2024-09-16T09:05:22.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 -- 172.21.15.31:0/4178175003 wait complete. 2024-09-16T09:05:22.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 Processor -- start 2024-09-16T09:05:22.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.398+0000 7fbdfda0c640 1 -- start start 2024-09-16T09:05:22.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdfda0c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8100680 0x7fbdf80fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdfda0c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf80fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdfda0c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80feb70 con 0x7fbdf8102fb0 2024-09-16T09:05:22.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdfda0c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf80fece0 con 0x7fbdf8100680 2024-09-16T09:05:22.402 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf80fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf80fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41954/0 (socket says 172.21.15.31:41954) 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf67fc640 1 -- 172.21.15.31:0/37345563 learned_addr learned my addr 172.21.15.31:0/37345563 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf6ffd640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8100680 0x7fbdf80fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf67fc640 1 -- 172.21.15.31:0/37345563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8100680 msgr2=0x7fbdf80fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf67fc640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8100680 0x7fbdf80fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf67fc640 1 -- 172.21.15.31:0/37345563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec009560 con 0x7fbdf8102fb0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.399+0000 7fbdf67fc640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf80fe630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbdec02a9f0 tx=0x7fbdec0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.400+0000 7fbdfca0a640 1 -- 172.21.15.31:0/37345563 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdec004aa0 con 0x7fbdf8102fb0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.400+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf80fef60 con 0x7fbdf8102fb0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.400+0000 7fbdfca0a640 1 -- 172.21.15.31:0/37345563 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdec005d50 con 0x7fbdf8102fb0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.400+0000 7fbdfca0a640 1 -- 172.21.15.31:0/37345563 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdec0039b0 con 0x7fbdf8102fb0 2024-09-16T09:05:22.403 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.400+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf80ff450 con 0x7fbdf8102fb0 2024-09-16T09:05:22.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.401+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf8108800 con 0x7fbdf8102fb0 2024-09-16T09:05:22.404 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.402+0000 7fbdfca0a640 1 -- 172.21.15.31:0/37345563 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fbdec031c00 con 0x7fbdf8102fb0 2024-09-16T09:05:22.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.403+0000 7fbdfca0a640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbde007af30 0x7fbde007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.403+0000 7fbdfca0a640 1 -- 172.21.15.31:0/37345563 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fbdec0befc0 con 0x7fbdf8102fb0 2024-09-16T09:05:22.405 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.403+0000 7fbdf6ffd640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbde007af30 0x7fbde007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.406 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.404+0000 7fbdf6ffd640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbde007af30 0x7fbde007d3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009bd0 tx=0x7fbde8005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:22.409 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.408+0000 7fbdfca0a640 1 -- 172.21.15.31:0/37345563 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fbdec082160 con 0x7fbdf8102fb0 2024-09-16T09:05:22.604 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.602+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf8103a50 con 0x7fbde007af30 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.622+0000 7fbdfca0a640 1 -- 172.21.15.31:0/37345563 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbdf8103a50 con 0x7fbde007af30 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (14m) 28s ago 16m 25.7M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (7m) 28s ago 17m 8908k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 0437f0a02ae0 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (7m) 13s ago 16m 8933k - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 88f8ab9c0aa5 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (5m) 28s ago 17m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (5m) 13s ago 15m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (14m) 28s ago 16m 87.0M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:05:22.624 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (52s) 28s ago 10m 13.0M - 19.3.0-4917-g8293d73f 425e316ba8eb 7e4ca0c031e9 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (34s) 28s ago 10m 12.2M - 19.3.0-4917-g8293d73f 425e316ba8eb f54e4116709a 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (16s) 13s ago 10m 14.2M - 19.3.0-4917-g8293d73f 425e316ba8eb 880891e445c6 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (10m) 13s ago 10m 84.8M - 18.2.4-853-gae7f3733 fd1d6ca2b4a5 a5c4986dfb62 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (7m) 28s ago 18m 520M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (6m) 13s ago 15m 455M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (6m) 28s ago 18m 54.2M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (5m) 13s ago 15m 41.0M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (16m) 28s ago 16m 15.7M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (15m) 13s ago 15m 16.6M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (4m) 28s ago 14m 116M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 28s ago 13m 112M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (3m) 28s ago 13m 130M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:05:22.625 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (2m) 13s ago 13m 134M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:05:22.626 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (2m) 13s ago 12m 97.2M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:05:22.626 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (88s) 13s ago 12m 107M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:05:22.626 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 28s ago 16m 65.4M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.624+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbde007af30 msgr2=0x7fbde007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.624+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbde007af30 0x7fbde007d3f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009bd0 tx=0x7fbde8005f90 comp rx=0 tx=0).stop 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 msgr2=0x7fbdf80fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf80fe630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbdec02a9f0 tx=0x7fbdec0055a0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 shutdown_connections 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbde007af30 0x7fbde007d3f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdf8102fb0 0x7fbdf80fe630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 --2- 172.21.15.31:0/37345563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdf8100680 0x7fbdf80fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 >> 172.21.15.31:0/37345563 conn(0x7fbdf80fa280 msgr2=0x7fbdf80fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:22.627 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 shutdown_connections 2024-09-16T09:05:22.628 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.625+0000 7fbdfda0c640 1 -- 172.21.15.31:0/37345563 wait complete. 2024-09-16T09:05:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 -- 172.21.15.31:0/1776118863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 msgr2=0x7f111c105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 --2- 172.21.15.31:0/1776118863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c105460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f110c009880 tx=0x7f110c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 -- 172.21.15.31:0/1776118863 shutdown_connections 2024-09-16T09:05:22.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 --2- 172.21.15.31:0/1776118863 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c105460 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 --2- 172.21.15.31:0/1776118863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c100740 0x7f111c102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 -- 172.21.15.31:0/1776118863 >> 172.21.15.31:0/1776118863 conn(0x7f111c0fa280 msgr2=0x7f111c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 -- 172.21.15.31:0/1776118863 shutdown_connections 2024-09-16T09:05:22.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.711+0000 7f112393c640 1 -- 172.21.15.31:0/1776118863 wait complete. 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f112393c640 1 Processor -- start 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f112393c640 1 -- start start 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f112393c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c100740 0x7f111c1000a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f112393c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c0fe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f112393c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0fec30 con 0x7f111c103070 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f112393c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0feda0 con 0x7f111c100740 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f1120ed7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c0fe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f1120ed7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c0fe6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41970/0 (socket says 172.21.15.31:41970) 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f1120ed7640 1 -- 172.21.15.31:0/3102155727 learned_addr learned my addr 172.21.15.31:0/3102155727 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.712+0000 7f11216d8640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c100740 0x7f111c1000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.713+0000 7f1120ed7640 1 -- 172.21.15.31:0/3102155727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c100740 msgr2=0x7f111c1000a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.713+0000 7f1120ed7640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c100740 0x7f111c1000a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.713+0000 7f1120ed7640 1 -- 172.21.15.31:0/3102155727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c009560 con 0x7f111c103070 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.713+0000 7f11216d8640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c100740 0x7f111c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.713+0000 7f1120ed7640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c0fe6f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f110c032b90 tx=0x7f110c032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:22.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.713+0000 7f11127fc640 1 -- 172.21.15.31:0/3102155727 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110c005220 con 0x7f111c103070 2024-09-16T09:05:22.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.713+0000 7f11127fc640 1 -- 172.21.15.31:0/3102155727 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f110c005380 con 0x7f111c103070 2024-09-16T09:05:22.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.714+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c0ff020 con 0x7f111c103070 2024-09-16T09:05:22.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.714+0000 7f11127fc640 1 -- 172.21.15.31:0/3102155727 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110c003ac0 con 0x7f111c103070 2024-09-16T09:05:22.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.714+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c0ff510 con 0x7f111c103070 2024-09-16T09:05:22.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.715+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c1087e0 con 0x7f111c103070 2024-09-16T09:05:22.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.717+0000 7f11127fc640 1 -- 172.21.15.31:0/3102155727 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f110c004c80 con 0x7f111c103070 2024-09-16T09:05:22.719 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.717+0000 7f11127fc640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f110807af30 0x7f110807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.718+0000 7f11127fc640 1 -- 172.21.15.31:0/3102155727 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f110c0bf060 con 0x7f111c103070 2024-09-16T09:05:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.718+0000 7f11216d8640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f110807af30 0x7f110807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:22.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.718+0000 7f11216d8640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f110807af30 0x7f110807d3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1118005d00 tx=0x7f111800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:22.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:22.721+0000 7f11127fc640 1 -- 172.21.15.31:0/3102155727 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f110c082200 con 0x7f111c103070 2024-09-16T09:05:23.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.000+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f111c1089a0 con 0x7f111c103070 2024-09-16T09:05:23.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:22 smithi031 ceph-mon[91884]: pgmap v231: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-09-16T09:05:23.003 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:22 smithi031 ceph-mon[91884]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:23.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.001+0000 7f11127fc640 1 -- 172.21.15.31:0/3102155727 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f110c085320 con 0x7f111c103070 2024-09-16T09:05:23.003 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 3 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:05:23.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 13 2024-09-16T09:05:23.005 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:05:23.005 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:05:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f110807af30 msgr2=0x7f110807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f110807af30 0x7f110807d3f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1118005d00 tx=0x7f111800b410 comp rx=0 tx=0).stop 2024-09-16T09:05:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 msgr2=0x7f111c0fe6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c0fe6f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f110c032b90 tx=0x7f110c032bc0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 shutdown_connections 2024-09-16T09:05:23.006 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f110807af30 0x7f110807d3f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f111c103070 0x7f111c0fe6f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 --2- 172.21.15.31:0/3102155727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f111c100740 0x7f111c1000a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 >> 172.21.15.31:0/3102155727 conn(0x7f111c0fa280 msgr2=0x7f111c0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 shutdown_connections 2024-09-16T09:05:23.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.003+0000 7f112393c640 1 -- 172.21.15.31:0/3102155727 wait complete. 2024-09-16T09:05:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:22 smithi097 ceph-mon[79892]: pgmap v231: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 13 MiB/s rd, 4 op/s 2024-09-16T09:05:23.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:22 smithi097 ceph-mon[79892]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:23.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.087+0000 7fdf55721640 1 -- 172.21.15.31:0/2066026529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 msgr2=0x7fdf50102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.087+0000 7fdf55721640 1 --2- 172.21.15.31:0/2066026529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50102af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009880 tx=0x7fdf4002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 -- 172.21.15.31:0/2066026529 shutdown_connections 2024-09-16T09:05:23.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 --2- 172.21.15.31:0/2066026529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf50103030 0x7fdf50105420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.090 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 --2- 172.21.15.31:0/2066026529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50102af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 -- 172.21.15.31:0/2066026529 >> 172.21.15.31:0/2066026529 conn(0x7fdf500fa280 msgr2=0x7fdf500fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:23.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 -- 172.21.15.31:0/2066026529 shutdown_connections 2024-09-16T09:05:23.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 -- 172.21.15.31:0/2066026529 wait complete. 2024-09-16T09:05:23.091 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 Processor -- start 2024-09-16T09:05:23.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.088+0000 7fdf55721640 1 -- start start 2024-09-16T09:05:23.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf55721640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf55721640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf50103030 0x7fdf50195c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf55721640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50196210 con 0x7fdf50100700 2024-09-16T09:05:23.092 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf55721640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf50196380 con 0x7fdf50103030 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50195700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41990/0 (socket says 172.21.15.31:41990) 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4effd640 1 -- 172.21.15.31:0/3430219692 learned_addr learned my addr 172.21.15.31:0/3430219692 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4e7fc640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf50103030 0x7fdf50195c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4effd640 1 -- 172.21.15.31:0/3430219692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf50103030 msgr2=0x7fdf50195c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4effd640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf50103030 0x7fdf50195c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4effd640 1 -- 172.21.15.31:0/3430219692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf40009560 con 0x7fdf50100700 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.089+0000 7fdf4effd640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50195700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005640 tx=0x7fdf40005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.090+0000 7fdf37fff640 1 -- 172.21.15.31:0/3430219692 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf40004ba0 con 0x7fdf50100700 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.090+0000 7fdf37fff640 1 -- 172.21.15.31:0/3430219692 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf40004d00 con 0x7fdf50100700 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.090+0000 7fdf37fff640 1 -- 172.21.15.31:0/3430219692 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf40003c30 con 0x7fdf50100700 2024-09-16T09:05:23.093 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.090+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5019ad70 con 0x7fdf50100700 2024-09-16T09:05:23.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.090+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5019b1c0 con 0x7fdf50100700 2024-09-16T09:05:23.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.091+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf50108820 con 0x7fdf50100700 2024-09-16T09:05:23.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.092+0000 7fdf37fff640 1 -- 172.21.15.31:0/3430219692 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fdf40031640 con 0x7fdf50100700 2024-09-16T09:05:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.092+0000 7fdf37fff640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf3807af30 0x7fdf3807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.093+0000 7fdf37fff640 1 -- 172.21.15.31:0/3430219692 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fdf400bf6c0 con 0x7fdf50100700 2024-09-16T09:05:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.093+0000 7fdf4e7fc640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf3807af30 0x7fdf3807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.093+0000 7fdf4e7fc640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf3807af30 0x7fdf3807d3f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf50196bd0 tx=0x7fdf4400a250 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:23.098 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.096+0000 7fdf37fff640 1 -- 172.21.15.31:0/3430219692 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fdf40082860 con 0x7fdf50100700 2024-09-16T09:05:23.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.320+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdf5019b400 con 0x7fdf50100700 2024-09-16T09:05:23.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.320+0000 7fdf37fff640 1 -- 172.21.15.31:0/3430219692 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 33 v33) v1 ==== 76+0+1786 (secure 0 0 0) 0x7fdf40085980 con 0x7fdf50100700 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:e33 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T09:05:16:143457+0000 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 33 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:05:23.323 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T09:05:16.143437+0000 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 99 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 1 2024-09-16T09:05:23.324 INFO:teuthology.orchestra.run.smithi031.stdout:in 0 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34430} 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 1 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 0 members: 2024-09-16T09:05:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{0:34430} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:05:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:23.326 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:05:23.326 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:23.326 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{-1:34434} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:05:23.326 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6824/257407185,v1:172.21.15.97:6825/257407185] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 33 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.323+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf3807af30 msgr2=0x7fdf3807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.323+0000 7fdf55721640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf3807af30 0x7fdf3807d3f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdf50196bd0 tx=0x7fdf4400a250 comp rx=0 tx=0).stop 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.323+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 msgr2=0x7fdf50195700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.323+0000 7fdf55721640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50195700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdf40005640 tx=0x7fdf40005670 comp rx=0 tx=0).stop 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.324+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 shutdown_connections 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.324+0000 7fdf55721640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fdf3807af30 0x7fdf3807d3f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.324+0000 7fdf55721640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdf50103030 0x7fdf50195c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.324+0000 7fdf55721640 1 --2- 172.21.15.31:0/3430219692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf50100700 0x7fdf50195700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.327 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.324+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 >> 172.21.15.31:0/3430219692 conn(0x7fdf500fa280 msgr2=0x7fdf500fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:23.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.324+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 shutdown_connections 2024-09-16T09:05:23.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.324+0000 7fdf55721640 1 -- 172.21.15.31:0/3430219692 wait complete. 2024-09-16T09:05:23.410 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 -- 172.21.15.31:0/3641263014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 msgr2=0x7f0340100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 --2- 172.21.15.31:0/3641263014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340100120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0334009880 tx=0x7f033402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 -- 172.21.15.31:0/3641263014 shutdown_connections 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 --2- 172.21.15.31:0/3641263014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340100120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 --2- 172.21.15.31:0/3641263014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03400feaa0 0x7f03400feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 -- 172.21.15.31:0/3641263014 >> 172.21.15.31:0/3641263014 conn(0x7f03400fa250 msgr2=0x7f03400fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 -- 172.21.15.31:0/3641263014 shutdown_connections 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.408+0000 7f0345083640 1 -- 172.21.15.31:0/3641263014 wait complete. 2024-09-16T09:05:23.411 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f0345083640 1 Processor -- start 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f0345083640 1 -- start start 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f0345083640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03400feaa0 0x7f0340195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f0345083640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f0345083640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0340196110 con 0x7f03400ffca0 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f0345083640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0340196280 con 0x7f03400feaa0 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f033e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f033e59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340195b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42006/0 (socket says 172.21.15.31:42006) 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f033e59c640 1 -- 172.21.15.31:0/3486465096 learned_addr learned my addr 172.21.15.31:0/3486465096 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.409+0000 7f033ed9d640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03400feaa0 0x7f0340195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f033e59c640 1 -- 172.21.15.31:0/3486465096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03400feaa0 msgr2=0x7f0340195600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.413 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f033e59c640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03400feaa0 0x7f0340195600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f033e59c640 1 -- 172.21.15.31:0/3486465096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0334009560 con 0x7f03400ffca0 2024-09-16T09:05:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f033e59c640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340195b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0334005640 tx=0x7f0334005670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f0327fff640 1 -- 172.21.15.31:0/3486465096 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0334004ba0 con 0x7f03400ffca0 2024-09-16T09:05:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f0327fff640 1 -- 172.21.15.31:0/3486465096 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0334004d00 con 0x7f03400ffca0 2024-09-16T09:05:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034019acc0 con 0x7f03400ffca0 2024-09-16T09:05:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f0327fff640 1 -- 172.21.15.31:0/3486465096 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0334003c30 con 0x7f03400ffca0 2024-09-16T09:05:23.414 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.410+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f034019b230 con 0x7f03400ffca0 2024-09-16T09:05:23.415 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.411+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0340107690 con 0x7f03400ffca0 2024-09-16T09:05:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.412+0000 7f0327fff640 1 -- 172.21.15.31:0/3486465096 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f0334031640 con 0x7f03400ffca0 2024-09-16T09:05:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.412+0000 7f0327fff640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f032807af30 0x7f032807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.412+0000 7f0327fff640 1 -- 172.21.15.31:0/3486465096 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f03340bf360 con 0x7f03400ffca0 2024-09-16T09:05:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.412+0000 7f033ed9d640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f032807af30 0x7f032807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.416 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.413+0000 7f033ed9d640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f032807af30 0x7f032807d3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f03400ffb00 tx=0x7f033000a590 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:23.419 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.417+0000 7f0327fff640 1 -- 172.21.15.31:0/3486465096 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f03340823e0 con 0x7f03400ffca0 2024-09-16T09:05:23.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.634+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f034019b420 con 0x7f032807af30 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.635+0000 7f0327fff640 1 -- 172.21.15.31:0/3486465096 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f034019b420 con 0x7f032807af30 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:05:23.637 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:05:23.638 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-09-16T09:05:23.638 INFO:teuthology.orchestra.run.smithi031.stdout: "osd" 2024-09-16T09:05:23.638 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:05:23.638 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "15/23 daemons upgraded", 2024-09-16T09:05:23.638 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading mds daemons", 2024-09-16T09:05:23.638 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:05:23.638 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:05:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f032807af30 msgr2=0x7f032807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f032807af30 0x7f032807d3f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f03400ffb00 tx=0x7f033000a590 comp rx=0 tx=0).stop 2024-09-16T09:05:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 msgr2=0x7f0340195b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340195b40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0334005640 tx=0x7f0334005670 comp rx=0 tx=0).stop 2024-09-16T09:05:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 shutdown_connections 2024-09-16T09:05:23.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f032807af30 0x7f032807d3f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03400ffca0 0x7f0340195b40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 --2- 172.21.15.31:0/3486465096 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03400feaa0 0x7f0340195600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 >> 172.21.15.31:0/3486465096 conn(0x7f03400fa250 msgr2=0x7f0340068500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 shutdown_connections 2024-09-16T09:05:23.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.637+0000 7f0345083640 1 -- 172.21.15.31:0/3486465096 wait complete. 2024-09-16T09:05:23.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.720+0000 7f87b9317640 1 -- 172.21.15.31:0/2549030517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40ffca0 msgr2=0x7f87b4100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.720+0000 7f87b9317640 1 --2- 172.21.15.31:0/2549030517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40ffca0 0x7f87b4100120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f879c009880 tx=0x7f879c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.720+0000 7f87b9317640 1 -- 172.21.15.31:0/2549030517 shutdown_connections 2024-09-16T09:05:23.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.720+0000 7f87b9317640 1 --2- 172.21.15.31:0/2549030517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40ffca0 0x7f87b4100120 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.720+0000 7f87b9317640 1 --2- 172.21.15.31:0/2549030517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87b40feaa0 0x7f87b40feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.720+0000 7f87b9317640 1 -- 172.21.15.31:0/2549030517 >> 172.21.15.31:0/2549030517 conn(0x7f87b40fa250 msgr2=0x7f87b40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 -- 172.21.15.31:0/2549030517 shutdown_connections 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 -- 172.21.15.31:0/2549030517 wait complete. 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 Processor -- start 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 -- start start 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40feaa0 0x7f87b4195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87b40ffca0 0x7f87b4195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b4196160 con 0x7f87b40feaa0 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b9317640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b41962d0 con 0x7f87b40ffca0 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40feaa0 0x7f87b4195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b2ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40feaa0 0x7f87b4195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:42028/0 (socket says 172.21.15.31:42028) 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.721+0000 7f87b2ffd640 1 -- 172.21.15.31:0/3278571665 learned_addr learned my addr 172.21.15.31:0/3278571665 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87abfff640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87b40ffca0 0x7f87b4195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b2ffd640 1 -- 172.21.15.31:0/3278571665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87b40ffca0 msgr2=0x7f87b4195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b2ffd640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87b40ffca0 0x7f87b4195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b2ffd640 1 -- 172.21.15.31:0/3278571665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c009560 con 0x7f87b40feaa0 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b2ffd640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40feaa0 0x7f87b4195650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c920 tx=0x7f87a400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b0ff9640 1 -- 172.21.15.31:0/3278571665 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a40041f0 con 0x7f87b40feaa0 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b0ff9640 1 -- 172.21.15.31:0/3278571665 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87a4007e70 con 0x7f87b40feaa0 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b419ad70 con 0x7f87b40feaa0 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b0ff9640 1 -- 172.21.15.31:0/3278571665 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a40049d0 con 0x7f87b40feaa0 2024-09-16T09:05:23.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.722+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b419b2b0 con 0x7f87b40feaa0 2024-09-16T09:05:23.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.723+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b41076b0 con 0x7f87b40feaa0 2024-09-16T09:05:23.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.724+0000 7f87b0ff9640 1 -- 172.21.15.31:0/3278571665 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f87a4004350 con 0x7f87b40feaa0 2024-09-16T09:05:23.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.724+0000 7f87b0ff9640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a007af30 0x7f87a007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:23.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.725+0000 7f87b0ff9640 1 -- 172.21.15.31:0/3278571665 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f87a409dda0 con 0x7f87b40feaa0 2024-09-16T09:05:23.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.725+0000 7f87abfff640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a007af30 0x7f87a007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:23.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.725+0000 7f87abfff640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a007af30 0x7f87a007d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f879c000c00 tx=0x7f879c0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:23.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.730+0000 7f87b0ff9640 1 -- 172.21.15.31:0/3278571665 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f87a4060e90 con 0x7f87b40feaa0 2024-09-16T09:05:24.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.999+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f87b4107900 con 0x7f87b40feaa0 2024-09-16T09:05:24.001 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:23 smithi031 ceph-mon[91884]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:24.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:23 smithi031 ceph-mon[91884]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:24.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:23 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3102155727' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:24.002 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:23 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3430219692' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:05:24.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:23.999+0000 7f87b0ff9640 1 -- 172.21.15.31:0/3278571665 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f87a401d080 con 0x7f87b40feaa0 2024-09-16T09:05:24.002 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_WARN 1 filesystem is degraded 2024-09-16T09:05:24.002 INFO:teuthology.orchestra.run.smithi031.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-16T09:05:24.002 INFO:teuthology.orchestra.run.smithi031.stdout: fs cephfs is degraded 2024-09-16T09:05:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a007af30 msgr2=0x7f87a007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a007af30 0x7f87a007d3f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f879c000c00 tx=0x7f879c0314e0 comp rx=0 tx=0).stop 2024-09-16T09:05:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40feaa0 msgr2=0x7f87b4195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40feaa0 0x7f87b4195650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f87a400c920 tx=0x7f87a400cde0 comp rx=0 tx=0).stop 2024-09-16T09:05:24.004 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 shutdown_connections 2024-09-16T09:05:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a007af30 0x7f87a007d3f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87b40ffca0 0x7f87b4195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 --2- 172.21.15.31:0/3278571665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87b40feaa0 0x7f87b4195650 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 >> 172.21.15.31:0/3278571665 conn(0x7f87b40fa250 msgr2=0x7f87b4068520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 shutdown_connections 2024-09-16T09:05:24.005 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:24.002+0000 7f87b9317640 1 -- 172.21.15.31:0/3278571665 wait complete. 2024-09-16T09:05:24.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:23 smithi097 ceph-mon[79892]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:24.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:23 smithi097 ceph-mon[79892]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:24.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:23 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3102155727' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:24.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:23 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3430219692' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: pgmap v232: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: reconnect by client.24365 v1:172.21.15.31:0/3242977461 after 0.000999984 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:reconnect 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: fsmap cephfs:1/1 {0=cephfs.smithi031.lcfutm=up:reconnect} 2 up:standby 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: reconnect by client.24369 v1:192.168.0.1:0/2254071746 after 0.00499992 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:25.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:24 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3278571665' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:05:25.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: pgmap v232: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 18 MiB/s rd, 5 op/s 2024-09-16T09:05:25.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:25.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: reconnect by client.24365 v1:172.21.15.31:0/3242977461 after 0.000999984 2024-09-16T09:05:25.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:reconnect 2024-09-16T09:05:25.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: fsmap cephfs:1/1 {0=cephfs.smithi031.lcfutm=up:reconnect} 2 up:standby 2024-09-16T09:05:25.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: reconnect by client.24369 v1:192.168.0.1:0/2254071746 after 0.00499992 2024-09-16T09:05:25.022 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:25.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:25.023 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:24 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3278571665' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:rejoin 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.97:6826/2995504601,v1:172.21.15.97:6827/2995504601] up:boot 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: fsmap cephfs:1/1 {0=cephfs.smithi031.lcfutm=up:rejoin} 3 up:standby 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:26.046 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:25 smithi097 ceph-mon[79892]: daemon mds.cephfs.smithi031.lcfutm is now active in filesystem cephfs as rank 0 2024-09-16T09:05:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:rejoin 2024-09-16T09:05:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.97:6826/2995504601,v1:172.21.15.97:6827/2995504601] up:boot 2024-09-16T09:05:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: fsmap cephfs:1/1 {0=cephfs.smithi031.lcfutm=up:rejoin} 3 up:standby 2024-09-16T09:05:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "mds metadata", "who": "cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T09:05:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:26.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:26.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:26.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:25 smithi031 ceph-mon[91884]: daemon mds.cephfs.smithi031.lcfutm is now active in filesystem cephfs as rank 0 2024-09-16T09:05:27.083 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:26 smithi097 ceph-mon[79892]: pgmap v233: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-09-16T09:05:27.083 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:26 smithi097 ceph-mon[79892]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T09:05:27.084 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:26 smithi097 ceph-mon[79892]: Cluster is now healthy 2024-09-16T09:05:27.084 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:26 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:active 2024-09-16T09:05:27.084 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:26 smithi097 ceph-mon[79892]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 3 up:standby 2024-09-16T09:05:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:26 smithi031 ceph-mon[91884]: pgmap v233: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 19 MiB/s rd, 5 op/s 2024-09-16T09:05:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:26 smithi031 ceph-mon[91884]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T09:05:27.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:26 smithi031 ceph-mon[91884]: Cluster is now healthy 2024-09-16T09:05:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:26 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] up:active 2024-09-16T09:05:27.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:26 smithi031 ceph-mon[91884]: fsmap cephfs:1 {0=cephfs.smithi031.lcfutm=up:active} 3 up:standby 2024-09-16T09:05:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:28 smithi097 ceph-mon[79892]: pgmap v234: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 91 B/s wr, 7 op/s 2024-09-16T09:05:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:28 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:28 smithi031 ceph-mon[91884]: pgmap v234: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 17 MiB/s rd, 91 B/s wr, 7 op/s 2024-09-16T09:05:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:28.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:28 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:30.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:30 smithi031 ceph-mon[91884]: pgmap v235: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-09-16T09:05:30.982 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:30 smithi097 ceph-mon[79892]: pgmap v235: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 16 MiB/s rd, 170 B/s wr, 7 op/s 2024-09-16T09:05:32.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: pgmap v236: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.624 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all mds 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.lcfutm"} : dispatch 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.lcfutm"}]': finished 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.mjhxgi"} : dispatch 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.mjhxgi"}]': finished 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.cagmus"} : dispatch 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.cagmus"}]': finished 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.sbwmfq"}]': finished 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-09-16T09:05:32.625 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} : dispatch 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: daemon mds.cephfs.smithi031.mjhxgi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi031.mjhxgi=up:starting} 2 up:standby 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all rgw 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: daemon mds.cephfs.smithi031.mjhxgi is now active in filesystem cephfs as rank 1 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.626 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:32 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: pgmap v236: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 12 MiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-16T09:05:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all mds 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.lcfutm"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.lcfutm"}]': finished 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.mjhxgi"} : dispatch 2024-09-16T09:05:32.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi031.mjhxgi"}]': finished 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.cagmus"} : dispatch 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.cagmus"}]': finished 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.sbwmfq"} : dispatch 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi097.sbwmfq"}]': finished 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: Upgrade: Scaling up filesystem cephfs max_mds to 2 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} : dispatch 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: daemon mds.cephfs.smithi031.mjhxgi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi031.mjhxgi=up:starting} 2 up:standby 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all rgw 2024-09-16T09:05:32.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: daemon mds.cephfs.smithi031.mjhxgi is now active in filesystem cephfs as rank 1 2024-09-16T09:05:32.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:32.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all rbd-mirror 2024-09-16T09:05:32.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:32.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:32 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:33.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:33 smithi031 ceph-mon[91884]: mds.? [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] up:active 2024-09-16T09:05:33.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:33 smithi031 ceph-mon[91884]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi031.mjhxgi=up:active} 2 up:standby 2024-09-16T09:05:33.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:33 smithi031 ceph-mon[91884]: Upgrade: Updating ceph-exporter.smithi031 (1/2) 2024-09-16T09:05:33.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:33 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:33.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:33 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T09:05:33.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:33 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:33.760 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:33 smithi031 ceph-mon[91884]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-09-16T09:05:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:33 smithi097 ceph-mon[79892]: mds.? [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] up:active 2024-09-16T09:05:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:33 smithi097 ceph-mon[79892]: fsmap cephfs:2 {0=cephfs.smithi031.lcfutm=up:active,1=cephfs.smithi031.mjhxgi=up:active} 2 up:standby 2024-09-16T09:05:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:33 smithi097 ceph-mon[79892]: Upgrade: Updating ceph-exporter.smithi031 (1/2) 2024-09-16T09:05:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:33 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:33 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi031", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T09:05:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:33 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:34.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:33 smithi097 ceph-mon[79892]: Deploying daemon ceph-exporter.smithi031 on smithi031 2024-09-16T09:05:35.346 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:35 smithi031 ceph-mon[91884]: pgmap v237: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 5.8 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-16T09:05:35.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:35 smithi097 ceph-mon[79892]: pgmap v237: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 5.8 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-16T09:05:37.351 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:37 smithi097 ceph-mon[79892]: pgmap v238: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-16T09:05:37.351 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:37 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:37.351 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:37 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:37 smithi031 ceph-mon[91884]: pgmap v238: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-16T09:05:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:37 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:37.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:37 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:39 smithi097 ceph-mon[79892]: pgmap v239: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 826 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-16T09:05:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T09:05:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:39 smithi031 ceph-mon[91884]: pgmap v239: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 826 KiB/s rd, 4.4 KiB/s wr, 11 op/s 2024-09-16T09:05:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi097", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]} : dispatch 2024-09-16T09:05:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:39.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:40.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:40 smithi031 ceph-mon[91884]: Upgrade: Updating ceph-exporter.smithi097 (2/2) 2024-09-16T09:05:40.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:40 smithi031 ceph-mon[91884]: Deploying daemon ceph-exporter.smithi097 on smithi097 2024-09-16T09:05:40.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:40 smithi097 ceph-mon[79892]: Upgrade: Updating ceph-exporter.smithi097 (2/2) 2024-09-16T09:05:40.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:40 smithi097 ceph-mon[79892]: Deploying daemon ceph-exporter.smithi097 on smithi097 2024-09-16T09:05:41.234 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:41 smithi097 ceph-mon[79892]: pgmap v240: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-16T09:05:41.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:41 smithi031 ceph-mon[91884]: pgmap v240: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 8.3 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-09-16T09:05:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:42 smithi097 ceph-mon[79892]: pgmap v241: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-16T09:05:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:42 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:42 smithi031 ceph-mon[91884]: pgmap v241: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 6.4 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-09-16T09:05:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:43.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:42 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:44 smithi031 ceph-mon[91884]: pgmap v242: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T09:05:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:44 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:44 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:44 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:45.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:44 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:45.138 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:44 smithi097 ceph-mon[79892]: pgmap v242: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T09:05:45.138 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:44 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:45.138 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:44 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:45.138 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:44 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:45.138 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:44 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:46 smithi031 ceph-mon[91884]: pgmap v243: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-09-16T09:05:47.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.199 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.200 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:46 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:46 smithi097 ceph-mon[79892]: pgmap v243: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-09-16T09:05:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:46 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:48.832 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:48 smithi031 ceph-mon[91884]: pgmap v244: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-16T09:05:48.832 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:48.832 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:48 smithi097 ceph-mon[79892]: pgmap v244: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-16T09:05:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:49.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.109 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: pgmap v245: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:51.109 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.110 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"} : dispatch 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"}]': finished 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi097"} : dispatch 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi097"}]': finished 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.111 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:51 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: pgmap v245: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi031"}]': finished 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi097"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi097"}]': finished 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:51.514 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:51 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:52 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all ceph-exporter 2024-09-16T09:05:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:52 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all iscsi 2024-09-16T09:05:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:52 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all nfs 2024-09-16T09:05:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:52 smithi097 ceph-mon[79892]: Upgrade: Setting container_image for all nvmeof 2024-09-16T09:05:52.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:52 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all ceph-exporter 2024-09-16T09:05:52.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:52 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all iscsi 2024-09-16T09:05:52.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:52 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all nfs 2024-09-16T09:05:52.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:52 smithi031 ceph-mon[91884]: Upgrade: Setting container_image for all nvmeof 2024-09-16T09:05:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:53 smithi031 ceph-mon[91884]: pgmap v246: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:53 smithi031 ceph-mon[91884]: Upgrade: Updating node-exporter.smithi031 (1/2) 2024-09-16T09:05:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:53 smithi031 ceph-mon[91884]: Deploying daemon node-exporter.smithi031 on smithi031 2024-09-16T09:05:53.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:53 smithi097 ceph-mon[79892]: pgmap v246: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:53.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:53 smithi097 ceph-mon[79892]: Upgrade: Updating node-exporter.smithi031 (1/2) 2024-09-16T09:05:53.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:53 smithi097 ceph-mon[79892]: Deploying daemon node-exporter.smithi031 on smithi031 2024-09-16T09:05:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 -- 172.21.15.31:0/1825402351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0103030 msgr2=0x7fc7d0105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/1825402351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0103030 0x7fc7d0105420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009880 tx=0x7fc7c402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 -- 172.21.15.31:0/1825402351 shutdown_connections 2024-09-16T09:05:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/1825402351 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0103030 0x7fc7d0105420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/1825402351 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0100700 0x7fc7d0102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.094 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 -- 172.21.15.31:0/1825402351 >> 172.21.15.31:0/1825402351 conn(0x7fc7d00fa280 msgr2=0x7fc7d00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 -- 172.21.15.31:0/1825402351 shutdown_connections 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 -- 172.21.15.31:0/1825402351 wait complete. 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 Processor -- start 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.091+0000 7fc7d65c5640 1 -- start start 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7d65c5640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0100700 0x7fc7d0100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7d65c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0103030 0x7fc7d00fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7d65c5640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00febf0 con 0x7fc7d0100700 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7d65c5640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00fed60 con 0x7fc7d0103030 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0100700 0x7fc7d0100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7cffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0100700 0x7fc7d0100060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45940/0 (socket says 172.21.15.31:45940) 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7cffff640 1 -- 172.21.15.31:0/2114093771 learned_addr learned my addr 172.21.15.31:0/2114093771 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7cf7fe640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0103030 0x7fc7d00fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.095 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7cffff640 1 -- 172.21.15.31:0/2114093771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0103030 msgr2=0x7fc7d00fe6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7cffff640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0103030 0x7fc7d00fe6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.092+0000 7fc7cffff640 1 -- 172.21.15.31:0/2114093771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4009560 con 0x7fc7d0100700 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.093+0000 7fc7cffff640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0100700 0x7fc7d0100060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c810 tx=0x7fc7c000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.093+0000 7fc7cd7fa640 1 -- 172.21.15.31:0/2114093771 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c00041f0 con 0x7fc7d0100700 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.093+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d00ff040 con 0x7fc7d0100700 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.093+0000 7fc7cd7fa640 1 -- 172.21.15.31:0/2114093771 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7c0007e70 con 0x7fc7d0100700 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.093+0000 7fc7cd7fa640 1 -- 172.21.15.31:0/2114093771 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c00049d0 con 0x7fc7d0100700 2024-09-16T09:05:54.096 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.093+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d019d710 con 0x7fc7d0100700 2024-09-16T09:05:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.094+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d0109130 con 0x7fc7d0100700 2024-09-16T09:05:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.095+0000 7fc7cd7fa640 1 -- 172.21.15.31:0/2114093771 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fc7c00079b0 con 0x7fc7d0100700 2024-09-16T09:05:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.095+0000 7fc7cd7fa640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc7b807af30 0x7fc7b807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.095+0000 7fc7cd7fa640 1 -- 172.21.15.31:0/2114093771 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fc7c009dde0 con 0x7fc7d0100700 2024-09-16T09:05:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.095+0000 7fc7cf7fe640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc7b807af30 0x7fc7b807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.097 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.096+0000 7fc7cf7fe640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc7b807af30 0x7fc7b807d3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009230 tx=0x7fc7c40314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:54.101 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.100+0000 7fc7cd7fa640 1 -- 172.21.15.31:0/2114093771 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fc7c0060f00 con 0x7fc7d0100700 2024-09-16T09:05:54.326 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.325+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7d0103a50 con 0x7fc7b807af30 2024-09-16T09:05:54.327 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:54.328 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.327+0000 7fc7cd7fa640 1 -- 172.21.15.31:0/2114093771 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fc7d0103a50 con 0x7fc7b807af30 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.329+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc7b807af30 msgr2=0x7fc7b807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.329+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc7b807af30 0x7fc7b807d3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4009230 tx=0x7fc7c40314e0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.329+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0100700 msgr2=0x7fc7d0100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.329+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0100700 0x7fc7d0100060 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c810 tx=0x7fc7c000ccd0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.330+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 shutdown_connections 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.330+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc7b807af30 0x7fc7b807d3f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.330+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc7d0103030 0x7fc7d00fe6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.330+0000 7fc7d65c5640 1 --2- 172.21.15.31:0/2114093771 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7d0100700 0x7fc7d0100060 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.330+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 >> 172.21.15.31:0/2114093771 conn(0x7fc7d00fa280 msgr2=0x7fc7d00fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.330+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 shutdown_connections 2024-09-16T09:05:54.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.330+0000 7fc7d65c5640 1 -- 172.21.15.31:0/2114093771 wait complete. 2024-09-16T09:05:54.341 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.422+0000 7fbb72724640 1 -- 172.21.15.31:0/4103195645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c100630 msgr2=0x7fbb6c102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.422+0000 7fbb72724640 1 --2- 172.21.15.31:0/4103195645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c100630 0x7fbb6c102a20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0098b0 tx=0x7fbb5c02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.422+0000 7fbb72724640 1 -- 172.21.15.31:0/4103195645 shutdown_connections 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.422+0000 7fbb72724640 1 --2- 172.21.15.31:0/4103195645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb6c102f60 0x7fbb6c105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.422+0000 7fbb72724640 1 --2- 172.21.15.31:0/4103195645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c100630 0x7fbb6c102a20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.422+0000 7fbb72724640 1 -- 172.21.15.31:0/4103195645 >> 172.21.15.31:0/4103195645 conn(0x7fbb6c0fa250 msgr2=0x7fbb6c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.422+0000 7fbb72724640 1 -- 172.21.15.31:0/4103195645 shutdown_connections 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.423+0000 7fbb72724640 1 -- 172.21.15.31:0/4103195645 wait complete. 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.423+0000 7fbb72724640 1 Processor -- start 2024-09-16T09:05:54.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.423+0000 7fbb72724640 1 -- start start 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.423+0000 7fbb72724640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb6c100630 0x7fbb6c0fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.423+0000 7fbb72724640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c102f60 0x7fbb6c0fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.423+0000 7fbb72724640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0feb20 con 0x7fbb6c102f60 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.423+0000 7fbb72724640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6c0fec90 con 0x7fbb6c100630 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c102f60 0x7fbb6c0fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6b7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c102f60 0x7fbb6c0fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45954/0 (socket says 172.21.15.31:45954) 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6b7fe640 1 -- 172.21.15.31:0/3758079439 learned_addr learned my addr 172.21.15.31:0/3758079439 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6bfff640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb6c100630 0x7fbb6c0fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6b7fe640 1 -- 172.21.15.31:0/3758079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb6c100630 msgr2=0x7fbb6c0fff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6b7fe640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb6c100630 0x7fbb6c0fff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6b7fe640 1 -- 172.21.15.31:0/3758079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c009560 con 0x7fbb6c102f60 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb6b7fe640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c102f60 0x7fbb6c0fe5e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d6c0 tx=0x7fbb6000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb697fa640 1 -- 172.21.15.31:0/3758079439 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb600051f0 con 0x7fbb6c102f60 2024-09-16T09:05:54.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb697fa640 1 -- 172.21.15.31:0/3758079439 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb60009ce0 con 0x7fbb6c102f60 2024-09-16T09:05:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c0fef70 con 0x7fbb6c102f60 2024-09-16T09:05:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb697fa640 1 -- 172.21.15.31:0/3758079439 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb60010460 con 0x7fbb6c102f60 2024-09-16T09:05:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.424+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6c19d6c0 con 0x7fbb6c102f60 2024-09-16T09:05:54.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.426+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb6c100a10 con 0x7fbb6c102f60 2024-09-16T09:05:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.426+0000 7fbb697fa640 1 -- 172.21.15.31:0/3758079439 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fbb60005350 con 0x7fbb6c102f60 2024-09-16T09:05:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.427+0000 7fbb697fa640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb5407af30 0x7fbb5407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.427+0000 7fbb697fa640 1 -- 172.21.15.31:0/3758079439 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fbb6009d9a0 con 0x7fbb6c102f60 2024-09-16T09:05:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.427+0000 7fbb6bfff640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb5407af30 0x7fbb5407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.427+0000 7fbb6bfff640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb5407af30 0x7fbb5407d3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c006010 tx=0x7fbb5c0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:54.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.431+0000 7fbb697fa640 1 -- 172.21.15.31:0/3758079439 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fbb60060b40 con 0x7fbb6c102f60 2024-09-16T09:05:54.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:05:54.635 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.634+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb6c103980 con 0x7fbb5407af30 2024-09-16T09:05:54.636 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.635+0000 7fbb697fa640 1 -- 172.21.15.31:0/3758079439 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7fbb6c103980 con 0x7fbb5407af30 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb5407af30 msgr2=0x7fbb5407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb5407af30 0x7fbb5407d3f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c006010 tx=0x7fbb5c0314e0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c102f60 msgr2=0x7fbb6c0fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c102f60 0x7fbb6c0fe5e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000d6c0 tx=0x7fbb6000db80 comp rx=0 tx=0).stop 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 shutdown_connections 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fbb5407af30 0x7fbb5407d3f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb6c102f60 0x7fbb6c0fe5e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.639 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 --2- 172.21.15.31:0/3758079439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb6c100630 0x7fbb6c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 >> 172.21.15.31:0/3758079439 conn(0x7fbb6c0fa250 msgr2=0x7fbb6c0fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:54.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 shutdown_connections 2024-09-16T09:05:54.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.638+0000 7fbb72724640 1 -- 172.21.15.31:0/3758079439 wait complete. 2024-09-16T09:05:54.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.721+0000 7fb888c07640 1 -- 172.21.15.31:0/42673213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 msgr2=0x7fb884102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.721+0000 7fb888c07640 1 --2- 172.21.15.31:0/42673213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb884102a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb874009c50 tx=0x7fb87402a9a0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 -- 172.21.15.31:0/42673213 shutdown_connections 2024-09-16T09:05:54.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 --2- 172.21.15.31:0/42673213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb884102fb0 0x7fb8841053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 --2- 172.21.15.31:0/42673213 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb884102a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 -- 172.21.15.31:0/42673213 >> 172.21.15.31:0/42673213 conn(0x7fb8840fa280 msgr2=0x7fb8840fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:54.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 -- 172.21.15.31:0/42673213 shutdown_connections 2024-09-16T09:05:54.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 -- 172.21.15.31:0/42673213 wait complete. 2024-09-16T09:05:54.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 Processor -- start 2024-09-16T09:05:54.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.722+0000 7fb888c07640 1 -- start start 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb888c07640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb8841956f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb888c07640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb884102fb0 0x7fb884195c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb888c07640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb884196200 con 0x7fb884100680 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb888c07640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb884196370 con 0x7fb884102fb0 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb88259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb8841956f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb88259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb8841956f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45978/0 (socket says 172.21.15.31:45978) 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb88259c640 1 -- 172.21.15.31:0/696494122 learned_addr learned my addr 172.21.15.31:0/696494122 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb881d9b640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb884102fb0 0x7fb884195c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb88259c640 1 -- 172.21.15.31:0/696494122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb884102fb0 msgr2=0x7fb884195c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb88259c640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb884102fb0 0x7fb884195c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb88259c640 1 -- 172.21.15.31:0/696494122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb874009900 con 0x7fb884100680 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb88259c640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb8841956f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb87402aec0 tx=0x7fb874030890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:54.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.723+0000 7fb87b7fe640 1 -- 172.21.15.31:0/696494122 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb874003970 con 0x7fb884100680 2024-09-16T09:05:54.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.724+0000 7fb87b7fe640 1 -- 172.21.15.31:0/696494122 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb874004050 con 0x7fb884100680 2024-09-16T09:05:54.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.724+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88419adb0 con 0x7fb884100680 2024-09-16T09:05:54.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.724+0000 7fb87b7fe640 1 -- 172.21.15.31:0/696494122 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb874044670 con 0x7fb884100680 2024-09-16T09:05:54.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.724+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88419b240 con 0x7fb884100680 2024-09-16T09:05:54.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.725+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb884108810 con 0x7fb884100680 2024-09-16T09:05:54.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.725+0000 7fb87b7fe640 1 -- 172.21.15.31:0/696494122 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fb87400a040 con 0x7fb884100680 2024-09-16T09:05:54.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.726+0000 7fb87b7fe640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87007af30 0x7fb87007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:54.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.726+0000 7fb87b7fe640 1 -- 172.21.15.31:0/696494122 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fb8740bf5d0 con 0x7fb884100680 2024-09-16T09:05:54.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.726+0000 7fb881d9b640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87007af30 0x7fb87007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:54.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.726+0000 7fb881d9b640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87007af30 0x7fb87007d3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005fd0 tx=0x7fb86c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:54.731 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.730+0000 7fb87b7fe640 1 -- 172.21.15.31:0/696494122 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fb874082620 con 0x7fb884100680 2024-09-16T09:05:54.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.933+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8841039d0 con 0x7fb87007af30 2024-09-16T09:05:54.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.945+0000 7fb87b7fe640 1 -- 172.21.15.31:0/696494122 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb8841039d0 con 0x7fb87007af30 2024-09-16T09:05:54.946 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:05:54.947 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (15m) 8s ago 17m 25.7M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:05:54.947 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (18s) 8s ago 17m 8254k - 19.3.0-4917-g8293d73f 425e316ba8eb 119433b14707 2024-09-16T09:05:54.947 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (13s) 11s ago 16m 7256k - 19.3.0-4917-g8293d73f 425e316ba8eb 3f8a92f1222e 2024-09-16T09:05:54.947 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (5m) 8s ago 17m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:05:54.947 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (5m) 11s ago 16m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:05:54.947 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (15m) 8s ago 17m 85.1M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (84s) 8s ago 11m 84.9M - 19.3.0-4917-g8293d73f 425e316ba8eb 7e4ca0c031e9 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (66s) 8s ago 11m 16.0M - 19.3.0-4917-g8293d73f 425e316ba8eb f54e4116709a 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (49s) 11s ago 11m 16.3M - 19.3.0-4917-g8293d73f 425e316ba8eb 880891e445c6 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (30s) 11s ago 11m 16.1M - 19.3.0-4917-g8293d73f 425e316ba8eb 08131d73c31f 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (8m) 8s ago 18m 522M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (6m) 11s ago 16m 455M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (6m) 8s ago 18m 57.0M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:05:54.948 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (6m) 11s ago 16m 44.9M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:05:54.950 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (17m) 8s ago 17m 15.4M - 1.5.0 0da6a335fe13 5e8b308f1271 2024-09-16T09:05:54.950 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (16m) 11s ago 16m 16.6M - 1.5.0 0da6a335fe13 bcaca4a8fcfb 2024-09-16T09:05:54.950 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 8s ago 14m 117M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:05:54.950 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (4m) 8s ago 14m 115M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:05:54.950 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 8s ago 13m 133M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:05:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (3m) 11s ago 13m 137M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:05:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (2m) 11s ago 13m 98.0M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:05:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (2m) 11s ago 12m 110M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:05:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (6m) 8s ago 16m 64.6M - 2.43.0 a07b618ecd1d 7e6a4303b9cc 2024-09-16T09:05:54.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87007af30 msgr2=0x7fb87007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87007af30 0x7fb87007d3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005fd0 tx=0x7fb86c005850 comp rx=0 tx=0).stop 2024-09-16T09:05:54.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 msgr2=0x7fb8841956f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:54.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb8841956f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb87402aec0 tx=0x7fb874030890 comp rx=0 tx=0).stop 2024-09-16T09:05:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 shutdown_connections 2024-09-16T09:05:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb87007af30 0x7fb87007d3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb884102fb0 0x7fb884195c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 --2- 172.21.15.31:0/696494122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb884100680 0x7fb8841956f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 >> 172.21.15.31:0/696494122 conn(0x7fb8840fa280 msgr2=0x7fb8840fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:54.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 shutdown_connections 2024-09-16T09:05:54.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:54.948+0000 7fb888c07640 1 -- 172.21.15.31:0/696494122 wait complete. 2024-09-16T09:05:55.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.041+0000 7fc5e811f640 1 -- 172.21.15.31:0/265059281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 msgr2=0x7fc5e00ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.041+0000 7fc5e811f640 1 --2- 172.21.15.31:0/265059281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e00ff1a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009880 tx=0x7fc5d002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 -- 172.21.15.31:0/265059281 shutdown_connections 2024-09-16T09:05:55.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 --2- 172.21.15.31:0/265059281 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e00ff1a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 --2- 172.21.15.31:0/265059281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00fe400 0x7fc5e00fe800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 -- 172.21.15.31:0/265059281 >> 172.21.15.31:0/265059281 conn(0x7fc5e00fa230 msgr2=0x7fc5e00fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:55.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 -- 172.21.15.31:0/265059281 shutdown_connections 2024-09-16T09:05:55.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 -- 172.21.15.31:0/265059281 wait complete. 2024-09-16T09:05:55.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 Processor -- start 2024-09-16T09:05:55.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 -- start start 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.042+0000 7fc5e811f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00fe400 0x7fc5e01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e811f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e0197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e811f640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e01982d0 con 0x7fc5e00fed40 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e811f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e0198440 con 0x7fc5e00fe400 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e56ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e0197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e56ba640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e0197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46004/0 (socket says 172.21.15.31:46004) 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e56ba640 1 -- 172.21.15.31:0/3479655699 learned_addr learned my addr 172.21.15.31:0/3479655699 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e5ebb640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00fe400 0x7fc5e01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.045 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e56ba640 1 -- 172.21.15.31:0/3479655699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00fe400 msgr2=0x7fc5e01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e56ba640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00fe400 0x7fc5e01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e56ba640 1 -- 172.21.15.31:0/3479655699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0009560 con 0x7fc5e00fed40 2024-09-16T09:05:55.046 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.043+0000 7fc5e56ba640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e0197d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00055f0 tx=0x7fc5d0005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:55.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.046+0000 7fc5d6ffd640 1 -- 172.21.15.31:0/3479655699 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5d0004b20 con 0x7fc5e00fed40 2024-09-16T09:05:55.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.046+0000 7fc5d6ffd640 1 -- 172.21.15.31:0/3479655699 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5d0004c80 con 0x7fc5e00fed40 2024-09-16T09:05:55.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.046+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e019ce80 con 0x7fc5e00fed40 2024-09-16T09:05:55.047 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.046+0000 7fc5d6ffd640 1 -- 172.21.15.31:0/3479655699 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5d0003bc0 con 0x7fc5e00fed40 2024-09-16T09:05:55.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.046+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e019d3c0 con 0x7fc5e00fed40 2024-09-16T09:05:55.048 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.048+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e00fe800 con 0x7fc5e00fed40 2024-09-16T09:05:55.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.048+0000 7fc5d6ffd640 1 -- 172.21.15.31:0/3479655699 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fc5d0031640 con 0x7fc5e00fed40 2024-09-16T09:05:55.049 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.049+0000 7fc5d6ffd640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc5cc07af30 0x7fc5cc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.049+0000 7fc5d6ffd640 1 -- 172.21.15.31:0/3479655699 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fc5d00bf250 con 0x7fc5e00fed40 2024-09-16T09:05:55.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.049+0000 7fc5e5ebb640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc5cc07af30 0x7fc5cc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.050 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.049+0000 7fc5e5ebb640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc5cc07af30 0x7fc5cc07d3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0058c0 tx=0x7fc5dc00a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:55.054 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.053+0000 7fc5d6ffd640 1 -- 172.21.15.31:0/3479655699 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fc5d00823f0 con 0x7fc5e00fed40 2024-09-16T09:05:55.329 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:55 smithi031 ceph-mon[91884]: pgmap v247: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.344+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc5e01918e0 con 0x7fc5e00fed40 2024-09-16T09:05:55.345 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.344+0000 7fc5d6ffd640 1 -- 172.21.15.31:0/3479655699 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fc5d0085510 con 0x7fc5e00fed40 2024-09-16T09:05:55.345 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:05:55.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:05:55.347 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:05:55.347 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 14 2024-09-16T09:05:55.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:05:55.347 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.346+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc5cc07af30 msgr2=0x7fc5cc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc5cc07af30 0x7fc5cc07d3f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0058c0 tx=0x7fc5dc00a2d0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 msgr2=0x7fc5e0197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e0197d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00055f0 tx=0x7fc5d0005620 comp rx=0 tx=0).stop 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 shutdown_connections 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fc5cc07af30 0x7fc5cc07d3f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5e00fed40 0x7fc5e0197d00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 --2- 172.21.15.31:0/3479655699 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5e00fe400 0x7fc5e01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.348 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 >> 172.21.15.31:0/3479655699 conn(0x7fc5e00fa230 msgr2=0x7fc5e0106e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 shutdown_connections 2024-09-16T09:05:55.349 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.347+0000 7fc5e811f640 1 -- 172.21.15.31:0/3479655699 wait complete. 2024-09-16T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.432+0000 7fde7fc75640 1 -- 172.21.15.31:0/2172340200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 msgr2=0x7fde78100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.432+0000 7fde7fc75640 1 --2- 172.21.15.31:0/2172340200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde78100170 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fde68009880 tx=0x7fde6802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.432+0000 7fde7fc75640 1 -- 172.21.15.31:0/2172340200 shutdown_connections 2024-09-16T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.432+0000 7fde7fc75640 1 --2- 172.21.15.31:0/2172340200 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde78100170 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.432+0000 7fde7fc75640 1 --2- 172.21.15.31:0/2172340200 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde780feaf0 0x7fde780feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.432+0000 7fde7fc75640 1 -- 172.21.15.31:0/2172340200 >> 172.21.15.31:0/2172340200 conn(0x7fde780fa280 msgr2=0x7fde780fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:55.434 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 -- 172.21.15.31:0/2172340200 shutdown_connections 2024-09-16T09:05:55.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 -- 172.21.15.31:0/2172340200 wait complete. 2024-09-16T09:05:55.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 Processor -- start 2024-09-16T09:05:55.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 -- start start 2024-09-16T09:05:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde780feaf0 0x7fde78102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde781010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78101610 con 0x7fde780ffcf0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.433+0000 7fde7fc75640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78101780 con 0x7fde780feaf0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7d210640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde781010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7d210640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde781010d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46022/0 (socket says 172.21.15.31:46022) 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7d210640 1 -- 172.21.15.31:0/3159822376 learned_addr learned my addr 172.21.15.31:0/3159822376 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7da11640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde780feaf0 0x7fde78102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7d210640 1 -- 172.21.15.31:0/3159822376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde780feaf0 msgr2=0x7fde78102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7d210640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde780feaf0 0x7fde78102a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7d210640 1 -- 172.21.15.31:0/3159822376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde68009560 con 0x7fde780ffcf0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7d210640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde781010d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde68004650 tx=0x7fde680055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde6effd640 1 -- 172.21.15.31:0/3159822376 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde68004aa0 con 0x7fde780ffcf0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde78101a00 con 0x7fde780ffcf0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.434+0000 7fde6effd640 1 -- 172.21.15.31:0/3159822376 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde68005d50 con 0x7fde780ffcf0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.435+0000 7fde6effd640 1 -- 172.21.15.31:0/3159822376 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde680039e0 con 0x7fde780ffcf0 2024-09-16T09:05:55.437 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.435+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78101ef0 con 0x7fde780ffcf0 2024-09-16T09:05:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.435+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde78107700 con 0x7fde780ffcf0 2024-09-16T09:05:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.436+0000 7fde6effd640 1 -- 172.21.15.31:0/3159822376 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fde68031c00 con 0x7fde780ffcf0 2024-09-16T09:05:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.437+0000 7fde6effd640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fde6407af30 0x7fde6407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.437+0000 7fde6effd640 1 -- 172.21.15.31:0/3159822376 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fde680bf130 con 0x7fde780ffcf0 2024-09-16T09:05:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.437+0000 7fde7da11640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fde6407af30 0x7fde6407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.438+0000 7fde7da11640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fde6407af30 0x7fde6407d3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde780ffb50 tx=0x7fde74005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:55.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.442+0000 7fde6effd640 1 -- 172.21.15.31:0/3159822376 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fde68082250 con 0x7fde780ffcf0 2024-09-16T09:05:55.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:55 smithi097 ceph-mon[79892]: pgmap v247: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:55.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.668+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fde781078e0 con 0x7fde780ffcf0 2024-09-16T09:05:55.669 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.668+0000 7fde6effd640 1 -- 172.21.15.31:0/3159822376 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1987 (secure 0 0 0) 0x7fde68085370 con 0x7fde780ffcf0 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:e38 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T09:05:32:642886+0000 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 38 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:05:55.670 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T09:05:32.642884+0000 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 99 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:05:55.671 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34430,1=34434} 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 34430 members: 34430,34434 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{0:34430} state up:active seq 17 join_fscid=1 addr [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{1:34434} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:55.672 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:55.673 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:05:55.673 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:05:55.673 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6824/257407185,v1:172.21.15.97:6825/257407185] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:05:55.673 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:44375} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6826/2995504601,v1:172.21.15.97:6827/2995504601] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:05:55.673 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fde6407af30 msgr2=0x7fde6407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fde6407af30 0x7fde6407d3f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fde780ffb50 tx=0x7fde74005f90 comp rx=0 tx=0).stop 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 msgr2=0x7fde781010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde781010d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde68004650 tx=0x7fde680055a0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 shutdown_connections 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fde6407af30 0x7fde6407d3f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde780ffcf0 0x7fde781010d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 --2- 172.21.15.31:0/3159822376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde780feaf0 0x7fde78102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 >> 172.21.15.31:0/3159822376 conn(0x7fde780fa280 msgr2=0x7fde78067d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 shutdown_connections 2024-09-16T09:05:55.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.672+0000 7fde7fc75640 1 -- 172.21.15.31:0/3159822376 wait complete. 2024-09-16T09:05:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 -- 172.21.15.31:0/81353050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470103030 msgr2=0x7f0470105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 --2- 172.21.15.31:0/81353050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470103030 0x7f0470105420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f045c009880 tx=0x7f045c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 -- 172.21.15.31:0/81353050 shutdown_connections 2024-09-16T09:05:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 --2- 172.21.15.31:0/81353050 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470103030 0x7f0470105420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 --2- 172.21.15.31:0/81353050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470100700 0x7f0470102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.763 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 -- 172.21.15.31:0/81353050 >> 172.21.15.31:0/81353050 conn(0x7f04700fa280 msgr2=0x7f04700fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 -- 172.21.15.31:0/81353050 shutdown_connections 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.758+0000 7f0476cd8640 1 -- 172.21.15.31:0/81353050 wait complete. 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0476cd8640 1 Processor -- start 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0476cd8640 1 -- start start 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0476cd8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470100700 0x7f04701956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0476cd8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470103030 0x7f0470195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0476cd8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04701961e0 con 0x7f0470100700 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0476cd8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470196350 con 0x7f0470103030 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0474a74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470100700 0x7f04701956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.764 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0474a74640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470100700 0x7f04701956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46052/0 (socket says 172.21.15.31:46052) 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.759+0000 7f0474a74640 1 -- 172.21.15.31:0/3771734521 learned_addr learned my addr 172.21.15.31:0/3771734521 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f0474a74640 1 -- 172.21.15.31:0/3771734521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470103030 msgr2=0x7f0470195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f046ffff640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470103030 0x7f0470195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f0474a74640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470103030 0x7f0470195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f0474a74640 1 -- 172.21.15.31:0/3771734521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c009560 con 0x7f0470100700 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f046ffff640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470103030 0x7f0470195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f0474a74640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470100700 0x7f04701956d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f046400d720 tx=0x7f046400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f046dffb640 1 -- 172.21.15.31:0/3771734521 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04640051f0 con 0x7f0470100700 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f046dffb640 1 -- 172.21.15.31:0/3771734521 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0464009ce0 con 0x7f0470100700 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047019adf0 con 0x7f0470100700 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f046dffb640 1 -- 172.21.15.31:0/3771734521 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0464010460 con 0x7f0470100700 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.760+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047019b2c0 con 0x7f0470100700 2024-09-16T09:05:55.765 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.761+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04701087f0 con 0x7f0470100700 2024-09-16T09:05:55.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.762+0000 7f046dffb640 1 -- 172.21.15.31:0/3771734521 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f0464005350 con 0x7f0470100700 2024-09-16T09:05:55.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.762+0000 7f046dffb640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f046007af30 0x7f046007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:55.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.763+0000 7f046dffb640 1 -- 172.21.15.31:0/3771734521 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f046409dc10 con 0x7f0470100700 2024-09-16T09:05:55.766 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.763+0000 7f046ffff640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f046007af30 0x7f046007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:55.767 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.763+0000 7f046ffff640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f046007af30 0x7f046007d3f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f045c000c00 tx=0x7f045c005de0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:55.768 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.767+0000 7f046dffb640 1 -- 172.21.15.31:0/3771734521 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f0464060c80 con 0x7f0470100700 2024-09-16T09:05:55.972 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.970+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f047019b4e0 con 0x7f046007af30 2024-09-16T09:05:55.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.972+0000 7f046dffb640 1 -- 172.21.15.31:0/3771734521 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f047019b4e0 con 0x7f046007af30 2024-09-16T09:05:55.973 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:05:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:05:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:05:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:05:55.973 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "osd", 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph-exporter", 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "mds" 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "18/23 daemons upgraded", 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading node-exporter daemons", 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:05:55.974 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:05:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f046007af30 msgr2=0x7f046007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f046007af30 0x7f046007d3f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f045c000c00 tx=0x7f045c005de0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.975 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470100700 msgr2=0x7f04701956d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470100700 0x7f04701956d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f046400d720 tx=0x7f046400dbe0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 shutdown_connections 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f046007af30 0x7f046007d3f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0470103030 0x7f0470195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 --2- 172.21.15.31:0/3771734521 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470100700 0x7f04701956d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 >> 172.21.15.31:0/3771734521 conn(0x7f04700fa280 msgr2=0x7f04700fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 shutdown_connections 2024-09-16T09:05:55.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:55.974+0000 7f0476cd8640 1 -- 172.21.15.31:0/3771734521 wait complete. 2024-09-16T09:05:56.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 -- 172.21.15.31:0/1675944460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00feaa0 msgr2=0x7ff0b00feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:56.062 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/1675944460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00feaa0 0x7ff0b00feea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009880 tx=0x7ff0a002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 -- 172.21.15.31:0/1675944460 shutdown_connections 2024-09-16T09:05:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/1675944460 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b00ffca0 0x7ff0b0100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/1675944460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00feaa0 0x7ff0b00feea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 -- 172.21.15.31:0/1675944460 >> 172.21.15.31:0/1675944460 conn(0x7ff0b00fa230 msgr2=0x7ff0b00fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 -- 172.21.15.31:0/1675944460 shutdown_connections 2024-09-16T09:05:56.063 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.061+0000 7ff0b5b94640 1 -- 172.21.15.31:0/1675944460 wait complete. 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.062+0000 7ff0b5b94640 1 Processor -- start 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.062+0000 7ff0b5b94640 1 -- start start 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.062+0000 7ff0b5b94640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b00feaa0 0x7ff0b0068fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.062+0000 7ff0b5b94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00ffca0 0x7ff0b019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.062+0000 7ff0b5b94640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b019a970 con 0x7ff0b00ffca0 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.062+0000 7ff0b5b94640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b019aab0 con 0x7ff0b00feaa0 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.062+0000 7ff0aeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00ffca0 0x7ff0b019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0aeffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00ffca0 0x7ff0b019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46068/0 (socket says 172.21.15.31:46068) 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0aeffd640 1 -- 172.21.15.31:0/2888797473 learned_addr learned my addr 172.21.15.31:0/2888797473 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0af7fe640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b00feaa0 0x7ff0b0068fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0aeffd640 1 -- 172.21.15.31:0/2888797473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b00feaa0 msgr2=0x7ff0b0068fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0aeffd640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b00feaa0 0x7ff0b0068fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0aeffd640 1 -- 172.21.15.31:0/2888797473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a0009560 con 0x7ff0b00ffca0 2024-09-16T09:05:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0aeffd640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00ffca0 0x7ff0b019a3a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e970 tx=0x7ff0a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0acff9640 1 -- 172.21.15.31:0/2888797473 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a400cc90 con 0x7ff0b00ffca0 2024-09-16T09:05:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0acff9640 1 -- 172.21.15.31:0/2888797473 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0a400cdf0 con 0x7ff0b00ffca0 2024-09-16T09:05:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b019ad60 con 0x7ff0b00ffca0 2024-09-16T09:05:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0acff9640 1 -- 172.21.15.31:0/2888797473 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0a4010430 con 0x7ff0b00ffca0 2024-09-16T09:05:56.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.063+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b019b330 con 0x7ff0b00ffca0 2024-09-16T09:05:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.064+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b018e640 con 0x7ff0b00ffca0 2024-09-16T09:05:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.065+0000 7ff0acff9640 1 -- 172.21.15.31:0/2888797473 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7ff0a40046c0 con 0x7ff0b00ffca0 2024-09-16T09:05:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.066+0000 7ff0acff9640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff09807af30 0x7ff09807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:05:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.066+0000 7ff0acff9640 1 -- 172.21.15.31:0/2888797473 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7ff0a4014070 con 0x7ff0b00ffca0 2024-09-16T09:05:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.066+0000 7ff0af7fe640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff09807af30 0x7ff09807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:05:56.067 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.067+0000 7ff0af7fe640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff09807af30 0x7ff09807d3f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009230 tx=0x7ff0a00314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:05:56.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.070+0000 7ff0acff9640 1 -- 172.21.15.31:0/2888797473 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ff0a4061640 con 0x7ff0b00ffca0 2024-09-16T09:05:56.278 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:56 smithi031 ceph-mon[91884]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:56.278 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:56 smithi031 ceph-mon[91884]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:56.278 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:56 smithi031 ceph-mon[91884]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:56.278 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:56 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3479655699' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:56.278 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:56 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3159822376' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:05:56.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.334+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff0b01078d0 con 0x7ff0b00ffca0 2024-09-16T09:05:56.336 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:05:56.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.334+0000 7ff0acff9640 1 -- 172.21.15.31:0/2888797473 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff0a4064760 con 0x7ff0b00ffca0 2024-09-16T09:05:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff09807af30 msgr2=0x7ff09807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff09807af30 0x7ff09807d3f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0a0009230 tx=0x7ff0a00314e0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00ffca0 msgr2=0x7ff0b019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:05:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00ffca0 0x7ff0b019a3a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff0a400e970 tx=0x7ff0a400ed70 comp rx=0 tx=0).stop 2024-09-16T09:05:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 shutdown_connections 2024-09-16T09:05:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff09807af30 0x7ff09807d3f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.339 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0b00ffca0 0x7ff0b019a3a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 --2- 172.21.15.31:0/2888797473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff0b00feaa0 0x7ff0b0068fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:05:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 >> 172.21.15.31:0/2888797473 conn(0x7ff0b00fa230 msgr2=0x7ff0b0103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:05:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 shutdown_connections 2024-09-16T09:05:56.340 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:05:56.337+0000 7ff0b5b94640 1 -- 172.21.15.31:0/2888797473 wait complete. 2024-09-16T09:05:56.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:56 smithi097 ceph-mon[79892]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:56.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:56 smithi097 ceph-mon[79892]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:56.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:56 smithi097 ceph-mon[79892]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:56.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:56 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3479655699' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:05:56.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:56 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3159822376' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:05:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:57 smithi031 ceph-mon[91884]: pgmap v248: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:57.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:57 smithi031 ceph-mon[91884]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:57.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:57 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2888797473' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:05:57.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:57.374 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:57 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:57.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:57 smithi097 ceph-mon[79892]: pgmap v248: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:57.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:57 smithi097 ceph-mon[79892]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:05:57.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:57 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2888797473' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:05:57.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:57.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:57 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:05:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:59 smithi031 ceph-mon[91884]: pgmap v249: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:59 smithi031 ceph-mon[91884]: Upgrade: Updating node-exporter.smithi097 (2/2) 2024-09-16T09:05:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:05:59 smithi031 ceph-mon[91884]: Deploying daemon node-exporter.smithi097 on smithi097 2024-09-16T09:05:59.438 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:59 smithi097 ceph-mon[79892]: pgmap v249: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:05:59.438 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:59 smithi097 ceph-mon[79892]: Upgrade: Updating node-exporter.smithi097 (2/2) 2024-09-16T09:05:59.438 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:05:59 smithi097 ceph-mon[79892]: Deploying daemon node-exporter.smithi097 on smithi097 2024-09-16T09:06:01.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:01 smithi031 ceph-mon[91884]: pgmap v250: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:01.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:01 smithi097 ceph-mon[79892]: pgmap v250: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:03 smithi031 ceph-mon[91884]: pgmap v251: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:03 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:03 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:03.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:03 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:03.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:03 smithi097 ceph-mon[79892]: pgmap v251: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:03.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:03 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:03.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:03 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:03.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:03 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:05 smithi031 ceph-mon[91884]: pgmap v252: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:05 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:05 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:05.497 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:05 smithi097 ceph-mon[79892]: pgmap v252: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:05.497 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:05 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:05.497 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:05 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:06.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:06.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:06 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:06.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:06 smithi031 ceph-mon[91884]: pgmap v253: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:06.497 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:06.497 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:06 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:06.497 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:06 smithi097 ceph-mon[79892]: pgmap v253: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:07.872 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:07.872 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:07.872 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:07.872 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:08.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:08.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:08.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:08.078 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:09.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:08 smithi031 ceph-mon[91884]: pgmap v254: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:09.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:09.016 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:09.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:08 smithi097 ceph-mon[79892]: pgmap v254: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:09.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:09.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:10.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:11.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:10 smithi031 ceph-mon[91884]: pgmap v255: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:11.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:11.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:11.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:06:11.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:06:11.124 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:10 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:11.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:10 smithi097 ceph-mon[79892]: pgmap v255: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:11.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:11.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:11.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:06:11.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:06:11.163 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:10 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.115 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.116 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.177 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:12.177 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.177 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.177 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.177 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.177 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:12.178 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:13.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:12 smithi031 ceph-mon[91884]: pgmap v256: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:12 smithi097 ceph-mon[79892]: pgmap v256: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:13 smithi031 ceph-mon[91884]: Upgrade: Updating prometheus.smithi031 2024-09-16T09:06:14.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:13 smithi031 ceph-mon[91884]: Deploying daemon prometheus.smithi031 on smithi031 2024-09-16T09:06:14.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:13 smithi097 ceph-mon[79892]: Upgrade: Updating prometheus.smithi031 2024-09-16T09:06:14.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:13 smithi097 ceph-mon[79892]: Deploying daemon prometheus.smithi031 on smithi031 2024-09-16T09:06:15.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:14 smithi031 ceph-mon[91884]: pgmap v257: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:14 smithi097 ceph-mon[79892]: pgmap v257: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:16 smithi031 ceph-mon[91884]: pgmap v258: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:17.253 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:16 smithi097 ceph-mon[79892]: pgmap v258: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:19.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:18 smithi031 ceph-mon[91884]: pgmap v259: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:19.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:18 smithi097 ceph-mon[79892]: pgmap v259: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:21.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:20 smithi031 ceph-mon[91884]: pgmap v260: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:21.215 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:20 smithi097 ceph-mon[79892]: pgmap v260: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:23.044 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:22 smithi031 ceph-mon[91884]: pgmap v261: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:23.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:22 smithi097 ceph-mon[79892]: pgmap v261: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:24.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:24 smithi031 ceph-mon[91884]: pgmap v262: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:25.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:25.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:24 smithi097 ceph-mon[79892]: pgmap v262: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:25.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:25 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:26.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:26.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:25 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 -- 172.21.15.31:0/1920050035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08102fb0 msgr2=0x7f4b081053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 --2- 172.21.15.31:0/1920050035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08102fb0 0x7f4b081053a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4af80098b0 tx=0x7f4af802a4d0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 -- 172.21.15.31:0/1920050035 shutdown_connections 2024-09-16T09:06:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 --2- 172.21.15.31:0/1920050035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08102fb0 0x7f4b081053a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 --2- 172.21.15.31:0/1920050035 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08100680 0x7f4b08102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 -- 172.21.15.31:0/1920050035 >> 172.21.15.31:0/1920050035 conn(0x7f4b080fa280 msgr2=0x7f4b080fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 -- 172.21.15.31:0/1920050035 shutdown_connections 2024-09-16T09:06:26.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 -- 172.21.15.31:0/1920050035 wait complete. 2024-09-16T09:06:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.426+0000 7f4b104f6640 1 Processor -- start 2024-09-16T09:06:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b104f6640 1 -- start start 2024-09-16T09:06:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b104f6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08100680 0x7f4b080fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b104f6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08102fb0 0x7f4b080fe630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b104f6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080feb70 con 0x7f4b08100680 2024-09-16T09:06:26.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b104f6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080fece0 con 0x7f4b08102fb0 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0e292640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08100680 0x7f4b080fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0e292640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08100680 0x7f4b080fffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53240/0 (socket says 172.21.15.31:53240) 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0e292640 1 -- 172.21.15.31:0/3850712834 learned_addr learned my addr 172.21.15.31:0/3850712834 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0da91640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08102fb0 0x7f4b080fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0e292640 1 -- 172.21.15.31:0/3850712834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08102fb0 msgr2=0x7f4b080fe630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0e292640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08102fb0 0x7f4b080fe630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0e292640 1 -- 172.21.15.31:0/3850712834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8009560 con 0x7f4b08100680 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0da91640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08102fb0 0x7f4b080fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4b0e292640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08100680 0x7f4b080fffe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400d8d0 tx=0x7f4b0400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.427+0000 7f4aff7fe640 1 -- 172.21.15.31:0/3850712834 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b0400bea0 con 0x7f4b08100680 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.428+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b080fefc0 con 0x7f4b08100680 2024-09-16T09:06:26.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.428+0000 7f4aff7fe640 1 -- 172.21.15.31:0/3850712834 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b04009c00 con 0x7f4b08100680 2024-09-16T09:06:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.428+0000 7f4aff7fe640 1 -- 172.21.15.31:0/3850712834 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b0400f710 con 0x7f4b08100680 2024-09-16T09:06:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.428+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0819d750 con 0x7f4b08100680 2024-09-16T09:06:26.431 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.429+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08100a60 con 0x7f4b08100680 2024-09-16T09:06:26.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.430+0000 7f4aff7fe640 1 -- 172.21.15.31:0/3850712834 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f4b0400f990 con 0x7f4b08100680 2024-09-16T09:06:26.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.430+0000 7f4aff7fe640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4af407af30 0x7f4af407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:26.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.430+0000 7f4aff7fe640 1 -- 172.21.15.31:0/3850712834 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f4b0409ddb0 con 0x7f4b08100680 2024-09-16T09:06:26.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.430+0000 7f4b0da91640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4af407af30 0x7f4af407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:26.432 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.431+0000 7f4b0da91640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4af407af30 0x7f4af407d3f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009880 tx=0x7f4af80314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:26.435 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.434+0000 7f4aff7fe640 1 -- 172.21.15.31:0/3850712834 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f4b04060f40 con 0x7f4b08100680 2024-09-16T09:06:26.640 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.639+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b08103a30 con 0x7f4af407af30 2024-09-16T09:06:26.641 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.640+0000 7f4aff7fe640 1 -- 172.21.15.31:0/3850712834 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f4b08103a30 con 0x7f4af407af30 2024-09-16T09:06:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4af407af30 msgr2=0x7f4af407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4af407af30 0x7f4af407d3f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009880 tx=0x7f4af80314e0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08100680 msgr2=0x7f4b080fffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08100680 0x7f4b080fffe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4b0400d8d0 tx=0x7f4b0400dd90 comp rx=0 tx=0).stop 2024-09-16T09:06:26.644 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 shutdown_connections 2024-09-16T09:06:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4af407af30 0x7f4af407d3f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4b08102fb0 0x7f4b080fe630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 --2- 172.21.15.31:0/3850712834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b08100680 0x7f4b080fffe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 >> 172.21.15.31:0/3850712834 conn(0x7f4b080fa280 msgr2=0x7f4b080fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 shutdown_connections 2024-09-16T09:06:26.645 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.643+0000 7f4b104f6640 1 -- 172.21.15.31:0/3850712834 wait complete. 2024-09-16T09:06:26.653 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:06:26.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.737+0000 7f6435b19640 1 -- 172.21.15.31:0/308537111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300744e0 msgr2=0x7f6430072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.737+0000 7f6435b19640 1 --2- 172.21.15.31:0/308537111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300744e0 0x7f6430072940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6420009880 tx=0x7f642002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.737+0000 7f6435b19640 1 -- 172.21.15.31:0/308537111 shutdown_connections 2024-09-16T09:06:26.739 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.737+0000 7f6435b19640 1 --2- 172.21.15.31:0/308537111 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6430072f10 0x7f6430073370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.737+0000 7f6435b19640 1 --2- 172.21.15.31:0/308537111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64300744e0 0x7f6430072940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.737+0000 7f6435b19640 1 -- 172.21.15.31:0/308537111 >> 172.21.15.31:0/308537111 conn(0x7f64300fb360 msgr2=0x7f64300fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.738+0000 7f6435b19640 1 -- 172.21.15.31:0/308537111 shutdown_connections 2024-09-16T09:06:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.738+0000 7f6435b19640 1 -- 172.21.15.31:0/308537111 wait complete. 2024-09-16T09:06:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.738+0000 7f6435b19640 1 Processor -- start 2024-09-16T09:06:26.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f6435b19640 1 -- start start 2024-09-16T09:06:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f6435b19640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6430072f10 0x7f643019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f6435b19640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64300744e0 0x7f643019c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f6435b19640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643019c860 con 0x7f6430072f10 2024-09-16T09:06:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f6435b19640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643019c9d0 con 0x7f64300744e0 2024-09-16T09:06:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f642f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6430072f10 0x7f643019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f642f7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6430072f10 0x7f643019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53260/0 (socket says 172.21.15.31:53260) 2024-09-16T09:06:26.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f642f7fe640 1 -- 172.21.15.31:0/881916577 learned_addr learned my addr 172.21.15.31:0/881916577 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.739+0000 7f642effd640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64300744e0 0x7f643019c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f642f7fe640 1 -- 172.21.15.31:0/881916577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64300744e0 msgr2=0x7f643019c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f642f7fe640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64300744e0 0x7f643019c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f642f7fe640 1 -- 172.21.15.31:0/881916577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6420009560 con 0x7f6430072f10 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f642f7fe640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6430072f10 0x7f643019bd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6420005610 tx=0x7f6420005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f642cff9640 1 -- 172.21.15.31:0/881916577 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6420004ba0 con 0x7f6430072f10 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f642cff9640 1 -- 172.21.15.31:0/881916577 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6420004d00 con 0x7f6430072f10 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f642cff9640 1 -- 172.21.15.31:0/881916577 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6420003bd0 con 0x7f6430072f10 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64301a1410 con 0x7f6430072f10 2024-09-16T09:06:26.742 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.740+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64301a1980 con 0x7f6430072f10 2024-09-16T09:06:26.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.742+0000 7f642cff9640 1 -- 172.21.15.31:0/881916577 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f6420031640 con 0x7f6430072f10 2024-09-16T09:06:26.743 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.743+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6430104850 con 0x7f6430072f10 2024-09-16T09:06:26.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.743+0000 7f642cff9640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f641807af30 0x7f641807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:26.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.743+0000 7f642cff9640 1 -- 172.21.15.31:0/881916577 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f64200bf320 con 0x7f6430072f10 2024-09-16T09:06:26.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.744+0000 7f642effd640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f641807af30 0x7f641807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:26.745 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.744+0000 7f642effd640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f641807af30 0x7f641807d3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f643019d270 tx=0x7f64240090e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:26.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.751+0000 7f642cff9640 1 -- 172.21.15.31:0/881916577 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6420082420 con 0x7f6430072f10 2024-09-16T09:06:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.972+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f643002c450 con 0x7f641807af30 2024-09-16T09:06:26.973 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.973+0000 7f642cff9640 1 -- 172.21.15.31:0/881916577 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f643002c450 con 0x7f641807af30 2024-09-16T09:06:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f641807af30 msgr2=0x7f641807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f641807af30 0x7f641807d3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f643019d270 tx=0x7f64240090e0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.976 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6430072f10 msgr2=0x7f643019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6430072f10 0x7f643019bd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6420005610 tx=0x7f6420005640 comp rx=0 tx=0).stop 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 shutdown_connections 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f641807af30 0x7f641807d3f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64300744e0 0x7f643019c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 --2- 172.21.15.31:0/881916577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6430072f10 0x7f643019bd50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 >> 172.21.15.31:0/881916577 conn(0x7f64300fb360 msgr2=0x7f6430101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 shutdown_connections 2024-09-16T09:06:26.977 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:26.975+0000 7f6435b19640 1 -- 172.21.15.31:0/881916577 wait complete. 2024-09-16T09:06:27.014 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:26 smithi031 ceph-mon[91884]: pgmap v263: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:27.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 -- 172.21.15.31:0/483277765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0100630 msgr2=0x7f46f0102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 --2- 172.21.15.31:0/483277765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0100630 0x7f46f0102a20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46e00098b0 tx=0x7f46e002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 -- 172.21.15.31:0/483277765 shutdown_connections 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 --2- 172.21.15.31:0/483277765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f0102f60 0x7f46f0105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 --2- 172.21.15.31:0/483277765 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0100630 0x7f46f0102a20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 -- 172.21.15.31:0/483277765 >> 172.21.15.31:0/483277765 conn(0x7f46f00fa250 msgr2=0x7f46f00fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 -- 172.21.15.31:0/483277765 shutdown_connections 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.064+0000 7f46f5420640 1 -- 172.21.15.31:0/483277765 wait complete. 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46f5420640 1 Processor -- start 2024-09-16T09:06:27.066 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46f5420640 1 -- start start 2024-09-16T09:06:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46f5420640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f0100630 0x7f46f0195640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46f5420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0102f60 0x7f46f0195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46f5420640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f0196150 con 0x7f46f0102f60 2024-09-16T09:06:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46f5420640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f01962c0 con 0x7f46f0100630 2024-09-16T09:06:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46ee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0102f60 0x7f46f0195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.068 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46ee7fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0102f60 0x7f46f0195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53272/0 (socket says 172.21.15.31:53272) 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46ee7fc640 1 -- 172.21.15.31:0/1763749610 learned_addr learned my addr 172.21.15.31:0/1763749610 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.065+0000 7f46eeffd640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f0100630 0x7f46f0195640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46ee7fc640 1 -- 172.21.15.31:0/1763749610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f0100630 msgr2=0x7f46f0195640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46ee7fc640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f0100630 0x7f46f0195640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46ee7fc640 1 -- 172.21.15.31:0/1763749610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009560 con 0x7f46f0102f60 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46ee7fc640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0102f60 0x7f46f0195b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d720 tx=0x7f46e400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46d7fff640 1 -- 172.21.15.31:0/1763749610 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e40099f0 con 0x7f46f0102f60 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f019ad60 con 0x7f46f0102f60 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46d7fff640 1 -- 172.21.15.31:0/1763749610 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46e4009b50 con 0x7f46f0102f60 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46d7fff640 1 -- 172.21.15.31:0/1763749610 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e400f5b0 con 0x7f46f0102f60 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.066+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f019b330 con 0x7f46f0102f60 2024-09-16T09:06:27.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.067+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f01087c0 con 0x7f46f0102f60 2024-09-16T09:06:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.068+0000 7f46d7fff640 1 -- 172.21.15.31:0/1763749610 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f46e4009cc0 con 0x7f46f0102f60 2024-09-16T09:06:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.069+0000 7f46d7fff640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46d807af30 0x7f46d807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.069+0000 7f46d7fff640 1 -- 172.21.15.31:0/1763749610 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f46e409dea0 con 0x7f46f0102f60 2024-09-16T09:06:27.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.069+0000 7f46eeffd640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46d807af30 0x7f46d807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.069+0000 7f46eeffd640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46d807af30 0x7f46d807d3f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f46e0006010 tx=0x7f46e00314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:27.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.073+0000 7f46d7fff640 1 -- 172.21.15.31:0/1763749610 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f46e4060e70 con 0x7f46f0102f60 2024-09-16T09:06:27.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:26 smithi097 ceph-mon[79892]: pgmap v263: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:27.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.286+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f46f019b520 con 0x7f46d807af30 2024-09-16T09:06:27.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.296+0000 7f46d7fff640 1 -- 172.21.15.31:0/1763749610 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f46f019b520 con 0x7f46d807af30 2024-09-16T09:06:27.297 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:06:27.297 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (15m) 20s ago 17m 25.7M - 0.25.0 c8568f914cd2 8b6c75115fa2 2024-09-16T09:06:27.297 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (51s) 20s ago 18m 8388k - 19.3.0-4917-g8293d73f 425e316ba8eb 119433b14707 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (45s) 22s ago 17m 8124k - 19.3.0-4917-g8293d73f 425e316ba8eb 3f8a92f1222e 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (6m) 20s ago 18m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (6m) 22s ago 17m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (15m) 20s ago 17m 86.2M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (117s) 20s ago 11m 84.9M - 19.3.0-4917-g8293d73f 425e316ba8eb 7e4ca0c031e9 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (98s) 20s ago 11m 16.5M - 19.3.0-4917-g8293d73f 425e316ba8eb f54e4116709a 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (81s) 22s ago 11m 16.5M - 19.3.0-4917-g8293d73f 425e316ba8eb 880891e445c6 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (63s) 22s ago 11m 16.2M - 19.3.0-4917-g8293d73f 425e316ba8eb 08131d73c31f 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (8m) 20s ago 19m 523M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (7m) 22s ago 16m 456M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (7m) 20s ago 19m 57.7M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (6m) 22s ago 16m 45.4M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:06:27.298 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (31s) 20s ago 18m 5226k - 1.7.0 72c9c2088986 7588c21d20a2 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (25s) 22s ago 16m 3502k - 1.7.0 72c9c2088986 818f167b5f3e 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (5m) 20s ago 15m 118M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 20s ago 14m 115M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (4m) 20s ago 14m 133M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (3m) 22s ago 14m 136M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (3m) 22s ago 13m 98.9M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (2m) 22s ago 13m 109M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:06:27.299 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 starting - - - - 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46d807af30 msgr2=0x7f46d807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46d807af30 0x7f46d807d3f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f46e0006010 tx=0x7f46e00314e0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0102f60 msgr2=0x7f46f0195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0102f60 0x7f46f0195b80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f46e400d720 tx=0x7f46e400dbe0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 shutdown_connections 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f46d807af30 0x7f46d807d3f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46f0102f60 0x7f46f0195b80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 --2- 172.21.15.31:0/1763749610 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46f0100630 0x7f46f0195640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.298+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 >> 172.21.15.31:0/1763749610 conn(0x7f46f00fa250 msgr2=0x7f46f00fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:27.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.299+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 shutdown_connections 2024-09-16T09:06:27.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.299+0000 7f46f5420640 1 -- 172.21.15.31:0/1763749610 wait complete. 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.380+0000 7fac07dae640 1 -- 172.21.15.31:0/3890820545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 msgr2=0x7fac000fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.380+0000 7fac07dae640 1 --2- 172.21.15.31:0/3890820545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac000fff90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009880 tx=0x7fabfc02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.380+0000 7fac07dae640 1 -- 172.21.15.31:0/3890820545 shutdown_connections 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 --2- 172.21.15.31:0/3890820545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac00100d90 0x7fac00101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 --2- 172.21.15.31:0/3890820545 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac000fff90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 -- 172.21.15.31:0/3890820545 >> 172.21.15.31:0/3890820545 conn(0x7fac000fb320 msgr2=0x7fac000fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 -- 172.21.15.31:0/3890820545 shutdown_connections 2024-09-16T09:06:27.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 -- 172.21.15.31:0/3890820545 wait complete. 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 Processor -- start 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 -- start start 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac00074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.381+0000 7fac07dae640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac00100d90 0x7fac000728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac07dae640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00074790 con 0x7fac000ffb90 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac07dae640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac00072e10 con 0x7fac00100d90 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05b4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac00074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05b4a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac00074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53284/0 (socket says 172.21.15.31:53284) 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05b4a640 1 -- 172.21.15.31:0/3045335014 learned_addr learned my addr 172.21.15.31:0/3045335014 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:27.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05349640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac00100d90 0x7fac000728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05b4a640 1 -- 172.21.15.31:0/3045335014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac00100d90 msgr2=0x7fac000728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05b4a640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac00100d90 0x7fac000728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05b4a640 1 -- 172.21.15.31:0/3045335014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc009560 con 0x7fac000ffb90 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac05b4a640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac00074250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fabfc006010 tx=0x7fabfc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fabf6ffd640 1 -- 172.21.15.31:0/3045335014 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabfc004aa0 con 0x7fac000ffb90 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac00073090 con 0x7fac000ffb90 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fabf6ffd640 1 -- 172.21.15.31:0/3045335014 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabfc004c00 con 0x7fac000ffb90 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.382+0000 7fabf6ffd640 1 -- 172.21.15.31:0/3045335014 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabfc0039b0 con 0x7fac000ffb90 2024-09-16T09:06:27.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.383+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac00073600 con 0x7fac000ffb90 2024-09-16T09:06:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.384+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac0018f710 con 0x7fac000ffb90 2024-09-16T09:06:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.384+0000 7fabf6ffd640 1 -- 172.21.15.31:0/3045335014 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fabfc031c00 con 0x7fac000ffb90 2024-09-16T09:06:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.385+0000 7fabf6ffd640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabec07af30 0x7fabec07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.385+0000 7fac05349640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabec07af30 0x7fabec07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.385+0000 7fabf6ffd640 1 -- 172.21.15.31:0/3045335014 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fabfc0bf260 con 0x7fac000ffb90 2024-09-16T09:06:27.387 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.385+0000 7fac05349640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabec07af30 0x7fabec07d3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fac00073fd0 tx=0x7fabf000a2d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:27.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.389+0000 7fabf6ffd640 1 -- 172.21.15.31:0/3045335014 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fabfc0822d0 con 0x7fac000ffb90 2024-09-16T09:06:27.657 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.656+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fac0018f180 con 0x7fac000ffb90 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.657+0000 7fabf6ffd640 1 -- 172.21.15.31:0/3045335014 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fabfc0853f0 con 0x7fac000ffb90 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:06:27.658 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 14 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:06:27.659 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:06:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabec07af30 msgr2=0x7fabec07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabec07af30 0x7fabec07d3f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fac00073fd0 tx=0x7fabf000a2d0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 msgr2=0x7fac00074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac00074250 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fabfc006010 tx=0x7fabfc0055a0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 shutdown_connections 2024-09-16T09:06:27.660 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabec07af30 0x7fabec07d3f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac00100d90 0x7fac000728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 --2- 172.21.15.31:0/3045335014 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac000ffb90 0x7fac00074250 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 >> 172.21.15.31:0/3045335014 conn(0x7fac000fb320 msgr2=0x7fac00104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 shutdown_connections 2024-09-16T09:06:27.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.659+0000 7fac07dae640 1 -- 172.21.15.31:0/3045335014 wait complete. 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 -- 172.21.15.31:0/3694872783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b0100d90 msgr2=0x7fb3b0101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3694872783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b0100d90 0x7fb3b0101210 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009880 tx=0x7fb3a402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 -- 172.21.15.31:0/3694872783 shutdown_connections 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3694872783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b0100d90 0x7fb3b0101210 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3694872783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b00ffb90 0x7fb3b00fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 -- 172.21.15.31:0/3694872783 >> 172.21.15.31:0/3694872783 conn(0x7fb3b00fb320 msgr2=0x7fb3b00fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 -- 172.21.15.31:0/3694872783 shutdown_connections 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.742+0000 7fb3b4829640 1 -- 172.21.15.31:0/3694872783 wait complete. 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3b4829640 1 Processor -- start 2024-09-16T09:06:27.744 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3b4829640 1 -- start start 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3b4829640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b00ffb90 0x7fb3b0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3b4829640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b0100d90 0x7fb3b00728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3b4829640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0072de0 con 0x7fb3b00ffb90 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3b4829640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0072f50 con 0x7fb3b0100d90 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b00ffb90 0x7fb3b0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3ae59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b00ffb90 0x7fb3b0074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53310/0 (socket says 172.21.15.31:53310) 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3ae59c640 1 -- 172.21.15.31:0/3486507387 learned_addr learned my addr 172.21.15.31:0/3486507387 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.743+0000 7fb3add9b640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b0100d90 0x7fb3b00728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb3ae59c640 1 -- 172.21.15.31:0/3486507387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b0100d90 msgr2=0x7fb3b00728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb3ae59c640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b0100d90 0x7fb3b00728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.746 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb3ae59c640 1 -- 172.21.15.31:0/3486507387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a4009560 con 0x7fb3b00ffb90 2024-09-16T09:06:27.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb3ae59c640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b00ffb90 0x7fb3b0074250 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000d8d0 tx=0x7fb3a000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:27.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb39f7fe640 1 -- 172.21.15.31:0/3486507387 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a0009a30 con 0x7fb3b00ffb90 2024-09-16T09:06:27.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b0073230 con 0x7fb3b00ffb90 2024-09-16T09:06:27.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb39f7fe640 1 -- 172.21.15.31:0/3486507387 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3a0009b90 con 0x7fb3b00ffb90 2024-09-16T09:06:27.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb39f7fe640 1 -- 172.21.15.31:0/3486507387 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3a000f680 con 0x7fb3b00ffb90 2024-09-16T09:06:27.747 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.744+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b019d690 con 0x7fb3b00ffb90 2024-09-16T09:06:27.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.745+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b00fff90 con 0x7fb3b00ffb90 2024-09-16T09:06:27.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.746+0000 7fb39f7fe640 1 -- 172.21.15.31:0/3486507387 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fb3a0009d00 con 0x7fb3b00ffb90 2024-09-16T09:06:27.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.746+0000 7fb39f7fe640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb39807af30 0x7fb39807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:27.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.747+0000 7fb39f7fe640 1 -- 172.21.15.31:0/3486507387 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fb3a009dec0 con 0x7fb3b00ffb90 2024-09-16T09:06:27.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.747+0000 7fb3add9b640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb39807af30 0x7fb39807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:27.748 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.747+0000 7fb3add9b640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb39807af30 0x7fb39807d3f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a40314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:27.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.751+0000 7fb39f7fe640 1 -- 172.21.15.31:0/3486507387 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fb3a0060f30 con 0x7fb3b00ffb90 2024-09-16T09:06:27.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:27 smithi031 ceph-mon[91884]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:27.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:27 smithi031 ceph-mon[91884]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:27.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:27 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3045335014' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:27.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.984+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb3b002c3e0 con 0x7fb3b00ffb90 2024-09-16T09:06:27.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.985+0000 7fb39f7fe640 1 -- 172.21.15.31:0/3486507387 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1987 (secure 0 0 0) 0x7fb3a0064050 con 0x7fb3b00ffb90 2024-09-16T09:06:27.986 INFO:teuthology.orchestra.run.smithi031.stdout:e38 2024-09-16T09:06:27.986 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T09:05:32:642886+0000 2024-09-16T09:06:27.986 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 38 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T09:05:32.642884+0000 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:06:27.987 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 99 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34430,1=34434} 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:06:27.988 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 34430 members: 34430,34434 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{0:34430} state up:active seq 17 join_fscid=1 addr [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{1:34434} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:06:27.989 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6824/257407185,v1:172.21.15.97:6825/257407185] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:06:27.990 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:44375} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6826/2995504601,v1:172.21.15.97:6827/2995504601] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:06:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2024-09-16T09:06:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb39807af30 msgr2=0x7fb39807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb39807af30 0x7fb39807d3f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4000c00 tx=0x7fb3a40314e0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b00ffb90 msgr2=0x7fb3b0074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b00ffb90 0x7fb3b0074250 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000d8d0 tx=0x7fb3a000dd90 comp rx=0 tx=0).stop 2024-09-16T09:06:27.990 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 shutdown_connections 2024-09-16T09:06:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb39807af30 0x7fb39807d3f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3b0100d90 0x7fb3b00728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 --2- 172.21.15.31:0/3486507387 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3b00ffb90 0x7fb3b0074250 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 >> 172.21.15.31:0/3486507387 conn(0x7fb3b00fb320 msgr2=0x7fb3b0104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 shutdown_connections 2024-09-16T09:06:27.991 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:27.987+0000 7fb3b4829640 1 -- 172.21.15.31:0/3486507387 wait complete. 2024-09-16T09:06:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 -- 172.21.15.31:0/4280743973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f00ffb90 msgr2=0x7f47f00fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4280743973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f00ffb90 0x7f47f00fff90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009880 tx=0x7f47ec02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 -- 172.21.15.31:0/4280743973 shutdown_connections 2024-09-16T09:06:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4280743973 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f0100d90 0x7f47f0101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4280743973 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f00ffb90 0x7f47f00fff90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 -- 172.21.15.31:0/4280743973 >> 172.21.15.31:0/4280743973 conn(0x7f47f00fb320 msgr2=0x7f47f00fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 -- 172.21.15.31:0/4280743973 shutdown_connections 2024-09-16T09:06:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.073+0000 7f47f7e85640 1 -- 172.21.15.31:0/4280743973 wait complete. 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f7e85640 1 Processor -- start 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f7e85640 1 -- start start 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f7e85640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f00ffb90 0x7f47f0197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f7e85640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f0100d90 0x7f47f0197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f7e85640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0198330 con 0x7f47f0100d90 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f7e85640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f01984a0 con 0x7f47f00ffb90 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f0100d90 0x7f47f0197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5420640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f0100d90 0x7f47f0197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53340/0 (socket says 172.21.15.31:53340) 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5420640 1 -- 172.21.15.31:0/4146125920 learned_addr learned my addr 172.21.15.31:0/4146125920 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5c21640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f00ffb90 0x7f47f0197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5420640 1 -- 172.21.15.31:0/4146125920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f00ffb90 msgr2=0x7f47f0197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5420640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f00ffb90 0x7f47f0197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5420640 1 -- 172.21.15.31:0/4146125920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ec009560 con 0x7f47f0100d90 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5c21640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f00ffb90 0x7f47f0197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.074+0000 7f47f5420640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f0100d90 0x7f47f0197d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d900 tx=0x7f47e000ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.075+0000 7f47e6ffd640 1 -- 172.21.15.31:0/4146125920 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47e0009c00 con 0x7f47f0100d90 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.075+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f019cf40 con 0x7f47f0100d90 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.075+0000 7f47e6ffd640 1 -- 172.21.15.31:0/4146125920 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47e000b660 con 0x7f47f0100d90 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.075+0000 7f47e6ffd640 1 -- 172.21.15.31:0/4146125920 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47e000f680 con 0x7f47f0100d90 2024-09-16T09:06:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.075+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f019d4e0 con 0x7f47f0100d90 2024-09-16T09:06:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.076+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f00fff90 con 0x7f47f0100d90 2024-09-16T09:06:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.076+0000 7f47e6ffd640 1 -- 172.21.15.31:0/4146125920 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f47e0010ab0 con 0x7f47f0100d90 2024-09-16T09:06:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.077+0000 7f47e6ffd640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f47dc07af30 0x7f47dc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:28.078 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.077+0000 7f47e6ffd640 1 -- 172.21.15.31:0/4146125920 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f47e009df40 con 0x7f47f0100d90 2024-09-16T09:06:28.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.077+0000 7f47f5c21640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f47dc07af30 0x7f47dc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:28.079 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.078+0000 7f47f5c21640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f47dc07af30 0x7f47dc07d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009230 tx=0x7f47ec0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:28.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.082+0000 7f47e6ffd640 1 -- 172.21.15.31:0/4146125920 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f47e0060f10 con 0x7f47f0100d90 2024-09-16T09:06:28.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:27 smithi097 ceph-mon[79892]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:28.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:27 smithi097 ceph-mon[79892]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:28.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:27 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3045335014' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:28.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.288+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47f002c450 con 0x7f47dc07af30 2024-09-16T09:06:28.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.289+0000 7f47e6ffd640 1 -- 172.21.15.31:0/4146125920 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+463 (secure 0 0 0) 0x7f47f002c450 con 0x7f47dc07af30 2024-09-16T09:06:28.292 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:06:28.292 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:06:28.292 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "osd", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph-exporter", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "mds" 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "18/23 daemons upgraded", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading prometheus daemons", 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:06:28.293 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:06:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f47dc07af30 msgr2=0x7f47dc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f47dc07af30 0x7f47dc07d3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009230 tx=0x7f47ec0314e0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f0100d90 msgr2=0x7f47f0197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f0100d90 0x7f47f0197d60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f47e000d900 tx=0x7f47e000ddc0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 shutdown_connections 2024-09-16T09:06:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f47dc07af30 0x7f47dc07d3f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47f0100d90 0x7f47f0197d60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 --2- 172.21.15.31:0/4146125920 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47f00ffb90 0x7f47f0197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 >> 172.21.15.31:0/4146125920 conn(0x7f47f00fb320 msgr2=0x7f47f0072b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 shutdown_connections 2024-09-16T09:06:28.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.292+0000 7f47f7e85640 1 -- 172.21.15.31:0/4146125920 wait complete. 2024-09-16T09:06:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 -- 172.21.15.31:0/904466827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00100740 msgr2=0x7f7f00102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 --2- 172.21.15.31:0/904466827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00100740 0x7f7f00102b30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009880 tx=0x7f7ef402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 -- 172.21.15.31:0/904466827 shutdown_connections 2024-09-16T09:06:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 --2- 172.21.15.31:0/904466827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f00103070 0x7f7f00105460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 --2- 172.21.15.31:0/904466827 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00100740 0x7f7f00102b30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 -- 172.21.15.31:0/904466827 >> 172.21.15.31:0/904466827 conn(0x7f7f000fa280 msgr2=0x7f7f000fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 -- 172.21.15.31:0/904466827 shutdown_connections 2024-09-16T09:06:28.380 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.378+0000 7f7f06dab640 1 -- 172.21.15.31:0/904466827 wait complete. 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7f06dab640 1 Processor -- start 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7f06dab640 1 -- start start 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7f06dab640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f00100740 0x7f7f001956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7f06dab640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00103070 0x7f7f00195c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7f06dab640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f001961f0 con 0x7f7f00103070 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7f06dab640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00196360 con 0x7f7f00100740 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7efffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00103070 0x7f7f00195c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7efffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00103070 0x7f7f00195c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:53356/0 (socket says 172.21.15.31:53356) 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7efffff640 1 -- 172.21.15.31:0/3643039467 learned_addr learned my addr 172.21.15.31:0/3643039467 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.379+0000 7f7f04b47640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f00100740 0x7f7f001956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7efffff640 1 -- 172.21.15.31:0/3643039467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f00100740 msgr2=0x7f7f001956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7efffff640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f00100740 0x7f7f001956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7efffff640 1 -- 172.21.15.31:0/3643039467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef4009560 con 0x7f7f00103070 2024-09-16T09:06:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7efffff640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00103070 0x7f7f00195c20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00d6f0 tx=0x7f7eec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7efdffb640 1 -- 172.21.15.31:0/3643039467 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eec0051f0 con 0x7f7f00103070 2024-09-16T09:06:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7efdffb640 1 -- 172.21.15.31:0/3643039467 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7eec009ce0 con 0x7f7f00103070 2024-09-16T09:06:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7efdffb640 1 -- 172.21.15.31:0/3643039467 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eec004e80 con 0x7f7f00103070 2024-09-16T09:06:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.380+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0019ae00 con 0x7f7f00103070 2024-09-16T09:06:28.383 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.381+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0019b320 con 0x7f7f00103070 2024-09-16T09:06:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.382+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f00100b20 con 0x7f7f00103070 2024-09-16T09:06:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.382+0000 7f7efdffb640 1 -- 172.21.15.31:0/3643039467 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f7eec010460 con 0x7f7f00103070 2024-09-16T09:06:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.383+0000 7f7efdffb640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7ef007af30 0x7f7ef007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.383+0000 7f7efdffb640 1 -- 172.21.15.31:0/3643039467 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f7eec09d950 con 0x7f7f00103070 2024-09-16T09:06:28.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.383+0000 7f7f04b47640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7ef007af30 0x7f7ef007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:28.385 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.383+0000 7f7f04b47640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7ef007af30 0x7f7ef007d3f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005750 tx=0x7f7ef40314a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:28.388 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.387+0000 7f7efdffb640 1 -- 172.21.15.31:0/3643039467 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f7eec060af0 con 0x7f7f00103070 2024-09-16T09:06:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.673+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7f00108830 con 0x7f7f00103070 2024-09-16T09:06:28.675 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.674+0000 7f7efdffb640 1 -- 172.21.15.31:0/3643039467 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f7eec063c10 con 0x7f7f00103070 2024-09-16T09:06:28.675 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:06:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7ef007af30 msgr2=0x7f7ef007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7ef007af30 0x7f7ef007d3f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4005750 tx=0x7f7ef40314a0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00103070 msgr2=0x7f7f00195c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00103070 0x7f7f00195c20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00d6f0 tx=0x7f7eec00dbb0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 shutdown_connections 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7ef007af30 0x7f7ef007d3f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f00103070 0x7f7f00195c20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 --2- 172.21.15.31:0/3643039467 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f00100740 0x7f7f001956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 >> 172.21.15.31:0/3643039467 conn(0x7f7f000fa280 msgr2=0x7f7f000fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 shutdown_connections 2024-09-16T09:06:28.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:28.676+0000 7f7f06dab640 1 -- 172.21.15.31:0/3643039467 wait complete. 2024-09-16T09:06:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:28 smithi031 ceph-mon[91884]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:28 smithi031 ceph-mon[91884]: pgmap v264: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:28 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3486507387' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:06:29.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:28 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3643039467' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:06:29.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:28 smithi097 ceph-mon[79892]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:29.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:28 smithi097 ceph-mon[79892]: pgmap v264: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:29.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:28 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3486507387' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:06:29.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:28 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3643039467' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:06:30.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:29 smithi097 ceph-mon[79892]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:30.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:29 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:30.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:29 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:30.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:29 smithi031 ceph-mon[91884]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:30.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:29 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:30.290 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:29 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:31.245 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:30 smithi031 ceph-mon[91884]: pgmap v265: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:31.245 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:31.245 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:30 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:31.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:30 smithi097 ceph-mon[79892]: pgmap v265: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:31.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:31.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:30 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:33.240 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:32 smithi031 ceph-mon[91884]: pgmap v266: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:33.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:32 smithi097 ceph-mon[79892]: pgmap v266: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:35.256 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: pgmap v267: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.257 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.258 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:34 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: pgmap v267: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:35.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:35.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:34 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:37.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:36 smithi031 ceph-mon[91884]: Upgrade: Updating alertmanager.smithi031 2024-09-16T09:06:37.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:36 smithi031 ceph-mon[91884]: Deploying daemon alertmanager.smithi031 on smithi031 2024-09-16T09:06:37.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:36 smithi031 ceph-mon[91884]: pgmap v268: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:36 smithi097 ceph-mon[79892]: Upgrade: Updating alertmanager.smithi031 2024-09-16T09:06:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:36 smithi097 ceph-mon[79892]: Deploying daemon alertmanager.smithi031 on smithi031 2024-09-16T09:06:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:36 smithi097 ceph-mon[79892]: pgmap v268: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:39.159 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:38 smithi031 ceph-mon[91884]: pgmap v269: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:39.160 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:39.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:38 smithi097 ceph-mon[79892]: pgmap v269: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:41.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:40 smithi031 ceph-mon[91884]: pgmap v270: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:41.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:40 smithi097 ceph-mon[79892]: pgmap v270: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:43.117 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:42 smithi031 ceph-mon[91884]: pgmap v271: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:42 smithi097 ceph-mon[79892]: pgmap v271: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:45.169 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:44 smithi031 ceph-mon[91884]: pgmap v272: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:45.169 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:44 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:45.169 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:44 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:45.169 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:44 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:44 smithi097 ceph-mon[79892]: pgmap v272: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:44 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:44 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:44 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:47.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:46 smithi031 ceph-mon[91884]: pgmap v273: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:46 smithi097 ceph-mon[79892]: pgmap v273: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:48 smithi097 ceph-mon[79892]: pgmap v274: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:48 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:48 smithi031 ceph-mon[91884]: pgmap v274: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:49.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:48 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:50 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:50 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:50.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:50 smithi031 ceph-mon[91884]: pgmap v275: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:50.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:50 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:50.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:50 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:50.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:50 smithi097 ceph-mon[79892]: pgmap v275: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:52.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:52 smithi097 ceph-mon[79892]: pgmap v276: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:52.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:52 smithi031 ceph-mon[91884]: pgmap v276: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:53.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:53.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:06:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:06:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:06:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T09:06:54.013 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T09:06:54.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:06:54.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.014 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.015 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:54.016 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:06:55.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:54 smithi097 ceph-mon[79892]: pgmap v277: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:55.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:55.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:54 smithi097 ceph-mon[79892]: Upgrade: Updating grafana.smithi031 2024-09-16T09:06:55.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:54 smithi097 ceph-mon[79892]: Deploying daemon grafana.smithi031 on smithi031 2024-09-16T09:06:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:54 smithi031 ceph-mon[91884]: pgmap v277: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:06:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:54 smithi031 ceph-mon[91884]: Upgrade: Updating grafana.smithi031 2024-09-16T09:06:55.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:54 smithi031 ceph-mon[91884]: Deploying daemon grafana.smithi031 on smithi031 2024-09-16T09:06:57.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:56 smithi097 ceph-mon[79892]: pgmap v278: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:56 smithi031 ceph-mon[91884]: pgmap v278: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:58.797 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:58 smithi031 ceph-mon[91884]: pgmap v279: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.795+0000 7fac0becf640 1 -- 172.21.15.31:0/224997309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040ffcf0 msgr2=0x7fac04100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.795+0000 7fac0becf640 1 --2- 172.21.15.31:0/224997309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040ffcf0 0x7fac04100170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009880 tx=0x7fabf402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.795+0000 7fac0becf640 1 -- 172.21.15.31:0/224997309 shutdown_connections 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.795+0000 7fac0becf640 1 --2- 172.21.15.31:0/224997309 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040ffcf0 0x7fac04100170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.795+0000 7fac0becf640 1 --2- 172.21.15.31:0/224997309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac040feaf0 0x7fac040feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.795+0000 7fac0becf640 1 -- 172.21.15.31:0/224997309 >> 172.21.15.31:0/224997309 conn(0x7fac040fa280 msgr2=0x7fac040fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.796+0000 7fac0becf640 1 -- 172.21.15.31:0/224997309 shutdown_connections 2024-09-16T09:06:58.798 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.796+0000 7fac0becf640 1 -- 172.21.15.31:0/224997309 wait complete. 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.796+0000 7fac0becf640 1 Processor -- start 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.796+0000 7fac0becf640 1 -- start start 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac0becf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040feaf0 0x7fac04102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac0becf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac040ffcf0 0x7fac041010d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac0becf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04101610 con 0x7fac040feaf0 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac0becf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac04101780 con 0x7fac040ffcf0 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac09c6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040feaf0 0x7fac04102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac09c6b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040feaf0 0x7fac04102a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57476/0 (socket says 172.21.15.31:57476) 2024-09-16T09:06:58.799 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac09c6b640 1 -- 172.21.15.31:0/3444364550 learned_addr learned my addr 172.21.15.31:0/3444364550 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac09c6b640 1 -- 172.21.15.31:0/3444364550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac040ffcf0 msgr2=0x7fac041010d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac09c6b640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac040ffcf0 0x7fac041010d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.797+0000 7fac09c6b640 1 -- 172.21.15.31:0/3444364550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf4009560 con 0x7fac040feaf0 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.798+0000 7fac09c6b640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040feaf0 0x7fac04102a80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac0000e940 tx=0x7fac0000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.798+0000 7fabfaffd640 1 -- 172.21.15.31:0/3444364550 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac000096e0 con 0x7fac040feaf0 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.798+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac04101a60 con 0x7fac040feaf0 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.798+0000 7fabfaffd640 1 -- 172.21.15.31:0/3444364550 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac000044f0 con 0x7fac040feaf0 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.798+0000 7fabfaffd640 1 -- 172.21.15.31:0/3444364550 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac00010430 con 0x7fac040feaf0 2024-09-16T09:06:58.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.798+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0419d730 con 0x7fac040feaf0 2024-09-16T09:06:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.800+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac040feef0 con 0x7fac040feaf0 2024-09-16T09:06:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.800+0000 7fabfaffd640 1 -- 172.21.15.31:0/3444364550 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fac0000cc90 con 0x7fac040feaf0 2024-09-16T09:06:58.801 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.801+0000 7fabfaffd640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabf007af30 0x7fabf007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.801+0000 7fabfaffd640 1 -- 172.21.15.31:0/3444364550 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fac00014070 con 0x7fac040feaf0 2024-09-16T09:06:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.801+0000 7fac0946a640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabf007af30 0x7fabf007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:58.802 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.801+0000 7fac0946a640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabf007af30 0x7fabf007d3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fabf402a9c0 tx=0x7fabf4000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:58.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:58.809+0000 7fabfaffd640 1 -- 172.21.15.31:0/3444364550 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fac000615e0 con 0x7fac040feaf0 2024-09-16T09:06:59.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:58 smithi097 ceph-mon[79892]: pgmap v279: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:06:59.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.070+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac04068430 con 0x7fabf007af30 2024-09-16T09:06:59.072 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.071+0000 7fabfaffd640 1 -- 172.21.15.31:0/3444364550 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fac04068430 con 0x7fabf007af30 2024-09-16T09:06:59.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabf007af30 msgr2=0x7fabf007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabf007af30 0x7fabf007d3f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fabf402a9c0 tx=0x7fabf4000bc0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040feaf0 msgr2=0x7fac04102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040feaf0 0x7fac04102a80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fac0000e940 tx=0x7fac0000ee00 comp rx=0 tx=0).stop 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 shutdown_connections 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fabf007af30 0x7fabf007d3f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fac040ffcf0 0x7fac041010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 --2- 172.21.15.31:0/3444364550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fac040feaf0 0x7fac04102a80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 >> 172.21.15.31:0/3444364550 conn(0x7fac040fa280 msgr2=0x7fac04067d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 shutdown_connections 2024-09-16T09:06:59.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.074+0000 7fac0becf640 1 -- 172.21.15.31:0/3444364550 wait complete. 2024-09-16T09:06:59.085 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:06:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.176+0000 7fb8beed7640 1 -- 172.21.15.31:0/1860794478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b8100d80 msgr2=0x7fb8b8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.176+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1860794478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b8100d80 0x7fb8b8101200 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009880 tx=0x7fb8a402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.176+0000 7fb8beed7640 1 -- 172.21.15.31:0/1860794478 shutdown_connections 2024-09-16T09:06:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.176+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1860794478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b8100d80 0x7fb8b8101200 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.176+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1860794478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b80ffb80 0x7fb8b80fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.176+0000 7fb8beed7640 1 -- 172.21.15.31:0/1860794478 >> 172.21.15.31:0/1860794478 conn(0x7fb8b80fb310 msgr2=0x7fb8b80fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.176+0000 7fb8beed7640 1 -- 172.21.15.31:0/1860794478 shutdown_connections 2024-09-16T09:06:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.177+0000 7fb8beed7640 1 -- 172.21.15.31:0/1860794478 wait complete. 2024-09-16T09:06:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.177+0000 7fb8beed7640 1 Processor -- start 2024-09-16T09:06:59.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.177+0000 7fb8beed7640 1 -- start start 2024-09-16T09:06:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.177+0000 7fb8beed7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b80ffb80 0x7fb8b8074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.177+0000 7fb8beed7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b8100d80 0x7fb8b80728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.177+0000 7fb8beed7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8072de0 con 0x7fb8b80ffb80 2024-09-16T09:06:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.177+0000 7fb8beed7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8072f50 con 0x7fb8b8100d80 2024-09-16T09:06:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8bcc73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b80ffb80 0x7fb8b8074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b8100d80 0x7fb8b80728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.180 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8bcc73640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b80ffb80 0x7fb8b8074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57500/0 (socket says 172.21.15.31:57500) 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8bcc73640 1 -- 172.21.15.31:0/1763326114 learned_addr learned my addr 172.21.15.31:0/1763326114 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b8100d80 0x7fb8b80728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.31:56680/0 (socket says 172.21.15.31:56680) 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8b7fff640 1 -- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b80ffb80 msgr2=0x7fb8b8074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8b7fff640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b80ffb80 0x7fb8b8074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8b7fff640 1 -- 172.21.15.31:0/1763326114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a4009560 con 0x7fb8b8100d80 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8bcc73640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b80ffb80 0x7fb8b8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.178+0000 7fb8b7fff640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b8100d80 0x7fb8b80728a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8a402a9c0 tx=0x7fb8a40054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.179+0000 7fb8b5ffb640 1 -- 172.21.15.31:0/1763326114 <== mon.1 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a4004aa0 con 0x7fb8b8100d80 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.179+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b80731d0 con 0x7fb8b8100d80 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.179+0000 7fb8b5ffb640 1 -- 172.21.15.31:0/1763326114 <== mon.1 v2:172.21.15.97:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8a4004c00 con 0x7fb8b8100d80 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.179+0000 7fb8b5ffb640 1 -- 172.21.15.31:0/1763326114 <== mon.1 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8a40039c0 con 0x7fb8b8100d80 2024-09-16T09:06:59.181 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.179+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b80736c0 con 0x7fb8b8100d80 2024-09-16T09:06:59.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.180+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b8108770 con 0x7fb8b8100d80 2024-09-16T09:06:59.182 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.181+0000 7fb8b5ffb640 1 -- 172.21.15.31:0/1763326114 <== mon.1 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fb8a4031b90 con 0x7fb8b8100d80 2024-09-16T09:06:59.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.182+0000 7fb8b5ffb640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb8a807af30 0x7fb8a807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.182+0000 7fb8b5ffb640 1 -- 172.21.15.31:0/1763326114 <== mon.1 v2:172.21.15.97:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fb8a40bf7e0 con 0x7fb8b8100d80 2024-09-16T09:06:59.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.182+0000 7fb8bcc73640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb8a807af30 0x7fb8a807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.183 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.182+0000 7fb8bcc73640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb8a807af30 0x7fb8a807d3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005fd0 tx=0x7fb8ac00b4a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:59.186 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.185+0000 7fb8b5ffb640 1 -- 172.21.15.31:0/1763326114 <== mon.1 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fb8a4082980 con 0x7fb8b8100d80 2024-09-16T09:06:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.424+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8b8105530 con 0x7fb8a807af30 2024-09-16T09:06:59.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.426+0000 7fb8b5ffb640 1 -- 172.21.15.31:0/1763326114 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7fb8b8105530 con 0x7fb8a807af30 2024-09-16T09:06:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb8a807af30 msgr2=0x7fb8a807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb8a807af30 0x7fb8a807d3f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac005fd0 tx=0x7fb8ac00b4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b8100d80 msgr2=0x7fb8b80728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b8100d80 0x7fb8b80728a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8a402a9c0 tx=0x7fb8a40054f0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 shutdown_connections 2024-09-16T09:06:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fb8a807af30 0x7fb8a807d3f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8b8100d80 0x7fb8b80728a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 --2- 172.21.15.31:0/1763326114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb8b80ffb80 0x7fb8b8074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 >> 172.21.15.31:0/1763326114 conn(0x7fb8b80fb310 msgr2=0x7fb8b80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 shutdown_connections 2024-09-16T09:06:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.428+0000 7fb8beed7640 1 -- 172.21.15.31:0/1763326114 wait complete. 2024-09-16T09:06:59.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 -- 172.21.15.31:0/2527012416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40feaa0 msgr2=0x7ff9e40feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/2527012416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40feaa0 0x7ff9e40feea0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0009880 tx=0x7ff9e002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 -- 172.21.15.31:0/2527012416 shutdown_connections 2024-09-16T09:06:59.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/2527012416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e40ffca0 0x7ff9e4100120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/2527012416 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40feaa0 0x7ff9e40feea0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 -- 172.21.15.31:0/2527012416 >> 172.21.15.31:0/2527012416 conn(0x7ff9e40fa230 msgr2=0x7ff9e40fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:59.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 -- 172.21.15.31:0/2527012416 shutdown_connections 2024-09-16T09:06:59.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.535+0000 7ff9ebde0640 1 -- 172.21.15.31:0/2527012416 wait complete. 2024-09-16T09:06:59.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9ebde0640 1 Processor -- start 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9ebde0640 1 -- start start 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9ebde0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e40feaa0 0x7ff9e4102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9ebde0640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40ffca0 0x7ff9e4101080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9ebde0640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e41015c0 con 0x7ff9e40ffca0 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9ebde0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e4101730 con 0x7ff9e40feaa0 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9e937b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40ffca0 0x7ff9e4101080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9e937b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40ffca0 0x7ff9e4101080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57518/0 (socket says 172.21.15.31:57518) 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9e937b640 1 -- 172.21.15.31:0/3481794125 learned_addr learned my addr 172.21.15.31:0/3481794125 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.536+0000 7ff9e9b7c640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e40feaa0 0x7ff9e4102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9e937b640 1 -- 172.21.15.31:0/3481794125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e40feaa0 msgr2=0x7ff9e4102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9e937b640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e40feaa0 0x7ff9e4102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9e937b640 1 -- 172.21.15.31:0/3481794125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009560 con 0x7ff9e40ffca0 2024-09-16T09:06:59.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9e937b640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40ffca0 0x7ff9e4101080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400e970 tx=0x7ff9d400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:59.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9daffd640 1 -- 172.21.15.31:0/3481794125 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9d400cc90 con 0x7ff9e40ffca0 2024-09-16T09:06:59.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9daffd640 1 -- 172.21.15.31:0/3481794125 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9d400cdf0 con 0x7ff9e40ffca0 2024-09-16T09:06:59.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e4101a10 con 0x7ff9e40ffca0 2024-09-16T09:06:59.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9daffd640 1 -- 172.21.15.31:0/3481794125 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9d4018810 con 0x7ff9e40ffca0 2024-09-16T09:06:59.540 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.537+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e419d700 con 0x7ff9e40ffca0 2024-09-16T09:06:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.538+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e4107690 con 0x7ff9e40ffca0 2024-09-16T09:06:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.539+0000 7ff9daffd640 1 -- 172.21.15.31:0/3481794125 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7ff9d4010c10 con 0x7ff9e40ffca0 2024-09-16T09:06:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.540+0000 7ff9daffd640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff9d007af30 0x7ff9d007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.541 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.540+0000 7ff9e9b7c640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff9d007af30 0x7ff9d007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.540+0000 7ff9daffd640 1 -- 172.21.15.31:0/3481794125 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7ff9d4014070 con 0x7ff9e40ffca0 2024-09-16T09:06:59.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.541+0000 7ff9e9b7c640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff9d007af30 0x7ff9d007d3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0005940 tx=0x7ff9e00058d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:59.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.545+0000 7ff9daffd640 1 -- 172.21.15.31:0/3481794125 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ff9d4064dd0 con 0x7ff9e40ffca0 2024-09-16T09:06:59.823 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.822+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9e4068bf0 con 0x7ff9d007af30 2024-09-16T09:06:59.823 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:06:59 smithi031 ceph-mon[91884]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:06:59.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.834+0000 7ff9daffd640 1 -- 172.21.15.31:0/3481794125 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7ff9e4068bf0 con 0x7ff9d007af30 2024-09-16T09:06:59.835 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:06:59.835 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (16s) 11s ago 18m 14.5M - 0.27.0 11f11916f8cd b523d960fed9 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (83s) 11s ago 18m 8488k - 19.3.0-4917-g8293d73f 425e316ba8eb 119433b14707 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (78s) 55s ago 17m 8124k - 19.3.0-4917-g8293d73f 425e316ba8eb 3f8a92f1222e 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (6m) 11s ago 18m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (6m) 55s ago 17m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (16m) 11s ago 18m 85.1M - 9.4.7 954c08fa6188 250534aca2a5 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (2m) 11s ago 12m 85.0M - 19.3.0-4917-g8293d73f 425e316ba8eb 7e4ca0c031e9 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (2m) 11s ago 12m 17.7M - 19.3.0-4917-g8293d73f 425e316ba8eb f54e4116709a 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (114s) 55s ago 12m 16.5M - 19.3.0-4917-g8293d73f 425e316ba8eb 880891e445c6 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (95s) 55s ago 12m 16.2M - 19.3.0-4917-g8293d73f 425e316ba8eb 08131d73c31f 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (9m) 11s ago 19m 525M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (7m) 55s ago 17m 456M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (7m) 11s ago 19m 59.0M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:06:59.836 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (7m) 55s ago 17m 45.4M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (63s) 11s ago 18m 7852k - 1.7.0 72c9c2088986 7588c21d20a2 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (57s) 55s ago 17m 3502k - 1.7.0 72c9c2088986 818f167b5f3e 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 11s ago 15m 117M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (5m) 11s ago 15m 115M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 11s ago 15m 132M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (4m) 55s ago 14m 136M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (3m) 55s ago 14m 98.9M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (3m) 55s ago 13m 109M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:06:59.837 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (35s) 11s ago 17m 44.0M - 2.51.0 1d3b7f56885b 0383f88326e0 2024-09-16T09:06:59.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff9d007af30 msgr2=0x7ff9d007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff9d007af30 0x7ff9d007d3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0005940 tx=0x7ff9e00058d0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40ffca0 msgr2=0x7ff9e4101080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40ffca0 0x7ff9e4101080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff9d400e970 tx=0x7ff9d400ee30 comp rx=0 tx=0).stop 2024-09-16T09:06:59.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 shutdown_connections 2024-09-16T09:06:59.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff9d007af30 0x7ff9d007d3f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.838 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9e40ffca0 0x7ff9e4101080 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 --2- 172.21.15.31:0/3481794125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff9e40feaa0 0x7ff9e4102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 >> 172.21.15.31:0/3481794125 conn(0x7ff9e40fa230 msgr2=0x7ff9e40684e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:59.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 shutdown_connections 2024-09-16T09:06:59.839 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.837+0000 7ff9ebde0640 1 -- 172.21.15.31:0/3481794125 wait complete. 2024-09-16T09:06:59.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 -- 172.21.15.31:0/1648244162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760103030 msgr2=0x7f2760105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 --2- 172.21.15.31:0/1648244162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760103030 0x7f2760105420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2754009880 tx=0x7f275402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 -- 172.21.15.31:0/1648244162 shutdown_connections 2024-09-16T09:06:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 --2- 172.21.15.31:0/1648244162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760103030 0x7f2760105420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 --2- 172.21.15.31:0/1648244162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2760100700 0x7f2760102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 -- 172.21.15.31:0/1648244162 >> 172.21.15.31:0/1648244162 conn(0x7f27600fa280 msgr2=0x7f27600fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:06:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 -- 172.21.15.31:0/1648244162 shutdown_connections 2024-09-16T09:06:59.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.939+0000 7f2766455640 1 -- 172.21.15.31:0/1648244162 wait complete. 2024-09-16T09:06:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.940+0000 7f2766455640 1 Processor -- start 2024-09-16T09:06:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.940+0000 7f2766455640 1 -- start start 2024-09-16T09:06:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.940+0000 7f2766455640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760100700 0x7f27601956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.942 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.940+0000 7f2766455640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2760103030 0x7f2760195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.940+0000 7f2766455640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27601961e0 con 0x7f2760100700 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.940+0000 7f2766455640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2760196350 con 0x7f2760103030 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.940+0000 7f275ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760100700 0x7f27601956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275ffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760100700 0x7f27601956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57544/0 (socket says 172.21.15.31:57544) 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275ffff640 1 -- 172.21.15.31:0/3806273029 learned_addr learned my addr 172.21.15.31:0/3806273029 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275f7fe640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2760103030 0x7f2760195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275ffff640 1 -- 172.21.15.31:0/3806273029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2760103030 msgr2=0x7f2760195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275ffff640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2760103030 0x7f2760195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275ffff640 1 -- 172.21.15.31:0/3806273029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2754009560 con 0x7f2760100700 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275ffff640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760100700 0x7f27601956d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f275000e9a0 tx=0x7f275000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275d7fa640 1 -- 172.21.15.31:0/3806273029 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f275000cd50 con 0x7f2760100700 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276019adf0 con 0x7f2760100700 2024-09-16T09:06:59.943 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275d7fa640 1 -- 172.21.15.31:0/3806273029 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f275000ceb0 con 0x7f2760100700 2024-09-16T09:06:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.941+0000 7f275d7fa640 1 -- 172.21.15.31:0/3806273029 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2750010660 con 0x7f2760100700 2024-09-16T09:06:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.942+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276019b2c0 con 0x7f2760100700 2024-09-16T09:06:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.943+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2760100ae0 con 0x7f2760100700 2024-09-16T09:06:59.944 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.943+0000 7f275d7fa640 1 -- 172.21.15.31:0/3806273029 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f27500048a0 con 0x7f2760100700 2024-09-16T09:06:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.944+0000 7f275d7fa640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f274807af30 0x7f274807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:06:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.944+0000 7f275d7fa640 1 -- 172.21.15.31:0/3806273029 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f2750014070 con 0x7f2760100700 2024-09-16T09:06:59.945 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.944+0000 7f275f7fe640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f274807af30 0x7f274807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:06:59.946 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.945+0000 7f275f7fe640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f274807af30 0x7f274807d3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2754009230 tx=0x7f27540314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:06:59.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:06:59.950+0000 7f275d7fa640 1 -- 172.21.15.31:0/3806273029 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f2750064d50 con 0x7f2760100700 2024-09-16T09:07:00.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:06:59 smithi097 ceph-mon[79892]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.291+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f276019b900 con 0x7f2760100700 2024-09-16T09:07:00.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.291+0000 7f275d7fa640 1 -- 172.21.15.31:0/3806273029 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f27500646f0 con 0x7f2760100700 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:00.293 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:07:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:07:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:07:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 14 2024-09-16T09:07:00.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:07:00.294 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f274807af30 msgr2=0x7f274807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f274807af30 0x7f274807d3f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2754009230 tx=0x7f27540314e0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760100700 msgr2=0x7f27601956d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760100700 0x7f27601956d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f275000e9a0 tx=0x7f275000eda0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 shutdown_connections 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f274807af30 0x7f274807d3f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2760103030 0x7f2760195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 --2- 172.21.15.31:0/3806273029 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2760100700 0x7f27601956d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 >> 172.21.15.31:0/3806273029 conn(0x7f27600fa280 msgr2=0x7f27600fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 shutdown_connections 2024-09-16T09:07:00.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.294+0000 7f2766455640 1 -- 172.21.15.31:0/3806273029 wait complete. 2024-09-16T09:07:00.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 -- 172.21.15.31:0/3298106153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b740ffb80 msgr2=0x7f3b740fff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3298106153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b740ffb80 0x7f3b740fff80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009880 tx=0x7f3b6402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.390 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 -- 172.21.15.31:0/3298106153 shutdown_connections 2024-09-16T09:07:00.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3298106153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b74100d80 0x7f3b74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3298106153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b740ffb80 0x7f3b740fff80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 -- 172.21.15.31:0/3298106153 >> 172.21.15.31:0/3298106153 conn(0x7f3b740fb310 msgr2=0x7f3b740fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:00.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 -- 172.21.15.31:0/3298106153 shutdown_connections 2024-09-16T09:07:00.391 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.389+0000 7f3b791b7640 1 -- 172.21.15.31:0/3298106153 wait complete. 2024-09-16T09:07:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b791b7640 1 Processor -- start 2024-09-16T09:07:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b791b7640 1 -- start start 2024-09-16T09:07:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b791b7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b740ffb80 0x7f3b74074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b791b7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b74100d80 0x7f3b740728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:00.392 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b791b7640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74072de0 con 0x7f3b74100d80 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b791b7640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b74072f50 con 0x7f3b740ffb80 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b74100d80 0x7f3b740728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b7259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b74100d80 0x7f3b740728a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57560/0 (socket says 172.21.15.31:57560) 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.390+0000 7f3b7259c640 1 -- 172.21.15.31:0/3939297938 learned_addr learned my addr 172.21.15.31:0/3939297938 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b72d9d640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b740ffb80 0x7f3b74074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b7259c640 1 -- 172.21.15.31:0/3939297938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b740ffb80 msgr2=0x7f3b74074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b7259c640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b740ffb80 0x7f3b74074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b7259c640 1 -- 172.21.15.31:0/3939297938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b64009560 con 0x7f3b74100d80 2024-09-16T09:07:00.393 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b7259c640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b74100d80 0x7f3b740728a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800e970 tx=0x7f3b6800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b5bfff640 1 -- 172.21.15.31:0/3939297938 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b6800cc90 con 0x7f3b74100d80 2024-09-16T09:07:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b74073230 con 0x7f3b74100d80 2024-09-16T09:07:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b5bfff640 1 -- 172.21.15.31:0/3939297938 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b6800cdf0 con 0x7f3b74100d80 2024-09-16T09:07:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b5bfff640 1 -- 172.21.15.31:0/3939297938 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b68010430 con 0x7f3b74100d80 2024-09-16T09:07:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.391+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7419d650 con 0x7f3b74100d80 2024-09-16T09:07:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.392+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b740fff80 con 0x7f3b74100d80 2024-09-16T09:07:00.394 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.393+0000 7f3b5bfff640 1 -- 172.21.15.31:0/3939297938 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f3b680046c0 con 0x7f3b74100d80 2024-09-16T09:07:00.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.394+0000 7f3b5bfff640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3b5c07af30 0x7f3b5c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:00.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.394+0000 7f3b5bfff640 1 -- 172.21.15.31:0/3939297938 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f3b68014070 con 0x7f3b74100d80 2024-09-16T09:07:00.395 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.394+0000 7f3b72d9d640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3b5c07af30 0x7f3b5c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:00.396 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.395+0000 7f3b72d9d640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3b5c07af30 0x7f3b5c07d3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009230 tx=0x7f3b64005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:00.400 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.399+0000 7f3b5bfff640 1 -- 172.21.15.31:0/3939297938 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f3b680616f0 con 0x7f3b74100d80 2024-09-16T09:07:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.719+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3b74108790 con 0x7f3b74100d80 2024-09-16T09:07:00.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.719+0000 7f3b5bfff640 1 -- 172.21.15.31:0/3939297938 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1987 (secure 0 0 0) 0x7f3b68064810 con 0x7f3b74100d80 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:e38 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T09:05:32:642886+0000 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:07:00.721 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 38 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T09:05:32.642884+0000 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 99 2024-09-16T09:07:00.722 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34430,1=34434} 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 34430 members: 34430,34434 2024-09-16T09:07:00.723 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{0:34430} state up:active seq 17 join_fscid=1 addr [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:07:00.724 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{1:34434} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:07:00.724 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:07:00.724 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:07:00.724 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:07:00.724 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:07:00.724 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6824/257407185,v1:172.21.15.97:6825/257407185] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:07:00.724 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:44375} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6826/2995504601,v1:172.21.15.97:6827/2995504601] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3b5c07af30 msgr2=0x7f3b5c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3b5c07af30 0x7f3b5c07d3f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3b64009230 tx=0x7f3b64005d50 comp rx=0 tx=0).stop 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b74100d80 msgr2=0x7f3b740728a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b74100d80 0x7f3b740728a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3b6800e970 tx=0x7f3b6800ed70 comp rx=0 tx=0).stop 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 shutdown_connections 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f3b5c07af30 0x7f3b5c07d3f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b74100d80 0x7f3b740728a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 --2- 172.21.15.31:0/3939297938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b740ffb80 0x7f3b74074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 >> 172.21.15.31:0/3939297938 conn(0x7f3b740fb310 msgr2=0x7f3b74103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 shutdown_connections 2024-09-16T09:07:00.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.722+0000 7f3b791b7640 1 -- 172.21.15.31:0/3939297938 wait complete. 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 -- 172.21.15.31:0/1982680874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 msgr2=0x7ff2841029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 --2- 172.21.15.31:0/1982680874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2841029a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff2740098b0 tx=0x7ff27402a4d0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 -- 172.21.15.31:0/1982680874 shutdown_connections 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 --2- 172.21.15.31:0/1982680874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff284102ee0 0x7ff2841052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 --2- 172.21.15.31:0/1982680874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2841029a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 -- 172.21.15.31:0/1982680874 >> 172.21.15.31:0/1982680874 conn(0x7ff2840fa1f0 msgr2=0x7ff2840fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 -- 172.21.15.31:0/1982680874 shutdown_connections 2024-09-16T09:07:00.808 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.806+0000 7ff289a87640 1 -- 172.21.15.31:0/1982680874 wait complete. 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.807+0000 7ff289a87640 1 Processor -- start 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.807+0000 7ff289a87640 1 -- start start 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff289a87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2840ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff289a87640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff284102ee0 0x7ff2840fe560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff289a87640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840feaa0 con 0x7ff2841005b0 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff289a87640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840fec10 con 0x7ff284102ee0 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2840ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff2837fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2840ffec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57586/0 (socket says 172.21.15.31:57586) 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff2837fe640 1 -- 172.21.15.31:0/209356968 learned_addr learned my addr 172.21.15.31:0/209356968 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:00.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff282ffd640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff284102ee0 0x7ff2840fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff2837fe640 1 -- 172.21.15.31:0/209356968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff284102ee0 msgr2=0x7ff2840fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff2837fe640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff284102ee0 0x7ff2840fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.808+0000 7ff2837fe640 1 -- 172.21.15.31:0/209356968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff274009560 con 0x7ff2841005b0 2024-09-16T09:07:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.809+0000 7ff2837fe640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2840ffec0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff274005610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.809+0000 7ff280ff9640 1 -- 172.21.15.31:0/209356968 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff274004b20 con 0x7ff2841005b0 2024-09-16T09:07:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.809+0000 7ff280ff9640 1 -- 172.21.15.31:0/209356968 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff274004c80 con 0x7ff2841005b0 2024-09-16T09:07:00.811 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.809+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2840fee90 con 0x7ff2841005b0 2024-09-16T09:07:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.809+0000 7ff280ff9640 1 -- 172.21.15.31:0/209356968 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff274003bf0 con 0x7ff2841005b0 2024-09-16T09:07:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.809+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28419d6d0 con 0x7ff2841005b0 2024-09-16T09:07:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.811+0000 7ff280ff9640 1 -- 172.21.15.31:0/209356968 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7ff274031640 con 0x7ff2841005b0 2024-09-16T09:07:00.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.811+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff284100990 con 0x7ff2841005b0 2024-09-16T09:07:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.811+0000 7ff280ff9640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff26c07af30 0x7ff26c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.811+0000 7ff280ff9640 1 -- 172.21.15.31:0/209356968 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7ff2740bf240 con 0x7ff2841005b0 2024-09-16T09:07:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.811+0000 7ff282ffd640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff26c07af30 0x7ff26c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:00.813 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.812+0000 7ff282ffd640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff26c07af30 0x7ff26c07d3f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2840ffc40 tx=0x7ff27800a250 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:00.817 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:00.816+0000 7ff280ff9640 1 -- 172.21.15.31:0/209356968 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ff274082300 con 0x7ff2841005b0 2024-09-16T09:07:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:00 smithi097 ceph-mon[79892]: pgmap v280: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:00 smithi097 ceph-mon[79892]: from='client.44431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:00 smithi097 ceph-mon[79892]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:00 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3806273029' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.025+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff284103940 con 0x7ff26c07af30 2024-09-16T09:07:01.026 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:00 smithi031 ceph-mon[91884]: pgmap v280: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:01.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:00 smithi031 ceph-mon[91884]: from='client.44431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:01.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:00 smithi031 ceph-mon[91884]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:01.027 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:00 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3806273029' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.027+0000 7ff280ff9640 1 -- 172.21.15.31:0/209356968 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+460 (secure 0 0 0) 0x7ff284103940 con 0x7ff26c07af30 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": true, 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [ 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "osd", 2024-09-16T09:07:01.028 INFO:teuthology.orchestra.run.smithi031.stdout: "crash", 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr", 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph-exporter", 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mon", 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "mds" 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": "18/23 daemons upgraded", 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "Currently upgrading grafana daemons", 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:07:01.029 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:07:01.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff26c07af30 msgr2=0x7ff26c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:01.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff26c07af30 0x7ff26c07d3f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff2840ffc40 tx=0x7ff27800a250 comp rx=0 tx=0).stop 2024-09-16T09:07:01.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 msgr2=0x7ff2840ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:01.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2840ffec0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff274005610 comp rx=0 tx=0).stop 2024-09-16T09:07:01.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 shutdown_connections 2024-09-16T09:07:01.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff26c07af30 0x7ff26c07d3f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.030 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff284102ee0 0x7ff2840fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 --2- 172.21.15.31:0/209356968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2841005b0 0x7ff2840ffec0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 >> 172.21.15.31:0/209356968 conn(0x7ff2840fa1f0 msgr2=0x7ff2840fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:01.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 shutdown_connections 2024-09-16T09:07:01.031 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.029+0000 7ff289a87640 1 -- 172.21.15.31:0/209356968 wait complete. 2024-09-16T09:07:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 -- 172.21.15.31:0/1097983134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0ffca0 msgr2=0x7f234c100120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 --2- 172.21.15.31:0/1097983134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0ffca0 0x7f234c100120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f233c009880 tx=0x7f233c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 -- 172.21.15.31:0/1097983134 shutdown_connections 2024-09-16T09:07:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 --2- 172.21.15.31:0/1097983134 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0ffca0 0x7f234c100120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.114 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 --2- 172.21.15.31:0/1097983134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234c0feaa0 0x7f234c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 -- 172.21.15.31:0/1097983134 >> 172.21.15.31:0/1097983134 conn(0x7f234c0fa250 msgr2=0x7f234c0fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 -- 172.21.15.31:0/1097983134 shutdown_connections 2024-09-16T09:07:01.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.113+0000 7f2353e56640 1 -- 172.21.15.31:0/1097983134 wait complete. 2024-09-16T09:07:01.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2353e56640 1 Processor -- start 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2353e56640 1 -- start start 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2353e56640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0feaa0 0x7f234c195600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2353e56640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234c0ffca0 0x7f234c195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2353e56640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c196110 con 0x7f234c0feaa0 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2353e56640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c196280 con 0x7f234c0ffca0 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2351bf2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0feaa0 0x7f234c195600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2351bf2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0feaa0 0x7f234c195600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57604/0 (socket says 172.21.15.31:57604) 2024-09-16T09:07:01.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2351bf2640 1 -- 172.21.15.31:0/471526645 learned_addr learned my addr 172.21.15.31:0/471526645 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f23513f1640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234c0ffca0 0x7f234c195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2351bf2640 1 -- 172.21.15.31:0/471526645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234c0ffca0 msgr2=0x7f234c195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2351bf2640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234c0ffca0 0x7f234c195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.114+0000 7f2351bf2640 1 -- 172.21.15.31:0/471526645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c009560 con 0x7f234c0feaa0 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.115+0000 7f2351bf2640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0feaa0 0x7f234c195600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f234800d720 tx=0x7f234800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.115+0000 7f2342ffd640 1 -- 172.21.15.31:0/471526645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23480041f0 con 0x7f234c0feaa0 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.115+0000 7f2342ffd640 1 -- 172.21.15.31:0/471526645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2348004350 con 0x7f234c0feaa0 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.115+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c19ad20 con 0x7f234c0feaa0 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.115+0000 7f2342ffd640 1 -- 172.21.15.31:0/471526645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2348003df0 con 0x7f234c0feaa0 2024-09-16T09:07:01.118 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.115+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c19b260 con 0x7f234c0feaa0 2024-09-16T09:07:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.116+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c107690 con 0x7f234c0feaa0 2024-09-16T09:07:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.117+0000 7f2342ffd640 1 -- 172.21.15.31:0/471526645 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f2348010460 con 0x7f234c0feaa0 2024-09-16T09:07:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.117+0000 7f2342ffd640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f233807aee0 0x7f233807d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.117+0000 7f2342ffd640 1 -- 172.21.15.31:0/471526645 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f234809dac0 con 0x7f234c0feaa0 2024-09-16T09:07:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.117+0000 7f23513f1640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f233807aee0 0x7f233807d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:01.119 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.118+0000 7f23513f1640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f233807aee0 0x7f233807d3a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f233c000c00 tx=0x7f233c0314e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:01.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.122+0000 7f2342ffd640 1 -- 172.21.15.31:0/471526645 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f2348060b30 con 0x7f234c0feaa0 2024-09-16T09:07:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.380+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f234c19b4a0 con 0x7f234c0feaa0 2024-09-16T09:07:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.380+0000 7f2342ffd640 1 -- 172.21.15.31:0/471526645 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f2348063c50 con 0x7f234c0feaa0 2024-09-16T09:07:01.381 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.382+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f233807aee0 msgr2=0x7f233807d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.382+0000 7f2353e56640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f233807aee0 0x7f233807d3a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f233c000c00 tx=0x7f233c0314e0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.382+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0feaa0 msgr2=0x7f234c195600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.382+0000 7f2353e56640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0feaa0 0x7f234c195600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f234800d720 tx=0x7f234800dbe0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.383+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 shutdown_connections 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.383+0000 7f2353e56640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f233807aee0 0x7f233807d3a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.383+0000 7f2353e56640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f234c0ffca0 0x7f234c195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.383+0000 7f2353e56640 1 --2- 172.21.15.31:0/471526645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f234c0feaa0 0x7f234c195600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.383+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 >> 172.21.15.31:0/471526645 conn(0x7f234c0fa250 msgr2=0x7f234c0684c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.383+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 shutdown_connections 2024-09-16T09:07:01.384 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:01.383+0000 7f2353e56640 1 -- 172.21.15.31:0/471526645 wait complete. 2024-09-16T09:07:01.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:01 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3939297938' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:07:01.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:01 smithi031 ceph-mon[91884]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:01.967 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:01 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/471526645' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:07:02.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:01 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3939297938' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:07:02.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:01 smithi097 ceph-mon[79892]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:02.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:01 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/471526645' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:07:02.935 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:02 smithi031 ceph-mon[91884]: pgmap v281: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:02.939 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:02 smithi097 ceph-mon[79892]: pgmap v281: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:04.674 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:04 smithi031 ceph-mon[91884]: pgmap v282: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:05.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:04 smithi097 ceph-mon[79892]: pgmap v282: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:07 smithi031 ceph-mon[91884]: pgmap v283: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:07 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:07.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:07 smithi097 ceph-mon[79892]: pgmap v283: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:07.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:07.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:07.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:07 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:09.335 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:09 smithi031 ceph-mon[91884]: pgmap v284: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:09.335 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:09 smithi097 ceph-mon[79892]: pgmap v284: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:09.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:11.314 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:11 smithi097 ceph-mon[79892]: pgmap v285: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:11.314 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:11.314 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:11 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:11 smithi031 ceph-mon[91884]: pgmap v285: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:11 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:12.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:12.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:12 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:12.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:12 smithi031 ceph-mon[91884]: pgmap v286: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:12 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:12.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:12 smithi097 ceph-mon[79892]: pgmap v286: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:14.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:14 smithi097 ceph-mon[79892]: pgmap v287: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:14.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:14 smithi031 ceph-mon[91884]: pgmap v287: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:15.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.763 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.764 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: Upgrade: Finalizing container_image settings 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-16T09:07:15.765 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-16T09:07:15.766 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: Upgrade: Complete! 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T09:07:15.767 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:15.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:07:15.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:07:15.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:15.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:07:15.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:07:15.768 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.874 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: Upgrade: Finalizing container_image settings 2024-09-16T09:07:15.875 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mgr"} : dispatch 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.crash"} : dispatch 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "osd"} : dispatch 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mds"} : dispatch 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rgw"} : dispatch 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"} : dispatch 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-16T09:07:15.876 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.iscsi"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nfs"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config rm", "name": "container_image", "who": "mon"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: Upgrade: Complete! 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:07:15.878 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:17.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:16 smithi031 ceph-mon[91884]: pgmap v288: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:17.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:16 smithi097 ceph-mon[79892]: pgmap v288: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:19.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:18 smithi031 ceph-mon[91884]: pgmap v289: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:19.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:18 smithi097 ceph-mon[79892]: pgmap v289: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:20.264 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:19 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:20.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:19 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:07:21.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:20 smithi097 ceph-mon[79892]: pgmap v290: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:21.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:20 smithi031 ceph-mon[91884]: pgmap v290: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:23.242 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:22 smithi097 ceph-mon[79892]: pgmap v291: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:23.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:22 smithi031 ceph-mon[91884]: pgmap v291: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:24.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:23 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:24.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:23 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:25.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:24 smithi097 ceph-mon[79892]: pgmap v292: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:25.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:24 smithi031 ceph-mon[91884]: pgmap v292: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:27.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:26 smithi031 ceph-mon[91884]: pgmap v293: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:27.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:26 smithi097 ceph-mon[79892]: pgmap v293: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:29.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:28 smithi097 ceph-mon[79892]: pgmap v294: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:29.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:28 smithi031 ceph-mon[91884]: pgmap v294: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:31.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:30 smithi097 ceph-mon[79892]: pgmap v295: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:31.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:30 smithi031 ceph-mon[91884]: pgmap v295: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 -- 172.21.15.31:0/935196824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088100630 msgr2=0x7f5088102a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 --2- 172.21.15.31:0/935196824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088100630 0x7f5088102a20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f50780098b0 tx=0x7f507802a4d0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 -- 172.21.15.31:0/935196824 shutdown_connections 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 --2- 172.21.15.31:0/935196824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5088102f60 0x7f5088105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 --2- 172.21.15.31:0/935196824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088100630 0x7f5088102a20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 -- 172.21.15.31:0/935196824 >> 172.21.15.31:0/935196824 conn(0x7f50880fa250 msgr2=0x7f50880fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 -- 172.21.15.31:0/935196824 shutdown_connections 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.481+0000 7f508cf68640 1 -- 172.21.15.31:0/935196824 wait complete. 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.482+0000 7f508cf68640 1 Processor -- start 2024-09-16T09:07:31.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.482+0000 7f508cf68640 1 -- start start 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.482+0000 7f508cf68640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5088100630 0x7f50881978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.482+0000 7f508cf68640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088102f60 0x7f5088197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.482+0000 7f508cf68640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50881983c0 con 0x7f5088102f60 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.482+0000 7f508cf68640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5088198530 con 0x7f5088100630 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.482+0000 7f5085d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088102f60 0x7f5088197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f5085d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088102f60 0x7f5088197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54862/0 (socket says 172.21.15.31:54862) 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f5085d9b640 1 -- 172.21.15.31:0/435151580 learned_addr learned my addr 172.21.15.31:0/435151580 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f508659c640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5088100630 0x7f50881978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f5085d9b640 1 -- 172.21.15.31:0/435151580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5088100630 msgr2=0x7f50881978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f5085d9b640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5088100630 0x7f50881978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f5085d9b640 1 -- 172.21.15.31:0/435151580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078009560 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f5085d9b640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088102f60 0x7f5088197df0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f507c00d6f0 tx=0x7f507c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f50777fe640 1 -- 172.21.15.31:0/435151580 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f507c0051f0 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f50777fe640 1 -- 172.21.15.31:0/435151580 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f507c009ce0 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508819cf80 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f50777fe640 1 -- 172.21.15.31:0/435151580 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f507c004e80 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.483+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508819d520 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.485+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5088100a10 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.485+0000 7f50777fe640 1 -- 172.21.15.31:0/435151580 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f507c0046c0 con 0x7f5088102f60 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.486+0000 7f50777fe640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f507007aee0 0x7f507007d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:31.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.486+0000 7f50777fe640 1 -- 172.21.15.31:0/435151580 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f507c09db00 con 0x7f5088102f60 2024-09-16T09:07:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.486+0000 7f508659c640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f507007aee0 0x7f507007d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:31.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.486+0000 7f508659c640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f507007aee0 0x7f507007d3a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5078006010 tx=0x7f5078031500 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:31.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.491+0000 7f50777fe640 1 -- 172.21.15.31:0/435151580 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f507c060b70 con 0x7f5088102f60 2024-09-16T09:07:31.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.695+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f508802c5a0 con 0x7f507007aee0 2024-09-16T09:07:31.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.696+0000 7f50777fe640 1 -- 172.21.15.31:0/435151580 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f508802c5a0 con 0x7f507007aee0 2024-09-16T09:07:31.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f507007aee0 msgr2=0x7f507007d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f507007aee0 0x7f507007d3a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5078006010 tx=0x7f5078031500 comp rx=0 tx=0).stop 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088102f60 msgr2=0x7f5088197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088102f60 0x7f5088197df0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f507c00d6f0 tx=0x7f507c00dbb0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 shutdown_connections 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f507007aee0 0x7f507007d3a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5088102f60 0x7f5088197df0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 --2- 172.21.15.31:0/435151580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5088100630 0x7f50881978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.698+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 >> 172.21.15.31:0/435151580 conn(0x7f50880fa250 msgr2=0x7f50880fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.699+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 shutdown_connections 2024-09-16T09:07:31.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:31.699+0000 7f508cf68640 1 -- 172.21.15.31:0/435151580 wait complete. 2024-09-16T09:07:33.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:32 smithi097 ceph-mon[79892]: pgmap v296: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:33.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:32 smithi097 ceph-mon[79892]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:33.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:32 smithi031 ceph-mon[91884]: pgmap v296: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:33.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:32 smithi031 ceph-mon[91884]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:33.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch ps' 2024-09-16T09:07:33.693 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:07:35.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:34 smithi031 ceph-mon[91884]: pgmap v297: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:35.265 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:34 smithi097 ceph-mon[79892]: pgmap v297: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.337+0000 7f6680f87640 1 -- 172.21.15.31:0/2486307924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c105be0 msgr2=0x7f667c105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 --2- 172.21.15.31:0/2486307924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c105be0 0x7f667c105fc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f666c009880 tx=0x7f666c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.356 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 -- 172.21.15.31:0/2486307924 shutdown_connections 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 --2- 172.21.15.31:0/2486307924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f667c0ffb30 0x7f667c0fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 --2- 172.21.15.31:0/2486307924 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c105be0 0x7f667c105fc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 -- 172.21.15.31:0/2486307924 >> 172.21.15.31:0/2486307924 conn(0x7f667c0fb810 msgr2=0x7f667c0fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 -- 172.21.15.31:0/2486307924 shutdown_connections 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 -- 172.21.15.31:0/2486307924 wait complete. 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 Processor -- start 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.338+0000 7f6680f87640 1 -- start start 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f6680f87640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c0ffb30 0x7f667c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f6680f87640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f667c105be0 0x7f667c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:35.357 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f6680f87640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c19c330 con 0x7f667c0ffb30 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f6680f87640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c1958c0 con 0x7f667c105be0 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f667a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c0ffb30 0x7f667c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f667a59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c0ffb30 0x7f667c19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54892/0 (socket says 172.21.15.31:54892) 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f667a59c640 1 -- 172.21.15.31:0/3700690337 learned_addr learned my addr 172.21.15.31:0/3700690337 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f6679d9b640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f667c105be0 0x7f667c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f667a59c640 1 -- 172.21.15.31:0/3700690337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f667c105be0 msgr2=0x7f667c19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f667a59c640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f667c105be0 0x7f667c19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f667a59c640 1 -- 172.21.15.31:0/3700690337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c009560 con 0x7f667c0ffb30 2024-09-16T09:07:35.358 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f6679d9b640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f667c105be0 0x7f667c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f667a59c640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c0ffb30 0x7f667c19b7d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f666c006010 tx=0x7f666c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f666b7fe640 1 -- 172.21.15.31:0/3700690337 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c004aa0 con 0x7f667c0ffb30 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f666b7fe640 1 -- 172.21.15.31:0/3700690337 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f666c004c00 con 0x7f667c0ffb30 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667c195b40 con 0x7f667c0ffb30 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.339+0000 7f666b7fe640 1 -- 172.21.15.31:0/3700690337 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666c003a20 con 0x7f667c0ffb30 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.340+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c1960b0 con 0x7f667c0ffb30 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.341+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c1011e0 con 0x7f667c0ffb30 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.341+0000 7f666b7fe640 1 -- 172.21.15.31:0/3700690337 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f666c031c90 con 0x7f667c0ffb30 2024-09-16T09:07:35.359 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.342+0000 7f666b7fe640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f666407af30 0x7f666407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.342+0000 7f666b7fe640 1 -- 172.21.15.31:0/3700690337 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f666c0bf250 con 0x7f667c0ffb30 2024-09-16T09:07:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.342+0000 7f6679d9b640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f666407af30 0x7f666407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.342+0000 7f6679d9b640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f666407af30 0x7f666407d3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f667c196d30 tx=0x7f6670005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:35.360 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.346+0000 7f666b7fe640 1 -- 172.21.15.31:0/3700690337 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f666c0822c0 con 0x7f667c0ffb30 2024-09-16T09:07:35.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.547+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f667c1080c0 con 0x7f666407af30 2024-09-16T09:07:35.560 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.559+0000 7f666b7fe640 1 -- 172.21.15.31:0/3700690337 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f667c1080c0 con 0x7f666407af30 2024-09-16T09:07:35.560 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:07:35.560 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (51s) 24s ago 19m 14.6M - 0.27.0 11f11916f8cd b523d960fed9 2024-09-16T09:07:35.560 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (119s) 24s ago 19m 8518k - 19.3.0-4917-g8293d73f 425e316ba8eb 119433b14707 2024-09-16T09:07:35.560 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (114s) 91s ago 18m 8124k - 19.3.0-4917-g8293d73f 425e316ba8eb 3f8a92f1222e 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (7m) 24s ago 19m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (7m) 91s ago 18m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (30s) 24s ago 18m 50.6M - 10.4.0 c8b91775d855 56de3e5878a4 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (3m) 24s ago 12m 84.9M - 19.3.0-4917-g8293d73f 425e316ba8eb 7e4ca0c031e9 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (2m) 24s ago 12m 18.6M - 19.3.0-4917-g8293d73f 425e316ba8eb f54e4116709a 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (2m) 91s ago 12m 16.5M - 19.3.0-4917-g8293d73f 425e316ba8eb 880891e445c6 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (2m) 91s ago 12m 16.2M - 19.3.0-4917-g8293d73f 425e316ba8eb 08131d73c31f 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (9m) 24s ago 20m 533M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (8m) 91s ago 18m 456M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (8m) 24s ago 20m 59.5M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (7m) 91s ago 18m 45.4M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:07:35.561 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (99s) 24s ago 19m 8002k - 1.7.0 72c9c2088986 7588c21d20a2 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (93s) 91s ago 18m 3502k - 1.7.0 72c9c2088986 818f167b5f3e 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (6m) 24s ago 16m 117M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (6m) 24s ago 16m 115M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (5m) 24s ago 15m 133M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (5m) 91s ago 15m 136M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (4m) 91s ago 14m 98.9M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (3m) 91s ago 14m 109M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:07:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (71s) 24s ago 18m 44.1M - 2.51.0 1d3b7f56885b 0383f88326e0 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f666407af30 msgr2=0x7f666407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f666407af30 0x7f666407d3f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f667c196d30 tx=0x7f6670005d30 comp rx=0 tx=0).stop 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c0ffb30 msgr2=0x7f667c19b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c0ffb30 0x7f667c19b7d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f666c006010 tx=0x7f666c0055a0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 shutdown_connections 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f666407af30 0x7f666407d3f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f667c105be0 0x7f667c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 --2- 172.21.15.31:0/3700690337 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f667c0ffb30 0x7f667c19b7d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 >> 172.21.15.31:0/3700690337 conn(0x7f667c0fb810 msgr2=0x7f667c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 shutdown_connections 2024-09-16T09:07:35.564 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:35.562+0000 7f6680f87640 1 -- 172.21.15.31:0/3700690337 wait complete. 2024-09-16T09:07:36.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch upgrade status' 2024-09-16T09:07:36.422 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:07:37.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:36 smithi031 ceph-mon[91884]: pgmap v298: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:37.216 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:36 smithi031 ceph-mon[91884]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:36 smithi097 ceph-mon[79892]: pgmap v298: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:37.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:36 smithi097 ceph-mon[79892]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:38.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3581827464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 msgr2=0x7f1cb4105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:38.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3581827464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb4105d10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009880 tx=0x7f1ca402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3581827464 shutdown_connections 2024-09-16T09:07:38.105 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3581827464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb40ff930 0x7f1cb40ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3581827464 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb4105d10 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3581827464 >> 172.21.15.31:0/3581827464 conn(0x7f1cb40fb630 msgr2=0x7f1cb40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3581827464 shutdown_connections 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3581827464 wait complete. 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 Processor -- start 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.099+0000 7f1cbad9d640 1 -- start start 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cbad9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb40ff930 0x7f1cb419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cbad9d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb419bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cbad9d640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb419c330 con 0x7f1cb4105930 2024-09-16T09:07:38.106 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cbad9d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb41958c0 con 0x7f1cb40ff930 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb959a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb419bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb959a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb419bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54904/0 (socket says 172.21.15.31:54904) 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb959a640 1 -- 172.21.15.31:0/3636766833 learned_addr learned my addr 172.21.15.31:0/3636766833 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb9d9b640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb40ff930 0x7f1cb419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb959a640 1 -- 172.21.15.31:0/3636766833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb40ff930 msgr2=0x7f1cb419b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb959a640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb40ff930 0x7f1cb419b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb959a640 1 -- 172.21.15.31:0/3636766833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009560 con 0x7f1cb4105930 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb9d9b640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb40ff930 0x7f1cb419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cb959a640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb419bd10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b420 tx=0x7f1cb000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1caaffd640 1 -- 172.21.15.31:0/3636766833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb0011800 con 0x7f1cb4105930 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb4195ba0 con 0x7f1cb4105930 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1caaffd640 1 -- 172.21.15.31:0/3636766833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cb0011e20 con 0x7f1cb4105930 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1caaffd640 1 -- 172.21.15.31:0/3636766833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb000f2e0 con 0x7f1cb4105930 2024-09-16T09:07:38.107 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.100+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb41960f0 con 0x7f1cb4105930 2024-09-16T09:07:38.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.101+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb4101070 con 0x7f1cb4105930 2024-09-16T09:07:38.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.102+0000 7f1caaffd640 1 -- 172.21.15.31:0/3636766833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f1cb0010380 con 0x7f1cb4105930 2024-09-16T09:07:38.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.103+0000 7f1caaffd640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1ca007af30 0x7f1ca007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:38.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.103+0000 7f1caaffd640 1 -- 172.21.15.31:0/3636766833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f1cb009d8d0 con 0x7f1cb4105930 2024-09-16T09:07:38.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.103+0000 7f1cb9d9b640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1ca007af30 0x7f1ca007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:38.108 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.104+0000 7f1cb9d9b640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1ca007af30 0x7f1ca007d3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4000c00 tx=0x7f1ca4005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:38.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.108+0000 7f1caaffd640 1 -- 172.21.15.31:0/3636766833 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f1cb00609f0 con 0x7f1cb4105930 2024-09-16T09:07:38.313 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.312+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cb410cc20 con 0x7f1ca007af30 2024-09-16T09:07:38.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.313+0000 7f1caaffd640 1 -- 172.21.15.31:0/3636766833 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f1cb410cc20 con 0x7f1ca007af30 2024-09-16T09:07:38.314 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:07:38.314 INFO:teuthology.orchestra.run.smithi031.stdout: "target_image": null, 2024-09-16T09:07:38.314 INFO:teuthology.orchestra.run.smithi031.stdout: "in_progress": false, 2024-09-16T09:07:38.315 INFO:teuthology.orchestra.run.smithi031.stdout: "which": "", 2024-09-16T09:07:38.315 INFO:teuthology.orchestra.run.smithi031.stdout: "services_complete": [], 2024-09-16T09:07:38.315 INFO:teuthology.orchestra.run.smithi031.stdout: "progress": null, 2024-09-16T09:07:38.315 INFO:teuthology.orchestra.run.smithi031.stdout: "message": "", 2024-09-16T09:07:38.315 INFO:teuthology.orchestra.run.smithi031.stdout: "is_paused": false 2024-09-16T09:07:38.315 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1ca007af30 msgr2=0x7f1ca007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1ca007af30 0x7f1ca007d3f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4000c00 tx=0x7f1ca4005a20 comp rx=0 tx=0).stop 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 msgr2=0x7f1cb419bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb419bd10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000b420 tx=0x7f1cb000b8e0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 shutdown_connections 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f1ca007af30 0x7f1ca007d3f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1cb4105930 0x7f1cb419bd10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 --2- 172.21.15.31:0/3636766833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cb40ff930 0x7f1cb419b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.315+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 >> 172.21.15.31:0/3636766833 conn(0x7f1cb40fb630 msgr2=0x7f1cb40fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.316+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 shutdown_connections 2024-09-16T09:07:38.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:38.316+0000 7f1cbad9d640 1 -- 172.21.15.31:0/3636766833 wait complete. 2024-09-16T09:07:38.980 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:38 smithi031 ceph-mon[91884]: pgmap v299: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:38.980 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:38 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:39.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph health detail' 2024-09-16T09:07:39.213 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:07:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:38 smithi097 ceph-mon[79892]: pgmap v299: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:39.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:38 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:40.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:39 smithi097 ceph-mon[79892]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:40.365 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:39 smithi031 ceph-mon[91884]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 -- 172.21.15.31:0/2523793075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8104a30 msgr2=0x7f07e8104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 --2- 172.21.15.31:0/2523793075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8104a30 0x7f07e8104e10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f07e4009880 tx=0x7f07e402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 -- 172.21.15.31:0/2523793075 shutdown_connections 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 --2- 172.21.15.31:0/2523793075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e80fea30 0x7f07e80fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 --2- 172.21.15.31:0/2523793075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e8104a30 0x7f07e8104e10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 -- 172.21.15.31:0/2523793075 >> 172.21.15.31:0/2523793075 conn(0x7f07e80fa730 msgr2=0x7f07e80fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 -- 172.21.15.31:0/2523793075 shutdown_connections 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.904+0000 7f07efc93640 1 -- 172.21.15.31:0/2523793075 wait complete. 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07efc93640 1 Processor -- start 2024-09-16T09:07:40.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07efc93640 1 -- start start 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07efc93640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e80fea30 0x7f07e819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07efc93640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e8104a30 0x7f07e819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07efc93640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e819c3c0 con 0x7f07e80fea30 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07efc93640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e8195890 con 0x7f07e8104a30 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07eda2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e80fea30 0x7f07e819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07eda2f640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e80fea30 0x7f07e819b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:54918/0 (socket says 172.21.15.31:54918) 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07eda2f640 1 -- 172.21.15.31:0/2314467713 learned_addr learned my addr 172.21.15.31:0/2314467713 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.905+0000 7f07ed22e640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e8104a30 0x7f07e819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07eda2f640 1 -- 172.21.15.31:0/2314467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e8104a30 msgr2=0x7f07e819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07eda2f640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e8104a30 0x7f07e819bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07eda2f640 1 -- 172.21.15.31:0/2314467713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e4009560 con 0x7f07e80fea30 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07eda2f640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e80fea30 0x7f07e819b830 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07e4006010 tx=0x7f07e40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:40.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07daffd640 1 -- 172.21.15.31:0/2314467713 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e4004aa0 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07daffd640 1 -- 172.21.15.31:0/2314467713 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f07e4004c00 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e8195b10 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07daffd640 1 -- 172.21.15.31:0/2314467713 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e4003a20 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.906+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e8196000 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.907+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e8100170 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.908+0000 7f07daffd640 1 -- 172.21.15.31:0/2314467713 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f07e4031c90 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.908+0000 7f07daffd640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f07d407af30 0x7f07d407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.908+0000 7f07daffd640 1 -- 172.21.15.31:0/2314467713 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f07e40bf240 con 0x7f07e80fea30 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.908+0000 7f07ed22e640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f07d407af30 0x7f07d407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.909+0000 7f07ed22e640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f07d407af30 0x7f07d407d3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07dc005e10 tx=0x7f07dc005da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:40.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:40.913+0000 7f07daffd640 1 -- 172.21.15.31:0/2314467713 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f07e4082330 con 0x7f07e80fea30 2024-09-16T09:07:41.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:40 smithi031 ceph-mon[91884]: pgmap v300: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.174+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f07e810bcd0 con 0x7f07e80fea30 2024-09-16T09:07:41.176 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.175+0000 7f07daffd640 1 -- 172.21.15.31:0/2314467713 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f07e4085450 con 0x7f07e80fea30 2024-09-16T09:07:41.176 INFO:teuthology.orchestra.run.smithi031.stdout:HEALTH_OK 2024-09-16T09:07:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f07d407af30 msgr2=0x7f07d407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f07d407af30 0x7f07d407d3f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f07dc005e10 tx=0x7f07dc005da0 comp rx=0 tx=0).stop 2024-09-16T09:07:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e80fea30 msgr2=0x7f07e819b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:41.178 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e80fea30 0x7f07e819b830 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f07e4006010 tx=0x7f07e40055a0 comp rx=0 tx=0).stop 2024-09-16T09:07:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 shutdown_connections 2024-09-16T09:07:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f07d407af30 0x7f07d407d3f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f07e8104a30 0x7f07e819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 --2- 172.21.15.31:0/2314467713 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f07e80fea30 0x7f07e819b830 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 >> 172.21.15.31:0/2314467713 conn(0x7f07e80fa730 msgr2=0x7f07e8068770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 shutdown_connections 2024-09-16T09:07:41.179 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:41.177+0000 7f07efc93640 1 -- 172.21.15.31:0/2314467713 wait complete. 2024-09-16T09:07:41.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:40 smithi097 ceph-mon[79892]: pgmap v300: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:41.796 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph versions' 2024-09-16T09:07:42.004 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:07:42.260 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:41 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2314467713' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:07:42.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:41 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2314467713' entity='client.admin' cmd={"prefix": "health", "detail": "detail"} : dispatch 2024-09-16T09:07:43.021 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:42 smithi031 ceph-mon[91884]: pgmap v301: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:43.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:42 smithi097 ceph-mon[79892]: pgmap v301: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 -- 172.21.15.31:0/3917988075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 msgr2=0x7f7f14105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/3917988075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f14105fe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009880 tx=0x7f7efc02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 -- 172.21.15.31:0/3917988075 shutdown_connections 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/3917988075 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f140ffb50 0x7f7f140fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/3917988075 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f14105fe0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 -- 172.21.15.31:0/3917988075 >> 172.21.15.31:0/3917988075 conn(0x7f7f140fb830 msgr2=0x7f7f140fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 -- 172.21.15.31:0/3917988075 shutdown_connections 2024-09-16T09:07:43.535 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.532+0000 7f7f1ad86640 1 -- 172.21.15.31:0/3917988075 wait complete. 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.533+0000 7f7f1ad86640 1 Processor -- start 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.533+0000 7f7f1ad86640 1 -- start start 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.533+0000 7f7f1ad86640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f140ffb50 0x7f7f1419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.533+0000 7f7f1ad86640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f1419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.533+0000 7f7f1ad86640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1419c2f0 con 0x7f7f14105c00 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.533+0000 7f7f1ad86640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14195880 con 0x7f7f140ffb50 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.533+0000 7f7f13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f1419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f13fff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f1419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56922/0 (socket says 172.21.15.31:56922) 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f13fff640 1 -- 172.21.15.31:0/4166256373 learned_addr learned my addr 172.21.15.31:0/4166256373 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f18b22640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f140ffb50 0x7f7f1419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f13fff640 1 -- 172.21.15.31:0/4166256373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f140ffb50 msgr2=0x7f7f1419b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:43.536 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f13fff640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f140ffb50 0x7f7f1419b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f13fff640 1 -- 172.21.15.31:0/4166256373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc009560 con 0x7f7f14105c00 2024-09-16T09:07:43.537 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f13fff640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f1419bcd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400ea00 tx=0x7f7f0400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f11ffb640 1 -- 172.21.15.31:0/4166256373 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0400cd50 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f11ffb640 1 -- 172.21.15.31:0/4166256373 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f04005020 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f11ffb640 1 -- 172.21.15.31:0/4166256373 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f040116a0 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f14195b60 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.534+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14196130 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.535+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f14101200 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.536+0000 7f7f11ffb640 1 -- 172.21.15.31:0/4166256373 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f7f04005190 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.536+0000 7f7f11ffb640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7f0007af30 0x7f7f0007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.537+0000 7f7f11ffb640 1 -- 172.21.15.31:0/4166256373 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f7f0409e2c0 con 0x7f7f14105c00 2024-09-16T09:07:43.538 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.537+0000 7f7f18b22640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7f0007af30 0x7f7f0007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:07:43.539 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.537+0000 7f7f18b22640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7f0007af30 0x7f7f0007d3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009230 tx=0x7f7efc0058d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:07:43.542 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.541+0000 7f7f11ffb640 1 -- 172.21.15.31:0/4166256373 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f7f04061460 con 0x7f7f14105c00 2024-09-16T09:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.799+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7f140fffb0 con 0x7f7f14105c00 2024-09-16T09:07:43.800 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.799+0000 7f7f11ffb640 1 -- 172.21.15.31:0/4166256373 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7f04064580 con 0x7f7f14105c00 2024-09-16T09:07:43.800 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:07:43.800 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:07:43.801 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 14 2024-09-16T09:07:43.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:07:43.802 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7f0007af30 msgr2=0x7f7f0007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7f0007af30 0x7f7f0007d3f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009230 tx=0x7f7efc0058d0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 msgr2=0x7f7f1419bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f1419bcd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400ea00 tx=0x7f7f0400eec0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 shutdown_connections 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7f0007af30 0x7f7f0007d3f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f14105c00 0x7f7f1419bcd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 --2- 172.21.15.31:0/4166256373 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f140ffb50 0x7f7f1419b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 >> 172.21.15.31:0/4166256373 conn(0x7f7f140fb830 msgr2=0x7f7f140fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 shutdown_connections 2024-09-16T09:07:43.803 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:07:43.802+0000 7f7f1ad86640 1 -- 172.21.15.31:0/4166256373 wait complete. 2024-09-16T09:07:44.103 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:43 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/4166256373' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:44.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:43 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/4166256373' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:07:44.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-16T09:07:44.632 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:07:45.006 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:44 smithi031 ceph-mon[91884]: pgmap v302: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:44 smithi097 ceph-mon[79892]: pgmap v302: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:46.330 INFO:teuthology.orchestra.run.smithi031.stdout:wait for servicemap items w/ changing names to refresh 2024-09-16T09:07:47.010 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'sleep 60' 2024-09-16T09:07:47.215 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:46 smithi031 ceph-mon[91884]: pgmap v303: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:47.224 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:07:47.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:46 smithi097 ceph-mon[79892]: pgmap v303: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:49.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:48 smithi031 ceph-mon[91884]: pgmap v304: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:48 smithi097 ceph-mon[79892]: pgmap v304: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:50 smithi097 ceph-mon[79892]: pgmap v305: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:51.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:50 smithi031 ceph-mon[91884]: pgmap v305: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:53.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:52 smithi097 ceph-mon[79892]: pgmap v306: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:53.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:52 smithi031 ceph-mon[91884]: pgmap v306: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:54.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:53 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:54.274 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:53 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:07:55.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:54 smithi097 ceph-mon[79892]: pgmap v307: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:55.286 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:54 smithi031 ceph-mon[91884]: pgmap v307: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:57.217 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:56 smithi031 ceph-mon[91884]: pgmap v308: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:56 smithi097 ceph-mon[79892]: pgmap v308: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:07:58 smithi097 ceph-mon[79892]: pgmap v309: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:07:59.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:07:58 smithi031 ceph-mon[91884]: pgmap v309: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:01.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:00 smithi097 ceph-mon[79892]: pgmap v310: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:01.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:00 smithi031 ceph-mon[91884]: pgmap v310: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:02 smithi097 ceph-mon[79892]: pgmap v311: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:03.390 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:02 smithi031 ceph-mon[91884]: pgmap v311: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:04 smithi097 ceph-mon[79892]: pgmap v312: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:05.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:04 smithi031 ceph-mon[91884]: pgmap v312: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:06 smithi097 ceph-mon[79892]: pgmap v313: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:07.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:06 smithi031 ceph-mon[91884]: pgmap v313: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:09.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:08 smithi097 ceph-mon[79892]: pgmap v314: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:09.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:08 smithi031 ceph-mon[91884]: pgmap v314: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:09.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:11.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:10 smithi097 ceph-mon[79892]: pgmap v315: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:11.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:10 smithi031 ceph-mon[91884]: pgmap v315: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:13.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:12 smithi097 ceph-mon[79892]: pgmap v316: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:13.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:12 smithi031 ceph-mon[91884]: pgmap v316: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:14 smithi097 ceph-mon[79892]: pgmap v317: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:15.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:14 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:08:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:14 smithi031 ceph-mon[91884]: pgmap v317: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:15.373 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:14 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:08:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:08:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:08:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:16 smithi097 ceph-mon[79892]: pgmap v318: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:08:16.512 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:08:16.513 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:08:16.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:08:16.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:08:16.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:16 smithi031 ceph-mon[91884]: pgmap v318: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:16.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:08:16.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:08:16.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:08:18.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:18 smithi097 ceph-mon[79892]: pgmap v319: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:18.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:18 smithi031 ceph-mon[91884]: pgmap v319: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:20.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:20 smithi097 ceph-mon[79892]: pgmap v320: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:20.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:20 smithi031 ceph-mon[91884]: pgmap v320: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:22.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:22 smithi097 ceph-mon[79892]: pgmap v321: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:22.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:22 smithi031 ceph-mon[91884]: pgmap v321: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:24 smithi097 ceph-mon[79892]: pgmap v322: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:24.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:24.797 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:24 smithi031 ceph-mon[91884]: pgmap v322: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:24.797 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:26.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:26 smithi097 ceph-mon[79892]: pgmap v323: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:26.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:26 smithi031 ceph-mon[91884]: pgmap v323: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:28.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:28 smithi097 ceph-mon[79892]: pgmap v324: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:28 smithi031 ceph-mon[91884]: pgmap v324: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:30.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:30 smithi097 ceph-mon[79892]: pgmap v325: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:30.877 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:30 smithi031 ceph-mon[91884]: pgmap v325: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:32.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:32 smithi097 ceph-mon[79892]: pgmap v326: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:32.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:32 smithi031 ceph-mon[91884]: pgmap v326: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:34.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:34 smithi097 ceph-mon[79892]: pgmap v327: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:34.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:34 smithi031 ceph-mon[91884]: pgmap v327: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:36.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:36 smithi097 ceph-mon[79892]: pgmap v328: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:36.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:36 smithi031 ceph-mon[91884]: pgmap v328: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:38.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:38 smithi097 ceph-mon[79892]: pgmap v329: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:38.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:38 smithi031 ceph-mon[91884]: pgmap v329: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:39.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:39.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:40.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:40 smithi097 ceph-mon[79892]: pgmap v330: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:40.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:40 smithi031 ceph-mon[91884]: pgmap v330: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:42.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:42 smithi097 ceph-mon[79892]: pgmap v331: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:42.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:42 smithi031 ceph-mon[91884]: pgmap v331: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:44.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:44 smithi097 ceph-mon[79892]: pgmap v332: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:44.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:44 smithi031 ceph-mon[91884]: pgmap v332: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:46.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:46 smithi097 ceph-mon[79892]: pgmap v333: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:46.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:46 smithi031 ceph-mon[91884]: pgmap v333: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:48.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:48 smithi097 ceph-mon[79892]: pgmap v334: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:48.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:48 smithi031 ceph-mon[91884]: pgmap v334: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:49.624 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph orch ps' 2024-09-16T09:08:49.841 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:08:50.571 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:50 smithi031 ceph-mon[91884]: pgmap v335: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:50.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:50 smithi097 ceph-mon[79892]: pgmap v335: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.486+0000 7f87c23a8640 1 -- 172.21.15.31:0/2230670168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fedc0 msgr2=0x7f87bc102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.486+0000 7f87c23a8640 1 --2- 172.21.15.31:0/2230670168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fedc0 0x7f87bc102e10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009880 tx=0x7f87b002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.504 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.486+0000 7f87c23a8640 1 -- 172.21.15.31:0/2230670168 shutdown_connections 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.487+0000 7f87c23a8640 1 --2- 172.21.15.31:0/2230670168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fedc0 0x7f87bc102e10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.487+0000 7f87c23a8640 1 --2- 172.21.15.31:0/2230670168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0fe410 0x7f87bc0fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.487+0000 7f87c23a8640 1 -- 172.21.15.31:0/2230670168 >> 172.21.15.31:0/2230670168 conn(0x7f87bc0f9e70 msgr2=0x7f87bc0fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.487+0000 7f87c23a8640 1 -- 172.21.15.31:0/2230670168 shutdown_connections 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.487+0000 7f87c23a8640 1 -- 172.21.15.31:0/2230670168 wait complete. 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.487+0000 7f87c23a8640 1 Processor -- start 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.487+0000 7f87c23a8640 1 -- start start 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87c23a8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fe410 0x7f87bc19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87c23a8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0fedc0 0x7f87bc19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:51.505 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87c23a8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc19c360 con 0x7f87bc0fe410 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87c23a8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc1958f0 con 0x7f87bc0fedc0 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fe410 0x7f87bc19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fe410 0x7f87bc19b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33652/0 (socket says 172.21.15.31:33652) 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bbfff640 1 -- 172.21.15.31:0/3154346321 learned_addr learned my addr 172.21.15.31:0/3154346321 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0fedc0 0x7f87bc19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bbfff640 1 -- 172.21.15.31:0/3154346321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0fedc0 msgr2=0x7f87bc19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bbfff640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0fedc0 0x7f87bc19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bbfff640 1 -- 172.21.15.31:0/3154346321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0009560 con 0x7f87bc0fe410 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87bbfff640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fe410 0x7f87bc19b800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00c810 tx=0x7f87ac00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.488+0000 7f87b97fa640 1 -- 172.21.15.31:0/3154346321 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87ac0041f0 con 0x7f87bc0fe410 2024-09-16T09:08:51.506 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.489+0000 7f87b97fa640 1 -- 172.21.15.31:0/3154346321 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87ac007e70 con 0x7f87bc0fe410 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.489+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87bc195bd0 con 0x7f87bc0fe410 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.489+0000 7f87b97fa640 1 -- 172.21.15.31:0/3154346321 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87ac0049d0 con 0x7f87bc0fe410 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.489+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87bc1960f0 con 0x7f87bc0fe410 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.490+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87bc0ff8f0 con 0x7f87bc0fe410 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.493+0000 7f87b97fa640 1 -- 172.21.15.31:0/3154346321 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f87ac0079b0 con 0x7f87bc0fe410 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.493+0000 7f87b97fa640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a407af30 0x7f87a407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.493+0000 7f87b97fa640 1 -- 172.21.15.31:0/3154346321 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f87ac09e1b0 con 0x7f87bc0fe410 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.493+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a407af30 0x7f87a407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.494+0000 7f87bb7fe640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a407af30 0x7f87a407d3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009230 tx=0x7f87b00058d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:51.507 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.496+0000 7f87b97fa640 1 -- 172.21.15.31:0/3154346321 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f87ac0613f0 con 0x7f87bc0fe410 2024-09-16T09:08:51.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.703+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 --> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f87bc10b4f0 con 0x7f87a407af30 2024-09-16T09:08:51.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.712+0000 7f87b97fa640 1 -- 172.21.15.31:0/3154346321 <== mgr.34104 v2:172.21.15.31:6800/3442236616 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f87bc10b4f0 con 0x7f87a407af30 2024-09-16T09:08:51.715 INFO:teuthology.orchestra.run.smithi031.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T09:08:51.715 INFO:teuthology.orchestra.run.smithi031.stdout:alertmanager.smithi031 smithi031 *:9093,9094 running (2m) 101s ago 20m 14.6M - 0.27.0 11f11916f8cd b523d960fed9 2024-09-16T09:08:51.715 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi031 smithi031 running (3m) 101s ago 20m 8518k - 19.3.0-4917-g8293d73f 425e316ba8eb 119433b14707 2024-09-16T09:08:51.716 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-exporter.smithi097 smithi097 running (3m) 2m ago 19m 8124k - 19.3.0-4917-g8293d73f 425e316ba8eb 3f8a92f1222e 2024-09-16T09:08:51.716 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi031 smithi031 running (8m) 101s ago 20m 6899k - 19.3.0-4917-g8293d73f 425e316ba8eb d0cb5a2c6459 2024-09-16T09:08:51.716 INFO:teuthology.orchestra.run.smithi031.stdout:crash.smithi097 smithi097 running (8m) 2m ago 19m 6891k - 19.3.0-4917-g8293d73f 425e316ba8eb d30a48247659 2024-09-16T09:08:51.716 INFO:teuthology.orchestra.run.smithi031.stdout:grafana.smithi031 smithi031 *:3000 running (106s) 101s ago 20m 50.6M - 10.4.0 c8b91775d855 56de3e5878a4 2024-09-16T09:08:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.lcfutm smithi031 running (4m) 101s ago 14m 84.9M - 19.3.0-4917-g8293d73f 425e316ba8eb 7e4ca0c031e9 2024-09-16T09:08:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi031.mjhxgi smithi031 running (4m) 101s ago 14m 18.6M - 19.3.0-4917-g8293d73f 425e316ba8eb f54e4116709a 2024-09-16T09:08:51.717 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.cagmus smithi097 running (3m) 2m ago 14m 16.5M - 19.3.0-4917-g8293d73f 425e316ba8eb 880891e445c6 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:mds.cephfs.smithi097.sbwmfq smithi097 running (3m) 2m ago 14m 16.2M - 19.3.0-4917-g8293d73f 425e316ba8eb 08131d73c31f 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi031.dcmqpv smithi031 *:8443,9283,8765 running (10m) 101s ago 21m 533M - 19.3.0-4917-g8293d73f 425e316ba8eb 47c225fb18a0 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:mgr.smithi097.mpashf smithi097 *:8443,9283,8765 running (9m) 2m ago 19m 456M - 19.3.0-4917-g8293d73f 425e316ba8eb 12f9902a3858 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi031 smithi031 running (9m) 101s ago 21m 59.5M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb 13ee87bf1609 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:mon.smithi097 smithi097 running (9m) 2m ago 19m 45.4M 2048M 19.3.0-4917-g8293d73f 425e316ba8eb ab1d49446e37 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi031 smithi031 *:9100 running (2m) 101s ago 20m 8002k - 1.7.0 72c9c2088986 7588c21d20a2 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:node-exporter.smithi097 smithi097 *:9100 running (2m) 2m ago 19m 3502k - 1.7.0 72c9c2088986 818f167b5f3e 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 smithi031 running (8m) 101s ago 17m 117M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 8060870d2228 2024-09-16T09:08:51.718 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 smithi031 running (7m) 101s ago 17m 115M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb 539fc5d4e6cf 2024-09-16T09:08:51.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 smithi031 running (6m) 101s ago 16m 133M 1217M 19.3.0-4917-g8293d73f 425e316ba8eb cd7af82048da 2024-09-16T09:08:51.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 smithi097 running (6m) 2m ago 16m 136M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 9d23936a9f41 2024-09-16T09:08:51.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 smithi097 running (5m) 2m ago 16m 98.9M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 3c82b6b87cd9 2024-09-16T09:08:51.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 smithi097 running (4m) 2m ago 15m 109M 2240M 19.3.0-4917-g8293d73f 425e316ba8eb 857ff6eb4b2a 2024-09-16T09:08:51.719 INFO:teuthology.orchestra.run.smithi031.stdout:prometheus.smithi031 smithi031 *:9095 running (2m) 101s ago 19m 44.1M - 2.51.0 1d3b7f56885b 0383f88326e0 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a407af30 msgr2=0x7f87a407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a407af30 0x7f87a407d3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87b0009230 tx=0x7f87b00058d0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fe410 msgr2=0x7f87bc19b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fe410 0x7f87bc19b800 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f87ac00c810 tx=0x7f87ac00ccd0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 shutdown_connections 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f87a407af30 0x7f87a407d3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87bc0fedc0 0x7f87bc19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 --2- 172.21.15.31:0/3154346321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f87bc0fe410 0x7f87bc19b800 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 >> 172.21.15.31:0/3154346321 conn(0x7f87bc0f9e70 msgr2=0x7f87bc101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.715+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 shutdown_connections 2024-09-16T09:08:51.720 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:51.716+0000 7f87c23a8640 1 -- 172.21.15.31:0/3154346321 wait complete. 2024-09-16T09:08:52.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph versions' 2024-09-16T09:08:52.648 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:08:52.711 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:52 smithi031 ceph-mon[91884]: pgmap v336: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:52.711 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:52 smithi031 ceph-mon[91884]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:08:52.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:52 smithi097 ceph-mon[79892]: pgmap v336: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:52.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:52 smithi097 ceph-mon[79892]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:08:54.286 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.216+0000 7f4bf109e640 1 -- 172.21.15.31:0/4224439648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 msgr2=0x7f4bec104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.216+0000 7f4bf109e640 1 --2- 172.21.15.31:0/4224439648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec104e10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009c20 tx=0x7f4bdc02a970 comp rx=0 tx=0).stop 2024-09-16T09:08:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.216+0000 7f4bf109e640 1 -- 172.21.15.31:0/4224439648 shutdown_connections 2024-09-16T09:08:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.216+0000 7f4bf109e640 1 --2- 172.21.15.31:0/4224439648 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bec0fea30 0x7f4bec0fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.216+0000 7f4bf109e640 1 --2- 172.21.15.31:0/4224439648 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec104e10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.216+0000 7f4bf109e640 1 -- 172.21.15.31:0/4224439648 >> 172.21.15.31:0/4224439648 conn(0x7f4bec0fa710 msgr2=0x7f4bec0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:08:54.287 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.217+0000 7f4bf109e640 1 -- 172.21.15.31:0/4224439648 shutdown_connections 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.217+0000 7f4bf109e640 1 -- 172.21.15.31:0/4224439648 wait complete. 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.217+0000 7f4bf109e640 1 Processor -- start 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.217+0000 7f4bf109e640 1 -- start start 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bf109e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bec0fea30 0x7f4bec19b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bf109e640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec19bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bf109e640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec19c330 con 0x7f4bec104a30 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bf109e640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec195950 con 0x7f4bec0fea30 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bea59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec19bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bea59c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec19bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33662/0 (socket says 172.21.15.31:33662) 2024-09-16T09:08:54.288 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bea59c640 1 -- 172.21.15.31:0/495698507 learned_addr learned my addr 172.21.15.31:0/495698507 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:08:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bead9d640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bec0fea30 0x7f4bec19b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bea59c640 1 -- 172.21.15.31:0/495698507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bec0fea30 msgr2=0x7f4bec19b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bea59c640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bec0fea30 0x7f4bec19b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bea59c640 1 -- 172.21.15.31:0/495698507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc009900 con 0x7f4bec104a30 2024-09-16T09:08:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bead9d640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bec0fea30 0x7f4bec19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:08:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.218+0000 7f4bea59c640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec19bda0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c8a0 tx=0x7f4be000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:54.289 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.219+0000 7f4bd3fff640 1 -- 172.21.15.31:0/495698507 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be0007b80 con 0x7f4bec104a30 2024-09-16T09:08:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.219+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec195c30 con 0x7f4bec104a30 2024-09-16T09:08:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.219+0000 7f4bd3fff640 1 -- 172.21.15.31:0/495698507 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4be0007ce0 con 0x7f4bec104a30 2024-09-16T09:08:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.219+0000 7f4bd3fff640 1 -- 172.21.15.31:0/495698507 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be0018810 con 0x7f4bec104a30 2024-09-16T09:08:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.219+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec196180 con 0x7f4bec104a30 2024-09-16T09:08:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.220+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec100170 con 0x7f4bec104a30 2024-09-16T09:08:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.221+0000 7f4bd3fff640 1 -- 172.21.15.31:0/495698507 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f4be0018a80 con 0x7f4bec104a30 2024-09-16T09:08:54.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.221+0000 7f4bd3fff640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4bd407af30 0x7f4bd407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.221+0000 7f4bd3fff640 1 -- 172.21.15.31:0/495698507 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f4be009e2a0 con 0x7f4bec104a30 2024-09-16T09:08:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.221+0000 7f4bead9d640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4bd407af30 0x7f4bd407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.222+0000 7f4bead9d640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4bd407af30 0x7f4bd407d3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098d0 tx=0x7f4bdc009360 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:54.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.226+0000 7f4bd3fff640 1 -- 172.21.15.31:0/495698507 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f4be0061310 con 0x7f4bec104a30 2024-09-16T09:08:54.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.487+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4bec0fee90 con 0x7f4bec104a30 2024-09-16T09:08:54.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.487+0000 7f4bd3fff640 1 -- 172.21.15.31:0/495698507 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4be0064430 con 0x7f4bec104a30 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "mon": { 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": { 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 2 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": { 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 6 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "mds": { 2024-09-16T09:08:54.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 4 2024-09-16T09:08:54.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-09-16T09:08:54.490 INFO:teuthology.orchestra.run.smithi031.stdout: "overall": { 2024-09-16T09:08:54.490 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)": 14 2024-09-16T09:08:54.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-09-16T09:08:54.490 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-09-16T09:08:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4bd407af30 msgr2=0x7f4bd407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4bd407af30 0x7f4bd407d3f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0098d0 tx=0x7f4bdc009360 comp rx=0 tx=0).stop 2024-09-16T09:08:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 msgr2=0x7f4bec19bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec19bda0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c8a0 tx=0x7f4be000cd60 comp rx=0 tx=0).stop 2024-09-16T09:08:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 shutdown_connections 2024-09-16T09:08:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4bd407af30 0x7f4bd407d3f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:54.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4bec104a30 0x7f4bec19bda0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:54.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 --2- 172.21.15.31:0/495698507 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4bec0fea30 0x7f4bec19b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:54.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 >> 172.21.15.31:0/495698507 conn(0x7f4bec0fa710 msgr2=0x7f4bec068780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:08:54.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 shutdown_connections 2024-09-16T09:08:54.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:54.490+0000 7f4bf109e640 1 -- 172.21.15.31:0/495698507 wait complete. 2024-09-16T09:08:54.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:54 smithi031 ceph-mon[91884]: pgmap v337: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:54.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:54.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:54 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/495698507' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:08:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:54 smithi097 ceph-mon[79892]: pgmap v337: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:08:54.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:54 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/495698507' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:08:55.210 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-16T09:08:55.422 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:08:56.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:56 smithi097 ceph-mon[79892]: pgmap v338: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:56.829 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:56 smithi031 ceph-mon[91884]: pgmap v338: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 -- 172.21.15.31:0/4061112282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 msgr2=0x7fa9e0109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/4061112282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e0109ac0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009880 tx=0x7fa9c802a530 comp rx=0 tx=0).stop 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 -- 172.21.15.31:0/4061112282 shutdown_connections 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/4061112282 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e0109ac0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/4061112282 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e0100cb0 0x7fa9e0101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 -- 172.21.15.31:0/4061112282 >> 172.21.15.31:0/4061112282 conn(0x7fa9e00758a0 msgr2=0x7fa9e00fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 -- 172.21.15.31:0/4061112282 shutdown_connections 2024-09-16T09:08:57.032 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 -- 172.21.15.31:0/4061112282 wait complete. 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.031+0000 7fa9e6f01640 1 Processor -- start 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9e6f01640 1 -- start start 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9e6f01640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e0100cb0 0x7fa9e019dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9e6f01640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e019e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9e6f01640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0197b20 con 0x7fa9e01015d0 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9e6f01640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0197c60 con 0x7fa9e0100cb0 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e019e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e019e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33690/0 (socket says 172.21.15.31:33690) 2024-09-16T09:08:57.034 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9dffff640 1 -- 172.21.15.31:0/1951125207 learned_addr learned my addr 172.21.15.31:0/1951125207 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9e4c9d640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e0100cb0 0x7fa9e019dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9dffff640 1 -- 172.21.15.31:0/1951125207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e0100cb0 msgr2=0x7fa9e019dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9dffff640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e0100cb0 0x7fa9e019dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9dffff640 1 -- 172.21.15.31:0/1951125207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c8009560 con 0x7fa9e01015d0 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9e4c9d640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e0100cb0 0x7fa9e019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.032+0000 7fa9dffff640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e019e000 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9c802aa50 tx=0x7fa9c80055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.033+0000 7fa9ddffb640 1 -- 172.21.15.31:0/1951125207 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9c80048d0 con 0x7fa9e01015d0 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.033+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e0197ee0 con 0x7fa9e01015d0 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.033+0000 7fa9ddffb640 1 -- 172.21.15.31:0/1951125207 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9c8031640 con 0x7fa9e01015d0 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.033+0000 7fa9ddffb640 1 -- 172.21.15.31:0/1951125207 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9c80038d0 con 0x7fa9e01015d0 2024-09-16T09:08:57.035 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.033+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e01983d0 con 0x7fa9e01015d0 2024-09-16T09:08:57.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.034+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0068510 con 0x7fa9e01015d0 2024-09-16T09:08:57.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.034+0000 7fa9ddffb640 1 -- 172.21.15.31:0/1951125207 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fa9c802ad50 con 0x7fa9e01015d0 2024-09-16T09:08:57.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.035+0000 7fa9ddffb640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9cc07af30 0x7fa9cc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:57.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.035+0000 7fa9ddffb640 1 -- 172.21.15.31:0/1951125207 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fa9c80beea0 con 0x7fa9e01015d0 2024-09-16T09:08:57.036 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.035+0000 7fa9e4c9d640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9cc07af30 0x7fa9cc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:57.037 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.036+0000 7fa9e4c9d640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9cc07af30 0x7fa9cc07d3f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005fd0 tx=0x7fa9d0009520 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:57.040 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.040+0000 7fa9ddffb640 1 -- 172.21.15.31:0/1951125207 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fa9c8081e70 con 0x7fa9e01015d0 2024-09-16T09:08:57.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.299+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa9e010cda0 con 0x7fa9e01015d0 2024-09-16T09:08:57.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.300+0000 7fa9ddffb640 1 -- 172.21.15.31:0/1951125207 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa9c8084f90 con 0x7fa9e01015d0 2024-09-16T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9cc07af30 msgr2=0x7fa9cc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9cc07af30 0x7fa9cc07d3f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa9d0005fd0 tx=0x7fa9d0009520 comp rx=0 tx=0).stop 2024-09-16T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 msgr2=0x7fa9e019e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:57.304 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e019e000 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa9c802aa50 tx=0x7fa9c80055a0 comp rx=0 tx=0).stop 2024-09-16T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 shutdown_connections 2024-09-16T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fa9cc07af30 0x7fa9cc07d3f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9e01015d0 0x7fa9e019e000 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 --2- 172.21.15.31:0/1951125207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa9e0100cb0 0x7fa9e019dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 >> 172.21.15.31:0/1951125207 conn(0x7fa9e00758a0 msgr2=0x7fa9e00fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:08:57.305 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 shutdown_connections 2024-09-16T09:08:57.306 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:57.302+0000 7fa9e6f01640 1 -- 172.21.15.31:0/1951125207 wait complete. 2024-09-16T09:08:57.312 INFO:teuthology.orchestra.run.smithi031.stdout:true 2024-09-16T09:08:57.761 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:57 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1951125207' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:08:57.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:57 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1951125207' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:08:57.937 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -e sha1=8293d73f8690540e843a81caec373f9cc29cf705 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-09-16T09:08:58.151 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:08:58.530 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:08:58 smithi031 ceph-mon[91884]: pgmap v339: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:58.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:08:58 smithi097 ceph-mon[79892]: pgmap v339: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:08:59.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 -- 172.21.15.31:0/248343018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 msgr2=0x7f302010bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:59.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 --2- 172.21.15.31:0/248343018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302010bef0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3010009880 tx=0x7f301002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:08:59.854 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 -- 172.21.15.31:0/248343018 shutdown_connections 2024-09-16T09:08:59.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 --2- 172.21.15.31:0/248343018 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302010bef0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:59.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 --2- 172.21.15.31:0/248343018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30200fe880 0x7f30200fec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:59.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 -- 172.21.15.31:0/248343018 >> 172.21.15.31:0/248343018 conn(0x7f30200fa730 msgr2=0x7f30200fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:08:59.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 -- 172.21.15.31:0/248343018 shutdown_connections 2024-09-16T09:08:59.855 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 -- 172.21.15.31:0/248343018 wait complete. 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.845+0000 7f30278cf640 1 Processor -- start 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f30278cf640 1 -- start start 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f30278cf640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30200fe880 0x7f302019d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f30278cf640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f30278cf640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302019e530 con 0x7f30200ff230 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f30278cf640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3020197ac0 con 0x7f30200fe880 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f3024e6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f3024e6a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302019df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:33706/0 (socket says 172.21.15.31:33706) 2024-09-16T09:08:59.856 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f3024e6a640 1 -- 172.21.15.31:0/2856859861 learned_addr learned my addr 172.21.15.31:0/2856859861 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f302566b640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30200fe880 0x7f302019d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f3024e6a640 1 -- 172.21.15.31:0/2856859861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30200fe880 msgr2=0x7f302019d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f3024e6a640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30200fe880 0x7f302019d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f3024e6a640 1 -- 172.21.15.31:0/2856859861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010009560 con 0x7f30200ff230 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f302566b640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30200fe880 0x7f302019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f3024e6a640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302019df10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f301002a9c0 tx=0x7f30100055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.846+0000 7f30167fc640 1 -- 172.21.15.31:0/2856859861 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3010004aa0 con 0x7f30200ff230 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.847+0000 7f30167fc640 1 -- 172.21.15.31:0/2856859861 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3010004c00 con 0x7f30200ff230 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.847+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3020197da0 con 0x7f30200ff230 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.847+0000 7f30167fc640 1 -- 172.21.15.31:0/2856859861 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3010003980 con 0x7f30200ff230 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.847+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30201982f0 con 0x7f30200ff230 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.848+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3020068d30 con 0x7f30200ff230 2024-09-16T09:08:59.857 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.848+0000 7f30167fc640 1 -- 172.21.15.31:0/2856859861 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f3010003ae0 con 0x7f30200ff230 2024-09-16T09:08:59.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.849+0000 7f30167fc640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f300c07af30 0x7f300c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:08:59.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.849+0000 7f30167fc640 1 -- 172.21.15.31:0/2856859861 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f30100bf030 con 0x7f30200ff230 2024-09-16T09:08:59.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.849+0000 7f302566b640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f300c07af30 0x7f300c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:08:59.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.849+0000 7f302566b640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f300c07af30 0x7f300c07d3f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f301c009b40 tx=0x7f301c005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:08:59.858 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:08:59.853+0000 7f30167fc640 1 -- 172.21.15.31:0/2856859861 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f3010082120 con 0x7f30200ff230 2024-09-16T09:09:00.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.112+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f30200fec60 con 0x7f30200ff230 2024-09-16T09:09:00.113 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.112+0000 7f30167fc640 1 -- 172.21.15.31:0/2856859861 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3010085240 con 0x7f30200ff230 2024-09-16T09:09:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f300c07af30 msgr2=0x7f300c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:00.115 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f300c07af30 0x7f300c07d3f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f301c009b40 tx=0x7f301c005e50 comp rx=0 tx=0).stop 2024-09-16T09:09:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 msgr2=0x7f302019df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302019df10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f301002a9c0 tx=0x7f30100055a0 comp rx=0 tx=0).stop 2024-09-16T09:09:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 shutdown_connections 2024-09-16T09:09:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f300c07af30 0x7f300c07d3f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30200ff230 0x7f302019df10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:00.116 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 --2- 172.21.15.31:0/2856859861 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30200fe880 0x7f302019d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 >> 172.21.15.31:0/2856859861 conn(0x7f30200fa730 msgr2=0x7f30200fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 shutdown_connections 2024-09-16T09:09:00.117 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:00.114+0000 7f30278cf640 1 -- 172.21.15.31:0/2856859861 wait complete. 2024-09-16T09:09:00.124 INFO:teuthology.orchestra.run.smithi031.stdout: "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)" 2024-09-16T09:09:00.724 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:00 smithi031 ceph-mon[91884]: pgmap v340: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:00.725 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:00 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2856859861' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:09:00.749 DEBUG:teuthology.parallel:result is None 2024-09-16T09:09:00.749 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T09:09:00.761 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T09:09:00.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T09:09:00.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:00 smithi097 ceph-mon[79892]: pgmap v340: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:00.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:00 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2856859861' entity='client.admin' cmd={"prefix": "versions"} : dispatch 2024-09-16T09:09:00.975 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:02.584 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 -- 172.21.15.31:0/3520628385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec101630 msgr2=0x7fcdec109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/3520628385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec101630 0x7fcdec109b20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcde00098b0 tx=0x7fcde002a560 comp rx=0 tx=0).stop 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 -- 172.21.15.31:0/3520628385 shutdown_connections 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/3520628385 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec101630 0x7fcdec109b20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/3520628385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec100d10 0x7fcdec1010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 -- 172.21.15.31:0/3520628385 >> 172.21.15.31:0/3520628385 conn(0x7fcdec0758a0 msgr2=0x7fcdec0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 -- 172.21.15.31:0/3520628385 shutdown_connections 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.571+0000 7fcdf15a9640 1 -- 172.21.15.31:0/3520628385 wait complete. 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdf15a9640 1 Processor -- start 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdf15a9640 1 -- start start 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdf15a9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec100d10 0x7fcdec19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdf15a9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec101630 0x7fcdec19bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:02.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdf15a9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec19c410 con 0x7fcdec100d10 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdf15a9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1959a0 con 0x7fcdec101630 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec100d10 0x7fcdec19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec100d10 0x7fcdec19b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48428/0 (socket says 172.21.15.31:48428) 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdeaffd640 1 -- 172.21.15.31:0/1429566645 learned_addr learned my addr 172.21.15.31:0/1429566645 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdea7fc640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec101630 0x7fcdec19bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdeaffd640 1 -- 172.21.15.31:0/1429566645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec101630 msgr2=0x7fcdec19bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdeaffd640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec101630 0x7fcdec19bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.572+0000 7fcdeaffd640 1 -- 172.21.15.31:0/1429566645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0009560 con 0x7fcdec100d10 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.573+0000 7fcdeaffd640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec100d10 0x7fcdec19b8b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e970 tx=0x7fcddc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.573+0000 7fcdd3fff640 1 -- 172.21.15.31:0/1429566645 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcddc00cc90 con 0x7fcdec100d10 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.573+0000 7fcdd3fff640 1 -- 172.21.15.31:0/1429566645 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcddc00cdf0 con 0x7fcdec100d10 2024-09-16T09:09:02.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.573+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec195c80 con 0x7fcdec100d10 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.573+0000 7fcdd3fff640 1 -- 172.21.15.31:0/1429566645 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcddc010430 con 0x7fcdec100d10 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.573+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec1961d0 con 0x7fcdec100d10 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.574+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec0684b0 con 0x7fcdec100d10 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.575+0000 7fcdd3fff640 1 -- 172.21.15.31:0/1429566645 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fcddc0046c0 con 0x7fcdec100d10 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.575+0000 7fcdd3fff640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcdd407af30 0x7fcdd407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.575+0000 7fcdd3fff640 1 -- 172.21.15.31:0/1429566645 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fcddc014070 con 0x7fcdec100d10 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.575+0000 7fcdea7fc640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcdd407af30 0x7fcdd407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.576+0000 7fcdea7fc640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcdd407af30 0x7fcdd407d3f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0004040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:02.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.580+0000 7fcdd3fff640 1 -- 172.21.15.31:0/1429566645 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fcddc0616f0 con 0x7fcdec100d10 2024-09-16T09:09:02.623 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:02 smithi031 ceph-mon[91884]: pgmap v341: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:02.762 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:02 smithi097 ceph-mon[79892]: pgmap v341: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:02.804 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.803+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcdec1010f0 con 0x7fcdec100d10 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.804+0000 7fcdd3fff640 1 -- 172.21.15.31:0/1429566645 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1987 (secure 0 0 0) 0x7fcddc064810 con 0x7fcdec100d10 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:e38 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:btime 2024-09-16T09:05:32:642886+0000 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:legacy client fscid: 1 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:02.805 INFO:teuthology.orchestra.run.smithi031.stdout:Filesystem 'cephfs' (1) 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:fs_name cephfs 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 38 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-09-16T08:54:37.289736+0000 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-09-16T09:05:32.642884+0000 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:tableserver 0 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:root 0 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:session_timeout 60 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:session_autoclose 300 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:max_file_size 1099511627776 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:max_xattr_size 65536 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:required_client_features {} 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure 0 2024-09-16T09:09:02.806 INFO:teuthology.orchestra.run.smithi031.stdout:last_failure_osd_epoch 99 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:max_mds 2 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:in 0,1 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:up {0=34430,1=34434} 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:failed 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:damaged 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:stopped 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:data_pools [3] 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:metadata_pool 2 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:inline_data disabled 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:balancer 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:bal_rank_mask -1 2024-09-16T09:09:02.807 INFO:teuthology.orchestra.run.smithi031.stdout:standby_count_wanted 1 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:qdb_cluster leader: 34430 members: 34430,34434 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.lcfutm{0:34430} state up:active seq 17 join_fscid=1 addr [v2:172.21.15.31:6826/4118035133,v1:172.21.15.31:6827/4118035133] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi031.mjhxgi{1:34434} state up:active seq 12 join_fscid=1 addr [v2:172.21.15.31:6828/1583612894,v1:172.21.15.31:6829/1583612894] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:Standby daemons: 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.cagmus{-1:44353} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6824/257407185,v1:172.21.15.97:6825/257407185] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:09:02.808 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.cephfs.smithi097.sbwmfq{-1:44375} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.97:6826/2995504601,v1:172.21.15.97:6827/2995504601] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcdd407af30 msgr2=0x7fcdd407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcdd407af30 0x7fcdd407d3f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0004040 comp rx=0 tx=0).stop 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec100d10 msgr2=0x7fcdec19b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec100d10 0x7fcdec19b8b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00e970 tx=0x7fcddc00ed70 comp rx=0 tx=0).stop 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 shutdown_connections 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fcdd407af30 0x7fcdd407d3f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:02.809 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdec101630 0x7fcdec19bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 --2- 172.21.15.31:0/1429566645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdec100d10 0x7fcdec19b8b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 >> 172.21.15.31:0/1429566645 conn(0x7fcdec0758a0 msgr2=0x7fcdec101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 shutdown_connections 2024-09-16T09:09:02.810 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:02.806+0000 7fcdf15a9640 1 -- 172.21.15.31:0/1429566645 wait complete. 2024-09-16T09:09:03.403 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-09-16T09:09:03.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T09:09:03.625 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:03.772 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:03 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1429566645' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:09:04.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:03 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1429566645' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-16T09:09:04.762 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:04 smithi031 ceph-mon[91884]: pgmap v342: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:05.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:04 smithi097 ceph-mon[79892]: pgmap v342: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:05.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.449+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1032696329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 msgr2=0x7fd1e0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:05.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.449+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1032696329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e0109b00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0098b0 tx=0x7fd1cc02a4d0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.449+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1032696329 shutdown_connections 2024-09-16T09:09:05.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.449+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1032696329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e0109b00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.449+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1032696329 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0100c60 0x7fd1e0101040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.449+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1032696329 >> 172.21.15.31:0/1032696329 conn(0x7fd1e00fa730 msgr2=0x7fd1e00fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1032696329 shutdown_connections 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1032696329 wait complete. 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 Processor -- start 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 -- start start 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0100c60 0x7fd1e019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e019bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e019c4c0 con 0x7fd1e0101610 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1e6fbc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e0195a50 con 0x7fd1e0100c60 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e019bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:05.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1dffff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e019bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48448/0 (socket says 172.21.15.31:48448) 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.450+0000 7fd1dffff640 1 -- 172.21.15.31:0/1778711950 learned_addr learned my addr 172.21.15.31:0/1778711950 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1e4d58640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0100c60 0x7fd1e019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1dffff640 1 -- 172.21.15.31:0/1778711950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0100c60 msgr2=0x7fd1e019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1dffff640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0100c60 0x7fd1e019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1dffff640 1 -- 172.21.15.31:0/1778711950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc009560 con 0x7fd1e0101610 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1dffff640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e019bea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009230 tx=0x7fd1cc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1ddffb640 1 -- 172.21.15.31:0/1778711950 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1cc004aa0 con 0x7fd1e0101610 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e0195cd0 con 0x7fd1e0101610 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1ddffb640 1 -- 172.21.15.31:0/1778711950 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1cc004c00 con 0x7fd1e0101610 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1ddffb640 1 -- 172.21.15.31:0/1778711950 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1cc0039f0 con 0x7fd1e0101610 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.451+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e01961c0 con 0x7fd1e0101610 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.452+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e0068d30 con 0x7fd1e0101610 2024-09-16T09:09:05.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.453+0000 7fd1ddffb640 1 -- 172.21.15.31:0/1778711950 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fd1cc031c00 con 0x7fd1e0101610 2024-09-16T09:09:05.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.454+0000 7fd1ddffb640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1d007af30 0x7fd1d007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:05.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.454+0000 7fd1ddffb640 1 -- 172.21.15.31:0/1778711950 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fd1cc0bf3b0 con 0x7fd1e0101610 2024-09-16T09:09:05.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.454+0000 7fd1e4d58640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1d007af30 0x7fd1d007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:05.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.454+0000 7fd1e4d58640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1d007af30 0x7fd1d007d3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009890 tx=0x7fd1d4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:05.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.458+0000 7fd1ddffb640 1 -- 172.21.15.31:0/1778711950 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fd1cc082380 con 0x7fd1e0101610 2024-09-16T09:09:05.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.684+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1e01010c0 con 0x7fd1e0101610 2024-09-16T09:09:05.686 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.685+0000 7fd1ddffb640 1 -- 172.21.15.31:0/1778711950 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5269 (secure 0 0 0) 0x7fd1cc0854a0 con 0x7fd1e0101610 2024-09-16T09:09:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:05.687 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":38,"btime":"2024-09-16T09:05:32:642886+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44375,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6827/2995504601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2995504601},{"type":"v1","addr":"172.21.15.97:6827","nonce":2995504601}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:05:32.642884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":0,"incarnation":33,"state":"up:active","state_seq":17,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34434":{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":1,"incarnation":37,"state":"up:active","state_seq":12,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34430,"qdb_cluster":[34430,34434]},"id":1}]} 2024-09-16T09:09:05.687 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 38 2024-09-16T09:09:05.688 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1d007af30 msgr2=0x7fd1d007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1d007af30 0x7fd1d007d3f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd1d4009890 tx=0x7fd1d4008040 comp rx=0 tx=0).stop 2024-09-16T09:09:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 msgr2=0x7fd1e019bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e019bea0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009230 tx=0x7fd1cc0055a0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 shutdown_connections 2024-09-16T09:09:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1d007af30 0x7fd1d007d3f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1e0101610 0x7fd1e019bea0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.689 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 --2- 172.21.15.31:0/1778711950 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1e0100c60 0x7fd1e019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:05.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 >> 172.21.15.31:0/1778711950 conn(0x7fd1e00fa730 msgr2=0x7fd1e0101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:05.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 shutdown_connections 2024-09-16T09:09:05.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:05.687+0000 7fd1e6fbc640 1 -- 172.21.15.31:0/1778711950 wait complete. 2024-09-16T09:09:06.409 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-09-16T09:09:06.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 18 2024-09-16T09:09:06.621 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:06.742 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:06 smithi031 ceph-mon[91884]: pgmap v343: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:06.742 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:06 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1778711950' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-16T09:09:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:06 smithi097 ceph-mon[79892]: pgmap v343: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:07.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:06 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1778711950' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-16T09:09:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.221+0000 7fba6853a640 1 -- 172.21.15.31:0/2071251461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba60100750 msgr2=0x7fba601047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:08.237 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.221+0000 7fba6853a640 1 --2- 172.21.15.31:0/2071251461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba60100750 0x7fba601047a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fba50009880 tx=0x7fba5002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 -- 172.21.15.31:0/2071251461 shutdown_connections 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 --2- 172.21.15.31:0/2071251461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba60100750 0x7fba601047a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 --2- 172.21.15.31:0/2071251461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba600ffda0 0x7fba60100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 -- 172.21.15.31:0/2071251461 >> 172.21.15.31:0/2071251461 conn(0x7fba600fb840 msgr2=0x7fba600fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 -- 172.21.15.31:0/2071251461 shutdown_connections 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 -- 172.21.15.31:0/2071251461 wait complete. 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 Processor -- start 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 -- start start 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ffda0 0x7fba600727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba60100750 0x7fba60072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60073230 con 0x7fba600ffda0 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.222+0000 7fba6853a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60073370 con 0x7fba60100750 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba662d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ffda0 0x7fba600727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:08.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba662d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ffda0 0x7fba600727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48460/0 (socket says 172.21.15.31:48460) 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba662d6640 1 -- 172.21.15.31:0/2055923086 learned_addr learned my addr 172.21.15.31:0/2055923086 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba662d6640 1 -- 172.21.15.31:0/2055923086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba60100750 msgr2=0x7fba60072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba662d6640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba60100750 0x7fba60072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba662d6640 1 -- 172.21.15.31:0/2055923086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba50009560 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba662d6640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ffda0 0x7fba600727b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00d6f0 tx=0x7fba5c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba577fe640 1 -- 172.21.15.31:0/2055923086 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba5c009a30 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba577fe640 1 -- 172.21.15.31:0/2055923086 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba5c0054e0 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba577fe640 1 -- 172.21.15.31:0/2055923086 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba5c010460 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba601a1cd0 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.223+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba601a20e0 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.224+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba60101280 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.225+0000 7fba577fe640 1 -- 172.21.15.31:0/2055923086 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fba5c009b90 con 0x7fba600ffda0 2024-09-16T09:09:08.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.226+0000 7fba577fe640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba4c07af30 0x7fba4c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.226+0000 7fba577fe640 1 -- 172.21.15.31:0/2055923086 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fba5c09dc00 con 0x7fba600ffda0 2024-09-16T09:09:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.226+0000 7fba65ad5640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba4c07af30 0x7fba4c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.226+0000 7fba65ad5640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba4c07af30 0x7fba4c07d3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba600742b0 tx=0x7fba50000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:08.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.230+0000 7fba577fe640 1 -- 172.21.15.31:0/2055923086 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fba5c060c50 con 0x7fba600ffda0 2024-09-16T09:09:08.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.456+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7fba60100180 con 0x7fba600ffda0 2024-09-16T09:09:08.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.456+0000 7fba577fe640 1 -- 172.21.15.31:0/2055923086 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v38) v1 ==== 107+0+4920 (secure 0 0 0) 0x7fba5c017090 con 0x7fba600ffda0 2024-09-16T09:09:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:08.458 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":18,"btime":"2024-09-16T09:04:03:025751+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14560,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2549073061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2549073061},{"type":"v1","addr":"172.21.15.31:6827","nonce":2549073061}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:03.025748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14548,"mds_1":24325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14548":{"gid":14548,"name":"cephfs.smithi031.mjhxgi","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6829/1804644990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1804644990},{"type":"v1","addr":"172.21.15.31:6829","nonce":1804644990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24325":{"gid":24325,"name":"cephfs.smithi097.cagmus","rank":1,"incarnation":8,"state":"up:active","state_seq":4,"addr":"172.21.15.97:6825/17261087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":17261087},{"type":"v1","addr":"172.21.15.97:6825","nonce":17261087}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14548,"qdb_cluster":[14548,24325]},"id":1}]} 2024-09-16T09:09:08.458 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 18 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba4c07af30 msgr2=0x7fba4c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba4c07af30 0x7fba4c07d3f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fba600742b0 tx=0x7fba50000bc0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ffda0 msgr2=0x7fba600727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ffda0 0x7fba600727b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00d6f0 tx=0x7fba5c00dbb0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 shutdown_connections 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fba4c07af30 0x7fba4c07d3f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba60100750 0x7fba60072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.460 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 --2- 172.21.15.31:0/2055923086 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ffda0 0x7fba600727b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 >> 172.21.15.31:0/2055923086 conn(0x7fba600fb840 msgr2=0x7fba600fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 shutdown_connections 2024-09-16T09:09:08.461 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:08.459+0000 7fba6853a640 1 -- 172.21.15.31:0/2055923086 wait complete. 2024-09-16T09:09:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:08 smithi031 ceph-mon[91884]: pgmap v344: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:08.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:08 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2055923086' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 18, "format": "json"} : dispatch 2024-09-16T09:09:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:08 smithi097 ceph-mon[79892]: pgmap v344: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:09.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:08 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2055923086' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 18, "format": "json"} : dispatch 2024-09-16T09:09:09.071 DEBUG:tasks.fs:max_mds reduced in epoch 18 2024-09-16T09:09:09.072 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 19 2024-09-16T09:09:09.279 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:10.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:09 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:09 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:10.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:10 smithi031 ceph-mon[91884]: pgmap v345: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:11.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:10 smithi097 ceph-mon[79892]: pgmap v345: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:11.142 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/1138041274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 msgr2=0x7f4dcc109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/1138041274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc109b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009880 tx=0x7f4db402a530 comp rx=0 tx=0).stop 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/1138041274 shutdown_connections 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/1138041274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc109b60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/1138041274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dcc100d50 0x7f4dcc101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/1138041274 >> 172.21.15.31:0/1138041274 conn(0x7f4dcc0758a0 msgr2=0x7f4dcc0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/1138041274 shutdown_connections 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.101+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/1138041274 wait complete. 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd2fb9640 1 Processor -- start 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd2fb9640 1 -- start start 2024-09-16T09:09:11.143 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd2fb9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dcc100d50 0x7f4dcc0fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd2fb9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc0feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd2fb9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ff430 con 0x7f4dcc101670 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd2fb9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc0ff570 con 0x7f4dcc100d50 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc0feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc0feef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48468/0 (socket says 172.21.15.31:48468) 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dcbfff640 1 -- 172.21.15.31:0/606305833 learned_addr learned my addr 172.21.15.31:0/606305833 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd0d55640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dcc100d50 0x7f4dcc0fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dcbfff640 1 -- 172.21.15.31:0/606305833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dcc100d50 msgr2=0x7f4dcc0fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dcbfff640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dcc100d50 0x7f4dcc0fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dcbfff640 1 -- 172.21.15.31:0/606305833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db4009560 con 0x7f4dcc101670 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dd0d55640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dcc100d50 0x7f4dcc0fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.102+0000 7f4dcbfff640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc0feef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4db402aa50 tx=0x7f4db4032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:11.144 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.103+0000 7f4dc9ffb640 1 -- 172.21.15.31:0/606305833 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db40037c0 con 0x7f4dcc101670 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.103+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc0ff7f0 con 0x7f4dcc101670 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.103+0000 7f4dc9ffb640 1 -- 172.21.15.31:0/606305833 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4db4005d50 con 0x7f4dcc101670 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.103+0000 7f4dc9ffb640 1 -- 172.21.15.31:0/606305833 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4db403be30 con 0x7f4dcc101670 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.103+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc1a2090 con 0x7f4dcc101670 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.104+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc068470 con 0x7f4dcc101670 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.105+0000 7f4dc9ffb640 1 -- 172.21.15.31:0/606305833 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f4db402ad50 con 0x7f4dcc101670 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.105+0000 7f4dc9ffb640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4db807aee0 0x7f4db807d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:11.145 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.105+0000 7f4dc9ffb640 1 -- 172.21.15.31:0/606305833 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f4db40beca0 con 0x7f4dcc101670 2024-09-16T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.105+0000 7f4dd0d55640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4db807aee0 0x7f4db807d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:11.146 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.106+0000 7f4dd0d55640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4db807aee0 0x7f4db807d3a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009b40 tx=0x7f4dc4005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:11.147 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.110+0000 7f4dc9ffb640 1 -- 172.21.15.31:0/606305833 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f4db4081c70 con 0x7f4dcc101670 2024-09-16T09:09:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.331+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f4dcc101130 con 0x7f4dcc101670 2024-09-16T09:09:11.332 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.332+0000 7f4dc9ffb640 1 -- 172.21.15.31:0/606305833 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v38) v1 ==== 107+0+4916 (secure 0 0 0) 0x7f4db4084d90 con 0x7f4dcc101670 2024-09-16T09:09:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:11.333 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":19,"btime":"2024-09-16T09:04:03:030615+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14560,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2549073061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2549073061},{"type":"v1","addr":"172.21.15.31:6827","nonce":2549073061}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:03.030612+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0,1],"up":{"mds_0":14548,"mds_1":24325},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14548":{"gid":14548,"name":"cephfs.smithi031.mjhxgi","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6829/1804644990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1804644990},{"type":"v1","addr":"172.21.15.31:6829","nonce":1804644990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24325":{"gid":24325,"name":"cephfs.smithi097.cagmus","rank":1,"incarnation":8,"state":"up:stopping","state_seq":4,"addr":"172.21.15.97:6825/17261087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":17261087},{"type":"v1","addr":"172.21.15.97:6825","nonce":17261087}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14548,"qdb_cluster":[14548]},"id":1}]} 2024-09-16T09:09:11.333 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 19 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4db807aee0 msgr2=0x7f4db807d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4db807aee0 0x7f4db807d3a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009b40 tx=0x7f4dc4005e50 comp rx=0 tx=0).stop 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 msgr2=0x7f4dcc0feef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc0feef0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4db402aa50 tx=0x7f4db4032bb0 comp rx=0 tx=0).stop 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 shutdown_connections 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f4db807aee0 0x7f4db807d3a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4dcc101670 0x7f4dcc0feef0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 --2- 172.21.15.31:0/606305833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4dcc100d50 0x7f4dcc0fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:11.335 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 >> 172.21.15.31:0/606305833 conn(0x7f4dcc0758a0 msgr2=0x7f4dcc101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 shutdown_connections 2024-09-16T09:09:11.336 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:11.334+0000 7f4dd2fb9640 1 -- 172.21.15.31:0/606305833 wait complete. 2024-09-16T09:09:11.953 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:11 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/606305833' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 19, "format": "json"} : dispatch 2024-09-16T09:09:11.955 DEBUG:tasks.fs:max_mds reduced in epoch 19 2024-09-16T09:09:11.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 20 2024-09-16T09:09:12.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:11 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/606305833' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 19, "format": "json"} : dispatch 2024-09-16T09:09:12.168 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:12.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:12 smithi031 ceph-mon[91884]: pgmap v346: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:13.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:12 smithi097 ceph-mon[79892]: pgmap v346: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:13.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.689+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1745247694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a00ffda0 msgr2=0x7fd3a0100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:13.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.689+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1745247694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a00ffda0 0x7fd3a0100180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd390009c20 tx=0x7fd39002a970 comp rx=0 tx=0).stop 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.689+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1745247694 shutdown_connections 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.689+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1745247694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a0100750 0x7fd3a01047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.689+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1745247694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a00ffda0 0x7fd3a0100180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.689+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1745247694 >> 172.21.15.31:0/1745247694 conn(0x7fd3a00fb840 msgr2=0x7fd3a00fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1745247694 shutdown_connections 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1745247694 wait complete. 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 Processor -- start 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 -- start start 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a00ffda0 0x7fd3a019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:13.715 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0100750 0x7fd3a019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0197b40 con 0x7fd3a0100750 2024-09-16T09:09:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a7b4c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0197cb0 con 0x7fd3a00ffda0 2024-09-16T09:09:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.690+0000 7fd3a50e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0100750 0x7fd3a019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a50e7640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0100750 0x7fd3a019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46106/0 (socket says 172.21.15.31:46106) 2024-09-16T09:09:13.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a50e7640 1 -- 172.21.15.31:0/1864429828 learned_addr learned my addr 172.21.15.31:0/1864429828 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a58e8640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a00ffda0 0x7fd3a019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a50e7640 1 -- 172.21.15.31:0/1864429828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a00ffda0 msgr2=0x7fd3a019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a50e7640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a00ffda0 0x7fd3a019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a50e7640 1 -- 172.21.15.31:0/1864429828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390009900 con 0x7fd3a0100750 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a50e7640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0100750 0x7fd3a019df90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d6f0 tx=0x7fd39400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd39effd640 1 -- 172.21.15.31:0/1864429828 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd394009a30 con 0x7fd3a0100750 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd39effd640 1 -- 172.21.15.31:0/1864429828 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3940054e0 con 0x7fd3a0100750 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a0197f90 con 0x7fd3a0100750 2024-09-16T09:09:13.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd39effd640 1 -- 172.21.15.31:0/1864429828 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd394010460 con 0x7fd3a0100750 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.691+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a01984e0 con 0x7fd3a0100750 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.692+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a0101280 con 0x7fd3a0100750 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.693+0000 7fd39effd640 1 -- 172.21.15.31:0/1864429828 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fd394009b90 con 0x7fd3a0100750 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.693+0000 7fd39effd640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd38807af30 0x7fd38807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.694+0000 7fd39effd640 1 -- 172.21.15.31:0/1864429828 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fd39409db60 con 0x7fd3a0100750 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.694+0000 7fd3a58e8640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd38807af30 0x7fd38807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.694+0000 7fd3a58e8640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd38807af30 0x7fd38807d3f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd390000f80 tx=0x7fd39003d040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:13.718 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.698+0000 7fd39effd640 1 -- 172.21.15.31:0/1864429828 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fd394060c80 con 0x7fd3a0100750 2024-09-16T09:09:13.920 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.919+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7fd3a0198fe0 con 0x7fd3a0100750 2024-09-16T09:09:13.921 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.920+0000 7fd39effd640 1 -- 172.21.15.31:0/1864429828 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v38) v1 ==== 107+0+4127 (secure 0 0 0) 0x7fd394063da0 con 0x7fd3a0100750 2024-09-16T09:09:13.921 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:13.921 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":20,"btime":"2024-09-16T09:04:20:381391+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14560,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2549073061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2549073061},{"type":"v1","addr":"172.21.15.31:6827","nonce":2549073061}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:19.579464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14548},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14548":{"gid":14548,"name":"cephfs.smithi031.mjhxgi","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6829/1804644990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1804644990},{"type":"v1","addr":"172.21.15.31:6829","nonce":1804644990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14548,"qdb_cluster":[14548]},"id":1}]} 2024-09-16T09:09:13.922 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 20 2024-09-16T09:09:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd38807af30 msgr2=0x7fd38807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd38807af30 0x7fd38807d3f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd390000f80 tx=0x7fd39003d040 comp rx=0 tx=0).stop 2024-09-16T09:09:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0100750 msgr2=0x7fd3a019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:13.923 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0100750 0x7fd3a019df90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd39400d6f0 tx=0x7fd39400dbb0 comp rx=0 tx=0).stop 2024-09-16T09:09:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 shutdown_connections 2024-09-16T09:09:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd38807af30 0x7fd38807d3f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd3a0100750 0x7fd3a019df90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 --2- 172.21.15.31:0/1864429828 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd3a00ffda0 0x7fd3a019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 >> 172.21.15.31:0/1864429828 conn(0x7fd3a00fb840 msgr2=0x7fd3a00fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 shutdown_connections 2024-09-16T09:09:13.924 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:13.922+0000 7fd3a7b4c640 1 -- 172.21.15.31:0/1864429828 wait complete. 2024-09-16T09:09:14.611 DEBUG:tasks.fs:max_mds reduced in epoch 20 2024-09-16T09:09:14.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 21 2024-09-16T09:09:14.827 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:14.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:14 smithi031 ceph-mon[91884]: pgmap v347: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:14.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:14 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1864429828' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-16T09:09:15.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:14 smithi097 ceph-mon[79892]: pgmap v347: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:15.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:14 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1864429828' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 20, "format": "json"} : dispatch 2024-09-16T09:09:16.023 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:15 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:09:16.056 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:15 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:09:16.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 -- 172.21.15.31:0/327682800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 msgr2=0x7f8e10109b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 --2- 172.21.15.31:0/327682800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e10109b20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8e000098b0 tx=0x7f8e0002a560 comp rx=0 tx=0).stop 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 -- 172.21.15.31:0/327682800 shutdown_connections 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 --2- 172.21.15.31:0/327682800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e10109b20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 --2- 172.21.15.31:0/327682800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e10100d10 0x7f8e101010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 -- 172.21.15.31:0/327682800 >> 172.21.15.31:0/327682800 conn(0x7f8e100758a0 msgr2=0x7f8e100fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 -- 172.21.15.31:0/327682800 shutdown_connections 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.727+0000 7f8e16069640 1 -- 172.21.15.31:0/327682800 wait complete. 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e16069640 1 Processor -- start 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e16069640 1 -- start start 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e16069640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e10100d10 0x7f8e1019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e16069640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e1019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e16069640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1019c410 con 0x7f8e10101630 2024-09-16T09:09:16.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e16069640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e101959a0 con 0x7f8e10100d10 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e1019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e1019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46116/0 (socket says 172.21.15.31:46116) 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e0effd640 1 -- 172.21.15.31:0/1114637640 learned_addr learned my addr 172.21.15.31:0/1114637640 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.728+0000 7f8e0f7fe640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e10100d10 0x7f8e1019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e0effd640 1 -- 172.21.15.31:0/1114637640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e10100d10 msgr2=0x7f8e1019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e0effd640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e10100d10 0x7f8e1019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e0effd640 1 -- 172.21.15.31:0/1114637640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e00009560 con 0x7f8e10101630 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e0effd640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e1019bdf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009230 tx=0x7f8e000055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e0cff9640 1 -- 172.21.15.31:0/1114637640 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e00004aa0 con 0x7f8e10101630 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e0cff9640 1 -- 172.21.15.31:0/1114637640 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e00031470 con 0x7f8e10101630 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e10195c20 con 0x7f8e10101630 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e0cff9640 1 -- 172.21.15.31:0/1114637640 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e000039e0 con 0x7f8e10101630 2024-09-16T09:09:16.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.729+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e10196110 con 0x7f8e10101630 2024-09-16T09:09:16.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.730+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e100684b0 con 0x7f8e10101630 2024-09-16T09:09:16.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.731+0000 7f8e0cff9640 1 -- 172.21.15.31:0/1114637640 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f8e00031c00 con 0x7f8e10101630 2024-09-16T09:09:16.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.731+0000 7f8e0cff9640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8dfc07af30 0x7f8dfc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:16.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.732+0000 7f8e0cff9640 1 -- 172.21.15.31:0/1114637640 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f8e000beff0 con 0x7f8e10101630 2024-09-16T09:09:16.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.732+0000 7f8e0f7fe640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8dfc07af30 0x7f8dfc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:16.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.732+0000 7f8e0f7fe640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8dfc07af30 0x7f8dfc07d3f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005fd0 tx=0x7f8df800b390 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:16.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.736+0000 7f8e0cff9640 1 -- 172.21.15.31:0/1114637640 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f8e00082190 con 0x7f8e10101630 2024-09-16T09:09:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.961+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f8e101043c0 con 0x7f8e10101630 2024-09-16T09:09:16.964 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:16 smithi031 ceph-mon[91884]: pgmap v348: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:16.964 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:09:16.964 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:09:16.964 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:16 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:09:16.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.961+0000 7f8e0cff9640 1 -- 172.21.15.31:0/1114637640 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v38) v1 ==== 107+0+4906 (secure 0 0 0) 0x7f8e000852b0 con 0x7f8e10101630 2024-09-16T09:09:16.965 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:16.965 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":21,"btime":"2024-09-16T09:04:21:389245+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14560,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/2549073061","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":2549073061},{"type":"v1","addr":"172.21.15.31:6827","nonce":2549073061}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:19.579464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14548},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14548":{"gid":14548,"name":"cephfs.smithi031.mjhxgi","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6829/1804644990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1804644990},{"type":"v1","addr":"172.21.15.31:6829","nonce":1804644990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14548,"qdb_cluster":[14548]},"id":1}]} 2024-09-16T09:09:16.965 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 21 2024-09-16T09:09:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8dfc07af30 msgr2=0x7f8dfc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8dfc07af30 0x7f8dfc07d3f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8df8005fd0 tx=0x7f8df800b390 comp rx=0 tx=0).stop 2024-09-16T09:09:16.967 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 msgr2=0x7f8e1019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e1019bdf0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009230 tx=0x7f8e000055a0 comp rx=0 tx=0).stop 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 shutdown_connections 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8dfc07af30 0x7f8dfc07d3f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e10101630 0x7f8e1019bdf0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 --2- 172.21.15.31:0/1114637640 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8e10100d10 0x7f8e1019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 >> 172.21.15.31:0/1114637640 conn(0x7f8e100758a0 msgr2=0x7f8e10101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 shutdown_connections 2024-09-16T09:09:16.968 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:16.964+0000 7f8e16069640 1 -- 172.21.15.31:0/1114637640 wait complete. 2024-09-16T09:09:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:16 smithi097 ceph-mon[79892]: pgmap v348: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:09:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:09:17.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:16 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' 2024-09-16T09:09:17.600 DEBUG:tasks.fs:max_mds reduced in epoch 21 2024-09-16T09:09:17.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 22 2024-09-16T09:09:17.814 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:17.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:17 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1114637640' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-16T09:09:18.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:17 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1114637640' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 21, "format": "json"} : dispatch 2024-09-16T09:09:18.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:18 smithi031 ceph-mon[91884]: pgmap v349: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:19.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:18 smithi097 ceph-mon[79892]: pgmap v349: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.441+0000 7f613f874640 1 -- 172.21.15.31:0/419330568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 msgr2=0x7f61380fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.441+0000 7f613f874640 1 --2- 172.21.15.31:0/419330568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380fee90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6128009880 tx=0x7f612802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.441+0000 7f613f874640 1 -- 172.21.15.31:0/419330568 shutdown_connections 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.441+0000 7f613f874640 1 --2- 172.21.15.31:0/419330568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380fee90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.441+0000 7f613f874640 1 --2- 172.21.15.31:0/419330568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6138104a30 0x7f6138104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.442+0000 7f613f874640 1 -- 172.21.15.31:0/419330568 >> 172.21.15.31:0/419330568 conn(0x7f61380fa730 msgr2=0x7f61380fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.442+0000 7f613f874640 1 -- 172.21.15.31:0/419330568 shutdown_connections 2024-09-16T09:09:19.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.442+0000 7f613f874640 1 -- 172.21.15.31:0/419330568 wait complete. 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.442+0000 7f613f874640 1 Processor -- start 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.442+0000 7f613f874640 1 -- start start 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.442+0000 7f613f874640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380faff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.442+0000 7f613f874640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6138104a30 0x7f61380fb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613f874640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380fba70 con 0x7f61380fea30 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613f874640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61380fbbe0 con 0x7f6138104a30 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613d610640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613d610640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380faff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:46140/0 (socket says 172.21.15.31:46140) 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613d610640 1 -- 172.21.15.31:0/2971715949 learned_addr learned my addr 172.21.15.31:0/2971715949 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613ce0f640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6138104a30 0x7f61380fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:19.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613d610640 1 -- 172.21.15.31:0/2971715949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6138104a30 msgr2=0x7f61380fb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613d610640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6138104a30 0x7f61380fb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613d610640 1 -- 172.21.15.31:0/2971715949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6128009560 con 0x7f61380fea30 2024-09-16T09:09:19.447 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613d610640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380faff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f613400d6f0 tx=0x7f613400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f612e7fc640 1 -- 172.21.15.31:0/2971715949 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6134009a30 con 0x7f61380fea30 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f612e7fc640 1 -- 172.21.15.31:0/2971715949 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61340054e0 con 0x7f61380fea30 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61381a1cc0 con 0x7f61380fea30 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.443+0000 7f612e7fc640 1 -- 172.21.15.31:0/2971715949 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6134010460 con 0x7f61380fea30 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.444+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61381a2020 con 0x7f61380fea30 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.445+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6138100170 con 0x7f61380fea30 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.445+0000 7f612e7fc640 1 -- 172.21.15.31:0/2971715949 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f6134009b90 con 0x7f61380fea30 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.446+0000 7f612e7fc640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f612407af30 0x7f612407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:19.448 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.446+0000 7f612e7fc640 1 -- 172.21.15.31:0/2971715949 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f613409dc00 con 0x7f61380fea30 2024-09-16T09:09:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.446+0000 7f613ce0f640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f612407af30 0x7f612407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:19.449 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.446+0000 7f613ce0f640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f612407af30 0x7f612407d3f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f6128005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:19.452 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.450+0000 7f612e7fc640 1 -- 172.21.15.31:0/2971715949 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6134060c50 con 0x7f61380fea30 2024-09-16T09:09:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.671+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f61380fc550 con 0x7f61380fea30 2024-09-16T09:09:19.674 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.672+0000 7f612e7fc640 1 -- 172.21.15.31:0/2971715949 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v38) v1 ==== 107+0+4127 (secure 0 0 0) 0x7f6134017090 con 0x7f61380fea30 2024-09-16T09:09:19.674 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:19.674 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":22,"btime":"2024-09-16T09:04:27:405854+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:19.579464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14548},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14548":{"gid":14548,"name":"cephfs.smithi031.mjhxgi","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6829/1804644990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1804644990},{"type":"v1","addr":"172.21.15.31:6829","nonce":1804644990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14548,"qdb_cluster":[14548]},"id":1}]} 2024-09-16T09:09:19.675 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 22 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.674+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f612407af30 msgr2=0x7f612407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.674+0000 7f613f874640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f612407af30 0x7f612407d3f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6128000c00 tx=0x7f6128005a20 comp rx=0 tx=0).stop 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.674+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 msgr2=0x7f61380faff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.674+0000 7f613f874640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380faff0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f613400d6f0 tx=0x7f613400dbb0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.674+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 shutdown_connections 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.674+0000 7f613f874640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f612407af30 0x7f612407d3f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.675+0000 7f613f874640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6138104a30 0x7f61380fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.677 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.675+0000 7f613f874640 1 --2- 172.21.15.31:0/2971715949 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61380fea30 0x7f61380faff0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.675+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 >> 172.21.15.31:0/2971715949 conn(0x7f61380fa730 msgr2=0x7f6138068c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.675+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 shutdown_connections 2024-09-16T09:09:19.678 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:19.675+0000 7f613f874640 1 -- 172.21.15.31:0/2971715949 wait complete. 2024-09-16T09:09:19.991 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:19 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2971715949' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-16T09:09:20.048 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:19 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2971715949' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 22, "format": "json"} : dispatch 2024-09-16T09:09:20.413 DEBUG:tasks.fs:max_mds reduced in epoch 22 2024-09-16T09:09:20.413 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 23 2024-09-16T09:09:20.623 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:21.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:20 smithi097 ceph-mon[79892]: pgmap v350: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:21.032 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:20 smithi031 ceph-mon[91884]: pgmap v350: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:22.317 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 -- 172.21.15.31:0/392740402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c100750 msgr2=0x7fd19c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/392740402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c100750 0x7fd19c1047a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd184009880 tx=0x7fd18402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 -- 172.21.15.31:0/392740402 shutdown_connections 2024-09-16T09:09:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/392740402 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c100750 0x7fd19c1047a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/392740402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c0ffda0 0x7fd19c100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 -- 172.21.15.31:0/392740402 >> 172.21.15.31:0/392740402 conn(0x7fd19c0fb840 msgr2=0x7fd19c0fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:22.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 -- 172.21.15.31:0/392740402 shutdown_connections 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.313+0000 7fd1a14ca640 1 -- 172.21.15.31:0/392740402 wait complete. 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd1a14ca640 1 Processor -- start 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd1a14ca640 1 -- start start 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd1a14ca640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0ffda0 0x7fd19c19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd1a14ca640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c100750 0x7fd19c19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd1a14ca640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c19c2f0 con 0x7fd19c0ffda0 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd1a14ca640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c195910 con 0x7fd19c100750 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd19affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0ffda0 0x7fd19c19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:22.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd19affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0ffda0 0x7fd19c19b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34570/0 (socket says 172.21.15.31:34570) 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd19affd640 1 -- 172.21.15.31:0/3084137961 learned_addr learned my addr 172.21.15.31:0/3084137961 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd193fff640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c100750 0x7fd19c19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd19affd640 1 -- 172.21.15.31:0/3084137961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c100750 msgr2=0x7fd19c19bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd19affd640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c100750 0x7fd19c19bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd19affd640 1 -- 172.21.15.31:0/3084137961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184009560 con 0x7fd19c0ffda0 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd193fff640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c100750 0x7fd19c19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.314+0000 7fd19affd640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0ffda0 0x7fd19c19b820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00c8a0 tx=0x7fd18c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.315+0000 7fd198ff9640 1 -- 172.21.15.31:0/3084137961 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18c007b80 con 0x7fd19c0ffda0 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.315+0000 7fd198ff9640 1 -- 172.21.15.31:0/3084137961 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd18c007ce0 con 0x7fd19c0ffda0 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.315+0000 7fd198ff9640 1 -- 172.21.15.31:0/3084137961 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd18c018810 con 0x7fd19c0ffda0 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.315+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c195bf0 con 0x7fd19c0ffda0 2024-09-16T09:09:22.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.315+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c196140 con 0x7fd19c0ffda0 2024-09-16T09:09:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.316+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c101280 con 0x7fd19c0ffda0 2024-09-16T09:09:22.321 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.316+0000 7fd198ff9640 1 -- 172.21.15.31:0/3084137961 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fd18c018970 con 0x7fd19c0ffda0 2024-09-16T09:09:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.317+0000 7fd198ff9640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd18807af30 0x7fd18807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.317+0000 7fd198ff9640 1 -- 172.21.15.31:0/3084137961 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fd18c09e1e0 con 0x7fd19c0ffda0 2024-09-16T09:09:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.317+0000 7fd193fff640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd18807af30 0x7fd18807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:22.322 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.318+0000 7fd193fff640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd18807af30 0x7fd18807d3f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd184006010 tx=0x7fd184005e60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:22.324 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.322+0000 7fd198ff9640 1 -- 172.21.15.31:0/3084137961 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fd18c0612d0 con 0x7fd19c0ffda0 2024-09-16T09:09:22.545 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.543+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fd19c100180 con 0x7fd19c0ffda0 2024-09-16T09:09:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.543+0000 7fd198ff9640 1 -- 172.21.15.31:0/3084137961 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v38) v1 ==== 107+0+4974 (secure 0 0 0) 0x7fd18c0643f0 con 0x7fd19c0ffda0 2024-09-16T09:09:22.546 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:22.546 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":23,"btime":"2024-09-16T09:04:31:411092+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:19.579464+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14548},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_14548":{"gid":14548,"name":"cephfs.smithi031.mjhxgi","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.31:6829/1804644990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1804644990},{"type":"v1","addr":"172.21.15.31:6829","nonce":1804644990}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14548,"qdb_cluster":[14548]},"id":1}]} 2024-09-16T09:09:22.546 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 23 2024-09-16T09:09:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd18807af30 msgr2=0x7fd18807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd18807af30 0x7fd18807d3f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd184006010 tx=0x7fd184005e60 comp rx=0 tx=0).stop 2024-09-16T09:09:22.547 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0ffda0 msgr2=0x7fd19c19b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0ffda0 0x7fd19c19b820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00c8a0 tx=0x7fd18c00cd60 comp rx=0 tx=0).stop 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 shutdown_connections 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd18807af30 0x7fd18807d3f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd19c100750 0x7fd19c19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 --2- 172.21.15.31:0/3084137961 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd19c0ffda0 0x7fd19c19b820 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 >> 172.21.15.31:0/3084137961 conn(0x7fd19c0fb840 msgr2=0x7fd19c0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 shutdown_connections 2024-09-16T09:09:22.548 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:22.545+0000 7fd1a14ca640 1 -- 172.21.15.31:0/3084137961 wait complete. 2024-09-16T09:09:23.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:22 smithi097 ceph-mon[79892]: pgmap v351: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:23.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:22 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3084137961' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-16T09:09:23.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:22 smithi031 ceph-mon[91884]: pgmap v351: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:23.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:22 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3084137961' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 23, "format": "json"} : dispatch 2024-09-16T09:09:23.185 DEBUG:tasks.fs:max_mds reduced in epoch 23 2024-09-16T09:09:23.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 24 2024-09-16T09:09:23.399 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:24.822 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:24 smithi031 ceph-mon[91884]: pgmap v352: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:24.822 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:24 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:24 smithi097 ceph-mon[79892]: pgmap v352: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:25.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:24 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 -- 172.21.15.31:0/3237837615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 msgr2=0x7fde080fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/3237837615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde080fee50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009880 tx=0x7fddf802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 -- 172.21.15.31:0/3237837615 shutdown_connections 2024-09-16T09:09:25.166 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/3237837615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde080fee50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/3237837615 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde081049f0 0x7fde08104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 -- 172.21.15.31:0/3237837615 >> 172.21.15.31:0/3237837615 conn(0x7fde080758a0 msgr2=0x7fde080fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 -- 172.21.15.31:0/3237837615 shutdown_connections 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.147+0000 7fde0f0f9640 1 -- 172.21.15.31:0/3237837615 wait complete. 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0f0f9640 1 Processor -- start 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0f0f9640 1 -- start start 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0f0f9640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde08196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0f0f9640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde081049f0 0x7fde08196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0f0f9640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08197490 con 0x7fde080fe9f0 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0f0f9640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08197600 con 0x7fde081049f0 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0ce95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde08196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0ce95640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde08196980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34584/0 (socket says 172.21.15.31:34584) 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0ce95640 1 -- 172.21.15.31:0/732824931 learned_addr learned my addr 172.21.15.31:0/732824931 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:25.167 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fddfffff640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde081049f0 0x7fde08196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0ce95640 1 -- 172.21.15.31:0/732824931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde081049f0 msgr2=0x7fde08196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0ce95640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde081049f0 0x7fde08196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0ce95640 1 -- 172.21.15.31:0/732824931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf8009560 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fde0ce95640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde08196980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fde0400d6f0 tx=0x7fde0400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.148+0000 7fddfdffb640 1 -- 172.21.15.31:0/732824931 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde04009a30 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.149+0000 7fddfdffb640 1 -- 172.21.15.31:0/732824931 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde040054e0 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.149+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0819f9f0 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.149+0000 7fddfdffb640 1 -- 172.21.15.31:0/732824931 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde04010460 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.149+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0819fe00 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.150+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde08100130 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.150+0000 7fddfdffb640 1 -- 172.21.15.31:0/732824931 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fde04009b90 con 0x7fde080fe9f0 2024-09-16T09:09:25.168 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.151+0000 7fddfdffb640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf007af30 0x7fddf007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.151+0000 7fddfdffb640 1 -- 172.21.15.31:0/732824931 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fde0409dc10 con 0x7fde080fe9f0 2024-09-16T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.151+0000 7fddfffff640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf007af30 0x7fddf007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.152+0000 7fddfffff640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf007af30 0x7fddf007d3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddf8000c00 tx=0x7fddf8005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:25.169 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.155+0000 7fddfdffb640 1 -- 172.21.15.31:0/732824931 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fde04060c80 con 0x7fde080fe9f0 2024-09-16T09:09:25.378 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.376+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fde080682c0 con 0x7fde080fe9f0 2024-09-16T09:09:25.379 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.377+0000 7fddfdffb640 1 -- 172.21.15.31:0/732824931 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v38) v1 ==== 107+0+4175 (secure 0 0 0) 0x7fde04063da0 con 0x7fde080fe9f0 2024-09-16T09:09:25.379 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:25.379 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":24,"btime":"2024-09-16T09:04:42:710539+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:42.710536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:25.380 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 24 2024-09-16T09:09:25.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf007af30 msgr2=0x7fddf007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:25.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf007af30 0x7fddf007d3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fddf8000c00 tx=0x7fddf8005e30 comp rx=0 tx=0).stop 2024-09-16T09:09:25.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 msgr2=0x7fde08196980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:25.381 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde08196980 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fde0400d6f0 tx=0x7fde0400dbb0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 shutdown_connections 2024-09-16T09:09:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fddf007af30 0x7fddf007d3f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fde081049f0 0x7fde08196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 --2- 172.21.15.31:0/732824931 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde080fe9f0 0x7fde08196980 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 >> 172.21.15.31:0/732824931 conn(0x7fde080758a0 msgr2=0x7fde080fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 shutdown_connections 2024-09-16T09:09:25.382 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:25.379+0000 7fde0f0f9640 1 -- 172.21.15.31:0/732824931 wait complete. 2024-09-16T09:09:26.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:25 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/732824931' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-16T09:09:26.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:25 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/732824931' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 24, "format": "json"} : dispatch 2024-09-16T09:09:26.182 DEBUG:tasks.fs:max_mds reduced in epoch 24 2024-09-16T09:09:26.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 25 2024-09-16T09:09:26.420 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:26.817 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:26 smithi031 ceph-mon[91884]: pgmap v353: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:27.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:26 smithi097 ceph-mon[79892]: pgmap v353: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 -- 172.21.15.31:0/4082329252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e8105be0 msgr2=0x7f36e8105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 --2- 172.21.15.31:0/4082329252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e8105be0 0x7f36e8105fc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f36d8009c20 tx=0x7f36d802a970 comp rx=0 tx=0).stop 2024-09-16T09:09:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 -- 172.21.15.31:0/4082329252 shutdown_connections 2024-09-16T09:09:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 --2- 172.21.15.31:0/4082329252 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e80ffb30 0x7f36e80fff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 --2- 172.21.15.31:0/4082329252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e8105be0 0x7f36e8105fc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:28.073 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 -- 172.21.15.31:0/4082329252 >> 172.21.15.31:0/4082329252 conn(0x7f36e80fb810 msgr2=0x7f36e80fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 -- 172.21.15.31:0/4082329252 shutdown_connections 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.065+0000 7f36ecc27640 1 -- 172.21.15.31:0/4082329252 wait complete. 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36ecc27640 1 Processor -- start 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36ecc27640 1 -- start start 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36ecc27640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e80ffb30 0x7f36e80727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36ecc27640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e8105be0 0x7f36e8072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36ecc27640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8073230 con 0x7f36e80ffb30 2024-09-16T09:09:28.074 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36ecc27640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e8073370 con 0x7f36e8105be0 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36e659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e80ffb30 0x7f36e80727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36e659c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e80ffb30 0x7f36e80727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34592/0 (socket says 172.21.15.31:34592) 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36e659c640 1 -- 172.21.15.31:0/3643002274 learned_addr learned my addr 172.21.15.31:0/3643002274 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36e659c640 1 -- 172.21.15.31:0/3643002274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e8105be0 msgr2=0x7f36e8072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36e659c640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e8105be0 0x7f36e8072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.066+0000 7f36e659c640 1 -- 172.21.15.31:0/3643002274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d8009900 con 0x7f36e80ffb30 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.067+0000 7f36e659c640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e80ffb30 0x7f36e80727b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f36d802ae90 tx=0x7f36d80308a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.067+0000 7f36df7fe640 1 -- 172.21.15.31:0/3643002274 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36d802f590 con 0x7f36e80ffb30 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.067+0000 7f36df7fe640 1 -- 172.21.15.31:0/3643002274 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36d8030ca0 con 0x7f36e80ffb30 2024-09-16T09:09:28.075 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.067+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e80735f0 con 0x7f36e80ffb30 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.067+0000 7f36df7fe640 1 -- 172.21.15.31:0/3643002274 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36d8039680 con 0x7f36e80ffb30 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.067+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e81a2110 con 0x7f36e80ffb30 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.068+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e81011e0 con 0x7f36e80ffb30 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.069+0000 7f36df7fe640 1 -- 172.21.15.31:0/3643002274 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f36d800a040 con 0x7f36e80ffb30 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.069+0000 7f36df7fe640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f36d407af30 0x7f36d407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.069+0000 7f36df7fe640 1 -- 172.21.15.31:0/3643002274 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f36d802c040 con 0x7f36e80ffb30 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.069+0000 7f36e5d9b640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f36d407af30 0x7f36d407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:28.076 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.070+0000 7f36e5d9b640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f36d407af30 0x7f36d407d3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36e80742b0 tx=0x7f36d0006c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:28.077 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.074+0000 7f36df7fe640 1 -- 172.21.15.31:0/3643002274 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f36d8082520 con 0x7f36e80ffb30 2024-09-16T09:09:28.297 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.296+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f36e8100010 con 0x7f36e80ffb30 2024-09-16T09:09:28.298 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.297+0000 7f36df7fe640 1 -- 172.21.15.31:0/3643002274 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v38) v1 ==== 107+0+4186 (secure 0 0 0) 0x7f36d8085640 con 0x7f36e80ffb30 2024-09-16T09:09:28.298 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:28.298 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":25,"btime":"2024-09-16T09:04:42:715988+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:42.715976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24337},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24337":{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":0,"incarnation":25,"state":"up:replay","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:28.299 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 25 2024-09-16T09:09:28.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f36d407af30 msgr2=0x7f36d407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:28.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f36d407af30 0x7f36d407d3f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f36e80742b0 tx=0x7f36d0006c90 comp rx=0 tx=0).stop 2024-09-16T09:09:28.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e80ffb30 msgr2=0x7f36e80727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:28.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e80ffb30 0x7f36e80727b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f36d802ae90 tx=0x7f36d80308a0 comp rx=0 tx=0).stop 2024-09-16T09:09:28.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 shutdown_connections 2024-09-16T09:09:28.300 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f36d407af30 0x7f36d407d3f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:28.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36e8105be0 0x7f36e8072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:28.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 --2- 172.21.15.31:0/3643002274 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f36e80ffb30 0x7f36e80727b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:28.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 >> 172.21.15.31:0/3643002274 conn(0x7f36e80fb810 msgr2=0x7f36e80fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:28.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 shutdown_connections 2024-09-16T09:09:28.301 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:28.299+0000 7f36ecc27640 1 -- 172.21.15.31:0/3643002274 wait complete. 2024-09-16T09:09:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:28 smithi031 ceph-mon[91884]: pgmap v354: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:28.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:28 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3643002274' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-16T09:09:29.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:28 smithi097 ceph-mon[79892]: pgmap v354: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:29.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:28 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3643002274' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 25, "format": "json"} : dispatch 2024-09-16T09:09:29.036 DEBUG:tasks.fs:max_mds reduced in epoch 25 2024-09-16T09:09:29.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 26 2024-09-16T09:09:29.249 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 -- 172.21.15.31:0/999027198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658101690 msgr2=0x7f8658109b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 --2- 172.21.15.31:0/999027198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658101690 0x7f8658109b80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8640009880 tx=0x7f864002a530 comp rx=0 tx=0).stop 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 -- 172.21.15.31:0/999027198 shutdown_connections 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 --2- 172.21.15.31:0/999027198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658101690 0x7f8658109b80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 --2- 172.21.15.31:0/999027198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8658100d70 0x7f8658101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 -- 172.21.15.31:0/999027198 >> 172.21.15.31:0/999027198 conn(0x7f86580fa730 msgr2=0x7f86580fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 -- 172.21.15.31:0/999027198 shutdown_connections 2024-09-16T09:09:30.947 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.922+0000 7f865d4b8640 1 -- 172.21.15.31:0/999027198 wait complete. 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f865d4b8640 1 Processor -- start 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f865d4b8640 1 -- start start 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f865d4b8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658100d70 0x7f86580fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f865d4b8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8658101690 0x7f86580fef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f865d4b8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86580ff450 con 0x7f8658100d70 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f865d4b8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86580ff5c0 con 0x7f8658101690 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f8656ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658100d70 0x7f86580fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f8656ffd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658100d70 0x7f86580fe9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:34606/0 (socket says 172.21.15.31:34606) 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f8656ffd640 1 -- 172.21.15.31:0/3301013091 learned_addr learned my addr 172.21.15.31:0/3301013091 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f86567fc640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8658101690 0x7f86580fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f8656ffd640 1 -- 172.21.15.31:0/3301013091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8658101690 msgr2=0x7f86580fef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:30.948 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f8656ffd640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8658101690 0x7f86580fef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.923+0000 7f8656ffd640 1 -- 172.21.15.31:0/3301013091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640009560 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.924+0000 7f8656ffd640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658100d70 0x7f86580fe9d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f865000b420 tx=0x7f865000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.924+0000 7f864ffff640 1 -- 172.21.15.31:0/3301013091 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8650011800 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.924+0000 7f864ffff640 1 -- 172.21.15.31:0/3301013091 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8650011e20 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.924+0000 7f864ffff640 1 -- 172.21.15.31:0/3301013091 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f865000f2e0 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.924+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86581a1e80 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.924+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86581a21e0 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.925+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8658068c90 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.926+0000 7f864ffff640 1 -- 172.21.15.31:0/3301013091 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f8650010380 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.926+0000 7f864ffff640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f864407af30 0x7f864407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.926+0000 7f864ffff640 1 -- 172.21.15.31:0/3301013091 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f865009d8d0 con 0x7f8658100d70 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.926+0000 7f86567fc640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f864407af30 0x7f864407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:30.949 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.927+0000 7f86567fc640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f864407af30 0x7f864407d3f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8658100150 tx=0x7f86400042a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:30.950 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:30.931+0000 7f864ffff640 1 -- 172.21.15.31:0/3301013091 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f86500609f0 con 0x7f8658100d70 2024-09-16T09:09:30.950 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:30 smithi031 ceph-mon[91884]: pgmap v355: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:31.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:30 smithi097 ceph-mon[79892]: pgmap v355: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.154+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f86580ff780 con 0x7f8658100d70 2024-09-16T09:09:31.156 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.155+0000 7f864ffff640 1 -- 172.21.15.31:0/3301013091 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v38) v1 ==== 107+0+5033 (secure 0 0 0) 0x7f86580ff780 con 0x7f8658100d70 2024-09-16T09:09:31.156 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:31.156 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":26,"btime":"2024-09-16T09:04:48:777314+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:42.715976+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24337},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24337":{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":0,"incarnation":25,"state":"up:replay","state_seq":2,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:31.157 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 26 2024-09-16T09:09:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f864407af30 msgr2=0x7f864407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:31.158 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f864407af30 0x7f864407d3f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8658100150 tx=0x7f86400042a0 comp rx=0 tx=0).stop 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658100d70 msgr2=0x7f86580fe9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658100d70 0x7f86580fe9d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f865000b420 tx=0x7f865000b8e0 comp rx=0 tx=0).stop 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 shutdown_connections 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f864407af30 0x7f864407d3f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8658101690 0x7f86580fef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 --2- 172.21.15.31:0/3301013091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8658100d70 0x7f86580fe9d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 >> 172.21.15.31:0/3301013091 conn(0x7f86580fa730 msgr2=0x7f8658101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 shutdown_connections 2024-09-16T09:09:31.159 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:31.157+0000 7f865d4b8640 1 -- 172.21.15.31:0/3301013091 wait complete. 2024-09-16T09:09:31.847 DEBUG:tasks.fs:max_mds reduced in epoch 26 2024-09-16T09:09:31.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 27 2024-09-16T09:09:32.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:31 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3301013091' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-16T09:09:32.055 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:32.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:31 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3301013091' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 26, "format": "json"} : dispatch 2024-09-16T09:09:33.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:32 smithi097 ceph-mon[79892]: pgmap v356: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:33.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:32 smithi031 ceph-mon[91884]: pgmap v356: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:33.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 -- 172.21.15.31:0/417974163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 msgr2=0x7f6b20100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:33.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 --2- 172.21.15.31:0/417974163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b20100180 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c009880 tx=0x7f6b1c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 -- 172.21.15.31:0/417974163 shutdown_connections 2024-09-16T09:09:33.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 --2- 172.21.15.31:0/417974163 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b20100750 0x7f6b201047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.703 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 --2- 172.21.15.31:0/417974163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b20100180 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 -- 172.21.15.31:0/417974163 >> 172.21.15.31:0/417974163 conn(0x7f6b200fb840 msgr2=0x7f6b200fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 -- 172.21.15.31:0/417974163 shutdown_connections 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.685+0000 7f6b27727640 1 -- 172.21.15.31:0/417974163 wait complete. 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b27727640 1 Processor -- start 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b27727640 1 -- start start 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b27727640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b2019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b27727640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b20100750 0x7f6b2019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b27727640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2019c360 con 0x7f6b200ffda0 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b27727640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201958f0 con 0x7f6b20100750 2024-09-16T09:09:33.704 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b254c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b2019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b254c3640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b2019b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48812/0 (socket says 172.21.15.31:48812) 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b254c3640 1 -- 172.21.15.31:0/3894627135 learned_addr learned my addr 172.21.15.31:0/3894627135 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b24cc2640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b20100750 0x7f6b2019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b254c3640 1 -- 172.21.15.31:0/3894627135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b20100750 msgr2=0x7f6b2019bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b254c3640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b20100750 0x7f6b2019bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.686+0000 7f6b254c3640 1 -- 172.21.15.31:0/3894627135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c009560 con 0x7f6b200ffda0 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.687+0000 7f6b24cc2640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b20100750 0x7f6b2019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.687+0000 7f6b254c3640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b2019b800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c02a9c0 tx=0x7f6b1c0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.687+0000 7f6b167fc640 1 -- 172.21.15.31:0/3894627135 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1c004aa0 con 0x7f6b200ffda0 2024-09-16T09:09:33.705 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.687+0000 7f6b167fc640 1 -- 172.21.15.31:0/3894627135 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b1c004c00 con 0x7f6b200ffda0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.687+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b20195b70 con 0x7f6b200ffda0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.687+0000 7f6b167fc640 1 -- 172.21.15.31:0/3894627135 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b1c003980 con 0x7f6b200ffda0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.687+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b20196060 con 0x7f6b200ffda0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.688+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20101280 con 0x7f6b200ffda0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.689+0000 7f6b167fc640 1 -- 172.21.15.31:0/3894627135 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f6b1c003bf0 con 0x7f6b200ffda0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.689+0000 7f6b167fc640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6b0c07af30 0x7f6b0c07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.689+0000 7f6b167fc640 1 -- 172.21.15.31:0/3894627135 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f6b1c0bf160 con 0x7f6b200ffda0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.689+0000 7f6b24cc2640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6b0c07af30 0x7f6b0c07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.690+0000 7f6b24cc2640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6b0c07af30 0x7f6b0c07d3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b20196d60 tx=0x7f6b10005e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:33.706 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.694+0000 7f6b167fc640 1 -- 172.21.15.31:0/3894627135 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f6b1c0821d0 con 0x7f6b200ffda0 2024-09-16T09:09:33.918 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.917+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f6b201080b0 con 0x7f6b200ffda0 2024-09-16T09:09:33.919 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.918+0000 7f6b167fc640 1 -- 172.21.15.31:0/3894627135 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v38) v1 ==== 107+0+5038 (secure 0 0 0) 0x7f6b1c0852f0 con 0x7f6b200ffda0 2024-09-16T09:09:33.919 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:33.919 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":27,"btime":"2024-09-16T09:04:50:834523+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:49.876319+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24337},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24337":{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":0,"incarnation":25,"state":"up:reconnect","state_seq":151,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:33.920 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 27 2024-09-16T09:09:33.925 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6b0c07af30 msgr2=0x7f6b0c07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6b0c07af30 0x7f6b0c07d3f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b20196d60 tx=0x7f6b10005e50 comp rx=0 tx=0).stop 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 msgr2=0x7f6b2019b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b2019b800 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c02a9c0 tx=0x7f6b1c0055a0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 shutdown_connections 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f6b0c07af30 0x7f6b0c07d3f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b20100750 0x7f6b2019bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 --2- 172.21.15.31:0/3894627135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b200ffda0 0x7f6b2019b800 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 >> 172.21.15.31:0/3894627135 conn(0x7f6b200fb840 msgr2=0x7f6b200fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.920+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 shutdown_connections 2024-09-16T09:09:33.926 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:33.921+0000 7f6b27727640 1 -- 172.21.15.31:0/3894627135 wait complete. 2024-09-16T09:09:34.597 DEBUG:tasks.fs:max_mds reduced in epoch 27 2024-09-16T09:09:34.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 28 2024-09-16T09:09:34.808 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:34.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:34 smithi031 ceph-mon[91884]: pgmap v357: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:34.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:34 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3894627135' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-16T09:09:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:34 smithi097 ceph-mon[79892]: pgmap v357: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:35.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:34 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3894627135' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 27, "format": "json"} : dispatch 2024-09-16T09:09:36.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.478+0000 7f592f89a640 1 -- 172.21.15.31:0/280879170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 msgr2=0x7f59280fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.478+0000 7f592f89a640 1 --2- 172.21.15.31:0/280879170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f59280fee50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f591c009880 tx=0x7f591c02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.478+0000 7f592f89a640 1 -- 172.21.15.31:0/280879170 shutdown_connections 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.478+0000 7f592f89a640 1 --2- 172.21.15.31:0/280879170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f59280fee50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.478+0000 7f592f89a640 1 --2- 172.21.15.31:0/280879170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281049f0 0x7f5928104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.478+0000 7f592f89a640 1 -- 172.21.15.31:0/280879170 >> 172.21.15.31:0/280879170 conn(0x7f59280758a0 msgr2=0x7f59280fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 -- 172.21.15.31:0/280879170 shutdown_connections 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 -- 172.21.15.31:0/280879170 wait complete. 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 Processor -- start 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 -- start start 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f592819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:36.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281049f0 0x7f592819bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592819c510 con 0x7f59280fe9f0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.479+0000 7f592f89a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59281959e0 con 0x7f59281049f0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592d636640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f592819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592d636640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f592819b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48840/0 (socket says 172.21.15.31:48840) 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592d636640 1 -- 172.21.15.31:0/3267501508 learned_addr learned my addr 172.21.15.31:0/3267501508 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592ce35640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281049f0 0x7f592819bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592d636640 1 -- 172.21.15.31:0/3267501508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281049f0 msgr2=0x7f592819bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592d636640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281049f0 0x7f592819bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592d636640 1 -- 172.21.15.31:0/3267501508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c009560 con 0x7f59280fe9f0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592d636640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f592819b980 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f592400c8a0 tx=0x7f592400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f591a7fc640 1 -- 172.21.15.31:0/3267501508 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5924007b80 con 0x7f59280fe9f0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f591a7fc640 1 -- 172.21.15.31:0/3267501508 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5924007ce0 con 0x7f59280fe9f0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5928195cc0 con 0x7f59280fe9f0 2024-09-16T09:09:36.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f591a7fc640 1 -- 172.21.15.31:0/3267501508 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5924018810 con 0x7f59280fe9f0 2024-09-16T09:09:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.480+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59281961e0 con 0x7f59280fe9f0 2024-09-16T09:09:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.481+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5928100130 con 0x7f59280fe9f0 2024-09-16T09:09:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.482+0000 7f591a7fc640 1 -- 172.21.15.31:0/3267501508 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f5924018970 con 0x7f59280fe9f0 2024-09-16T09:09:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.483+0000 7f591a7fc640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f591407af30 0x7f591407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.483+0000 7f591a7fc640 1 -- 172.21.15.31:0/3267501508 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f592409e240 con 0x7f59280fe9f0 2024-09-16T09:09:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.483+0000 7f592ce35640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f591407af30 0x7f591407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:36.487 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.484+0000 7f592ce35640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f591407af30 0x7f591407d3f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f591c000c00 tx=0x7f591c005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:36.488 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.487+0000 7f591a7fc640 1 -- 172.21.15.31:0/3267501508 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f59240612d0 con 0x7f59280fe9f0 2024-09-16T09:09:36.724 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.723+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f592810bc60 con 0x7f59280fe9f0 2024-09-16T09:09:36.725 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.724+0000 7f591a7fc640 1 -- 172.21.15.31:0/3267501508 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v38) v1 ==== 107+0+5035 (secure 0 0 0) 0x7f59240643f0 con 0x7f59280fe9f0 2024-09-16T09:09:36.725 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:36.725 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":28,"btime":"2024-09-16T09:04:51:840518+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:50.843354+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24337},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24337":{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":0,"incarnation":25,"state":"up:rejoin","state_seq":152,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:36.725 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 28 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f591407af30 msgr2=0x7f591407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f591407af30 0x7f591407d3f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f591c000c00 tx=0x7f591c005a20 comp rx=0 tx=0).stop 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 msgr2=0x7f592819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f592819b980 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f592400c8a0 tx=0x7f592400cd60 comp rx=0 tx=0).stop 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 shutdown_connections 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f591407af30 0x7f591407d3f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59281049f0 0x7f592819bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 --2- 172.21.15.31:0/3267501508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f59280fe9f0 0x7f592819b980 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:36.727 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 >> 172.21.15.31:0/3267501508 conn(0x7f59280758a0 msgr2=0x7f59280fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 shutdown_connections 2024-09-16T09:09:36.728 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:36.726+0000 7f592f89a640 1 -- 172.21.15.31:0/3267501508 wait complete. 2024-09-16T09:09:37.011 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:36 smithi097 ceph-mon[79892]: pgmap v358: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:37.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:36 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3267501508' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-16T09:09:37.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:36 smithi031 ceph-mon[91884]: pgmap v358: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:37.088 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:36 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3267501508' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 28, "format": "json"} : dispatch 2024-09-16T09:09:37.542 DEBUG:tasks.fs:max_mds reduced in epoch 28 2024-09-16T09:09:37.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 29 2024-09-16T09:09:37.761 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:38.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:38 smithi031 ceph-mon[91884]: pgmap v359: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:39.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:38 smithi097 ceph-mon[79892]: pgmap v359: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:39.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.502+0000 7f85d88bd640 1 -- 172.21.15.31:0/2406034049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d41049f0 msgr2=0x7f85d4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:39.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.502+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2406034049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d41049f0 0x7f85d4104dd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f85c4009880 tx=0x7f85c402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.502+0000 7f85d88bd640 1 -- 172.21.15.31:0/2406034049 shutdown_connections 2024-09-16T09:09:39.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.503+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2406034049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85d40fe9f0 0x7f85d40fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.503+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2406034049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d41049f0 0x7f85d4104dd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.503+0000 7f85d88bd640 1 -- 172.21.15.31:0/2406034049 >> 172.21.15.31:0/2406034049 conn(0x7f85d40758a0 msgr2=0x7f85d40fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:39.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.503+0000 7f85d88bd640 1 -- 172.21.15.31:0/2406034049 shutdown_connections 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.503+0000 7f85d88bd640 1 -- 172.21.15.31:0/2406034049 wait complete. 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.503+0000 7f85d88bd640 1 Processor -- start 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.503+0000 7f85d88bd640 1 -- start start 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d88bd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d40fe9f0 0x7f85d4106020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d88bd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85d41049f0 0x7f85d4106560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d88bd640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4106aa0 con 0x7f85d40fe9f0 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d88bd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d4106be0 con 0x7f85d41049f0 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d40fe9f0 0x7f85d4106020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d259c640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d40fe9f0 0x7f85d4106020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:48852/0 (socket says 172.21.15.31:48852) 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d259c640 1 -- 172.21.15.31:0/2595754474 learned_addr learned my addr 172.21.15.31:0/2595754474 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d1d9b640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85d41049f0 0x7f85d4106560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:39.527 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d259c640 1 -- 172.21.15.31:0/2595754474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85d41049f0 msgr2=0x7f85d4106560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d259c640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85d41049f0 0x7f85d4106560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d259c640 1 -- 172.21.15.31:0/2595754474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c4009560 con 0x7f85d40fe9f0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85d259c640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d40fe9f0 0x7f85d4106020 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f85c4006010 tx=0x7f85c40055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.504+0000 7f85c37fe640 1 -- 172.21.15.31:0/2595754474 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c4004aa0 con 0x7f85d40fe9f0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.505+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d4106e60 con 0x7f85d40fe9f0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.505+0000 7f85c37fe640 1 -- 172.21.15.31:0/2595754474 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85c4031470 con 0x7f85d40fe9f0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.505+0000 7f85c37fe640 1 -- 172.21.15.31:0/2595754474 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85c4003940 con 0x7f85d40fe9f0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.505+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d41a1fa0 con 0x7f85d40fe9f0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.506+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d4100130 con 0x7f85d40fe9f0 2024-09-16T09:09:39.528 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.507+0000 7f85c37fe640 1 -- 172.21.15.31:0/2595754474 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f85c4031b90 con 0x7f85d40fe9f0 2024-09-16T09:09:39.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.507+0000 7f85c37fe640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f85bc07af30 0x7f85bc07d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:39.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.507+0000 7f85c37fe640 1 -- 172.21.15.31:0/2595754474 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f85c40bf270 con 0x7f85d40fe9f0 2024-09-16T09:09:39.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.507+0000 7f85d1d9b640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f85bc07af30 0x7f85bc07d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:39.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.508+0000 7f85d1d9b640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f85bc07af30 0x7f85bc07d3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005fd0 tx=0x7f85c8005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:39.529 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.512+0000 7f85c37fe640 1 -- 172.21.15.31:0/2595754474 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f85c4082280 con 0x7f85d40fe9f0 2024-09-16T09:09:39.737 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.736+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f85d40fee50 con 0x7f85d40fe9f0 2024-09-16T09:09:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.736+0000 7f85c37fe640 1 -- 172.21.15.31:0/2595754474 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v38) v1 ==== 107+0+5044 (secure 0 0 0) 0x7f85c40853a0 con 0x7f85d40fe9f0 2024-09-16T09:09:39.738 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:39.738 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":29,"btime":"2024-09-16T09:04:52:844896+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44323,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/1475349315","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":1475349315},{"type":"v1","addr":"172.21.15.97:6825","nonce":1475349315}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:52.844893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24337},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24337":{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":0,"incarnation":25,"state":"up:active","state_seq":153,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24337,"qdb_cluster":[24337]},"id":1}]} 2024-09-16T09:09:39.738 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 29 2024-09-16T09:09:39.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f85bc07af30 msgr2=0x7f85bc07d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:39.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f85bc07af30 0x7f85bc07d3f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f85c8005fd0 tx=0x7f85c8005850 comp rx=0 tx=0).stop 2024-09-16T09:09:39.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d40fe9f0 msgr2=0x7f85d4106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:39.740 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d40fe9f0 0x7f85d4106020 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f85c4006010 tx=0x7f85c40055a0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 shutdown_connections 2024-09-16T09:09:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f85bc07af30 0x7f85bc07d3f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f85d41049f0 0x7f85d4106560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 --2- 172.21.15.31:0/2595754474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d40fe9f0 0x7f85d4106020 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 >> 172.21.15.31:0/2595754474 conn(0x7f85d40758a0 msgr2=0x7f85d41084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 shutdown_connections 2024-09-16T09:09:39.741 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:39.739+0000 7f85d88bd640 1 -- 172.21.15.31:0/2595754474 wait complete. 2024-09-16T09:09:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:39 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:40.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:39 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2595754474' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-16T09:09:40.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:39 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:40.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:39 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2595754474' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 29, "format": "json"} : dispatch 2024-09-16T09:09:40.498 DEBUG:tasks.fs:max_mds reduced in epoch 29 2024-09-16T09:09:40.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 30 2024-09-16T09:09:40.711 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:41.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:40 smithi097 ceph-mon[79892]: pgmap v360: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:41.050 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:40 smithi031 ceph-mon[91884]: pgmap v360: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/3300705927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 msgr2=0x7fd1d81047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/3300705927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d81047a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc009880 tx=0x7fd1cc02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/3300705927 shutdown_connections 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/3300705927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d81047a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/3300705927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d80ffda0 0x7fd1d8100180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/3300705927 >> 172.21.15.31:0/3300705927 conn(0x7fd1d80fb840 msgr2=0x7fd1d80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/3300705927 shutdown_connections 2024-09-16T09:09:42.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.489+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/3300705927 wait complete. 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.490+0000 7fd1dd4d6640 1 Processor -- start 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.490+0000 7fd1dd4d6640 1 -- start start 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.490+0000 7fd1dd4d6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d80ffda0 0x7fd1d819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.490+0000 7fd1dd4d6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d819be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.490+0000 7fd1dd4d6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d819c480 con 0x7fd1d8100750 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.490+0000 7fd1dd4d6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d8195a10 con 0x7fd1d80ffda0 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d819be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d819be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45422/0 (socket says 172.21.15.31:45422) 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d67fc640 1 -- 172.21.15.31:0/279671702 learned_addr learned my addr 172.21.15.31:0/279671702 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:42.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d80ffda0 0x7fd1d819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d67fc640 1 -- 172.21.15.31:0/279671702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d80ffda0 msgr2=0x7fd1d819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d67fc640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d80ffda0 0x7fd1d819b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d67fc640 1 -- 172.21.15.31:0/279671702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc009560 con 0x7fd1d8100750 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1d67fc640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d819be60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006010 tx=0x7fd1cc0055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1bffff640 1 -- 172.21.15.31:0/279671702 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1cc004aa0 con 0x7fd1d8100750 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.491+0000 7fd1bffff640 1 -- 172.21.15.31:0/279671702 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1cc004c00 con 0x7fd1d8100750 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.492+0000 7fd1bffff640 1 -- 172.21.15.31:0/279671702 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1cc0038f0 con 0x7fd1d8100750 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.492+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d8195c90 con 0x7fd1d8100750 2024-09-16T09:09:42.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.492+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d8196180 con 0x7fd1d8100750 2024-09-16T09:09:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.493+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d8101280 con 0x7fd1d8100750 2024-09-16T09:09:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.493+0000 7fd1bffff640 1 -- 172.21.15.31:0/279671702 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fd1cc003bb0 con 0x7fd1d8100750 2024-09-16T09:09:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.494+0000 7fd1bffff640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1c007af30 0x7fd1c007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.494+0000 7fd1d6ffd640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1c007af30 0x7fd1c007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:42.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.494+0000 7fd1bffff640 1 -- 172.21.15.31:0/279671702 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fd1cc0bf2a0 con 0x7fd1d8100750 2024-09-16T09:09:42.496 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.495+0000 7fd1d6ffd640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1c007af30 0x7fd1c007d3f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009990 tx=0x7fd1c8009110 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:42.499 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.499+0000 7fd1bffff640 1 -- 172.21.15.31:0/279671702 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fd1cc082310 con 0x7fd1d8100750 2024-09-16T09:09:42.722 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.722+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fd1d8100200 con 0x7fd1d8100750 2024-09-16T09:09:42.723 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.722+0000 7fd1bffff640 1 -- 172.21.15.31:0/279671702 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v38) v1 ==== 107+0+4265 (secure 0 0 0) 0x7fd1cc003e60 con 0x7fd1d8100750 2024-09-16T09:09:42.723 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:42.723 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":30,"btime":"2024-09-16T09:05:01:436189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:52.844893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24337},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24337":{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":0,"incarnation":25,"state":"up:active","state_seq":153,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24337,"qdb_cluster":[24337]},"id":1}]} 2024-09-16T09:09:42.724 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 30 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.724+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1c007af30 msgr2=0x7fd1c007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.724+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1c007af30 0x7fd1c007d3f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009990 tx=0x7fd1c8009110 comp rx=0 tx=0).stop 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 msgr2=0x7fd1d819be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d819be60 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc006010 tx=0x7fd1cc0055a0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 shutdown_connections 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fd1c007af30 0x7fd1c007d3f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1d8100750 0x7fd1d819be60 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 --2- 172.21.15.31:0/279671702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1d80ffda0 0x7fd1d819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 >> 172.21.15.31:0/279671702 conn(0x7fd1d80fb840 msgr2=0x7fd1d80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 shutdown_connections 2024-09-16T09:09:42.726 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:42.725+0000 7fd1dd4d6640 1 -- 172.21.15.31:0/279671702 wait complete. 2024-09-16T09:09:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:42 smithi097 ceph-mon[79892]: pgmap v361: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:43.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:42 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/279671702' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-16T09:09:43.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:42 smithi031 ceph-mon[91884]: pgmap v361: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:43.095 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:42 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/279671702' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 30, "format": "json"} : dispatch 2024-09-16T09:09:43.428 DEBUG:tasks.fs:max_mds reduced in epoch 30 2024-09-16T09:09:43.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 31 2024-09-16T09:09:43.640 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:45.071 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:44 smithi031 ceph-mon[91884]: pgmap v362: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:45.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:44 smithi097 ceph-mon[79892]: pgmap v362: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 -- 172.21.15.31:0/1329612452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4040ffda0 msgr2=0x7ff404100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 --2- 172.21.15.31:0/1329612452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4040ffda0 0x7ff404100180 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009880 tx=0x7ff3f402a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 -- 172.21.15.31:0/1329612452 shutdown_connections 2024-09-16T09:09:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 --2- 172.21.15.31:0/1329612452 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff404100750 0x7ff4041047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.290 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 --2- 172.21.15.31:0/1329612452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff4040ffda0 0x7ff404100180 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 -- 172.21.15.31:0/1329612452 >> 172.21.15.31:0/1329612452 conn(0x7ff4040fb840 msgr2=0x7ff4040fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 -- 172.21.15.31:0/1329612452 shutdown_connections 2024-09-16T09:09:45.291 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 -- 172.21.15.31:0/1329612452 wait complete. 2024-09-16T09:09:45.292 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.289+0000 7ff408826640 1 Processor -- start 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff408826640 1 -- start start 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff408826640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4040ffda0 0x7ff40419b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff408826640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff404100750 0x7ff40419bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff408826640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404195a10 con 0x7ff404100750 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff408826640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff404195b50 con 0x7ff4040ffda0 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff401d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff404100750 0x7ff40419bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff401d9b640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff404100750 0x7ff40419bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45434/0 (socket says 172.21.15.31:45434) 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff401d9b640 1 -- 172.21.15.31:0/3552702936 learned_addr learned my addr 172.21.15.31:0/3552702936 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff40259c640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4040ffda0 0x7ff40419b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff401d9b640 1 -- 172.21.15.31:0/3552702936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4040ffda0 msgr2=0x7ff40419b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:45.293 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff401d9b640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4040ffda0 0x7ff40419b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.290+0000 7ff401d9b640 1 -- 172.21.15.31:0/3552702936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4009560 con 0x7ff404100750 2024-09-16T09:09:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.291+0000 7ff401d9b640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff404100750 0x7ff40419bef0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d6f0 tx=0x7ff3ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.291+0000 7ff3fb7fe640 1 -- 172.21.15.31:0/3552702936 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3ec009a30 con 0x7ff404100750 2024-09-16T09:09:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.291+0000 7ff3fb7fe640 1 -- 172.21.15.31:0/3552702936 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3ec0054e0 con 0x7ff404100750 2024-09-16T09:09:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.291+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff404195e30 con 0x7ff404100750 2024-09-16T09:09:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.291+0000 7ff3fb7fe640 1 -- 172.21.15.31:0/3552702936 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3ec010460 con 0x7ff404100750 2024-09-16T09:09:45.294 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.291+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff404196380 con 0x7ff404100750 2024-09-16T09:09:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.292+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff404101280 con 0x7ff404100750 2024-09-16T09:09:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.293+0000 7ff3fb7fe640 1 -- 172.21.15.31:0/3552702936 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7ff3ec009b90 con 0x7ff404100750 2024-09-16T09:09:45.295 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.293+0000 7ff3fb7fe640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff3f007af30 0x7ff3f007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.293+0000 7ff3fb7fe640 1 -- 172.21.15.31:0/3552702936 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7ff3ec09db60 con 0x7ff404100750 2024-09-16T09:09:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.293+0000 7ff40259c640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff3f007af30 0x7ff3f007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:45.296 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.294+0000 7ff40259c640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff3f007af30 0x7ff3f007d3f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009230 tx=0x7ff3f40058d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:45.299 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.298+0000 7ff3fb7fe640 1 -- 172.21.15.31:0/3552702936 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7ff3ec060c80 con 0x7ff404100750 2024-09-16T09:09:45.521 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.520+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7ff404196f70 con 0x7ff404100750 2024-09-16T09:09:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.520+0000 7ff3fb7fe640 1 -- 172.21.15.31:0/3552702936 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v38) v1 ==== 107+0+5109 (secure 0 0 0) 0x7ff3ec063da0 con 0x7ff404100750 2024-09-16T09:09:45.522 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:45.522 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":31,"btime":"2024-09-16T09:05:06:443206+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:04:52.844893+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24337},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_24337":{"gid":24337,"name":"cephfs.smithi097.sbwmfq","rank":0,"incarnation":25,"state":"up:active","state_seq":153,"addr":"172.21.15.97:6827/859148422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":859148422},{"type":"v1","addr":"172.21.15.97:6827","nonce":859148422}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24337,"qdb_cluster":[24337]},"id":1}]} 2024-09-16T09:09:45.522 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 31 2024-09-16T09:09:45.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff3f007af30 msgr2=0x7ff3f007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:45.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff3f007af30 0x7ff3f007d3f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009230 tx=0x7ff3f40058d0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff404100750 msgr2=0x7ff40419bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff404100750 0x7ff40419bef0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec00d6f0 tx=0x7ff3ec00dbb0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 shutdown_connections 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7ff3f007af30 0x7ff3f007d3f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff404100750 0x7ff40419bef0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 --2- 172.21.15.31:0/3552702936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff4040ffda0 0x7ff40419b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 >> 172.21.15.31:0/3552702936 conn(0x7ff4040fb840 msgr2=0x7ff4040fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 shutdown_connections 2024-09-16T09:09:45.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:45.523+0000 7ff408826640 1 -- 172.21.15.31:0/3552702936 wait complete. 2024-09-16T09:09:45.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:45 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3552702936' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-16T09:09:46.216 DEBUG:tasks.fs:max_mds reduced in epoch 31 2024-09-16T09:09:46.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 32 2024-09-16T09:09:46.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:45 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3552702936' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 31, "format": "json"} : dispatch 2024-09-16T09:09:46.425 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:46.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:46 smithi031 ceph-mon[91884]: pgmap v363: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:47.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:46 smithi097 ceph-mon[79892]: pgmap v363: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 -- 172.21.15.31:0/2843502233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 msgr2=0x7f7fb8109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 --2- 172.21.15.31:0/2843502233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb8109b40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009880 tx=0x7f7fa002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 -- 172.21.15.31:0/2843502233 shutdown_connections 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 --2- 172.21.15.31:0/2843502233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb8109b40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 --2- 172.21.15.31:0/2843502233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb8100d30 0x7f7fb8101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 -- 172.21.15.31:0/2843502233 >> 172.21.15.31:0/2843502233 conn(0x7f7fb80fa730 msgr2=0x7f7fb80fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 -- 172.21.15.31:0/2843502233 shutdown_connections 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 -- 172.21.15.31:0/2843502233 wait complete. 2024-09-16T09:09:48.204 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.197+0000 7f7fbd833640 1 Processor -- start 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fbd833640 1 -- start start 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fbd833640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb8100d30 0x7f7fb819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fbd833640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb819bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fbd833640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8195a30 con 0x7f7fb8101650 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fbd833640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb8195b70 con 0x7f7fb8100d30 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fb67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb819bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fb67fc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb819bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45452/0 (socket says 172.21.15.31:45452) 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fb67fc640 1 -- 172.21.15.31:0/1420443350 learned_addr learned my addr 172.21.15.31:0/1420443350 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fb6ffd640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb8100d30 0x7f7fb819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fb67fc640 1 -- 172.21.15.31:0/1420443350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb8100d30 msgr2=0x7f7fb819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fb67fc640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb8100d30 0x7f7fb819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.198+0000 7f7fb67fc640 1 -- 172.21.15.31:0/1420443350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0009560 con 0x7f7fb8101650 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.199+0000 7f7fb67fc640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb819bf10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002a9c0 tx=0x7f7fa0032bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:48.205 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.199+0000 7f7fbc831640 1 -- 172.21.15.31:0/1420443350 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa0005220 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.199+0000 7f7fbc831640 1 -- 172.21.15.31:0/1420443350 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa0005d50 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.199+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb8195df0 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.199+0000 7f7fbc831640 1 -- 172.21.15.31:0/1420443350 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa0003940 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.199+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb81962e0 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.200+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb8068cf0 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.201+0000 7f7fbc831640 1 -- 172.21.15.31:0/1420443350 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f7fa0005380 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.201+0000 7f7fbc831640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7fa407af30 0x7f7fa407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.201+0000 7f7fbc831640 1 -- 172.21.15.31:0/1420443350 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f7fa00bebf0 con 0x7f7fb8101650 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.201+0000 7f7fb6ffd640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7fa407af30 0x7f7fa407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:48.206 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.202+0000 7f7fb6ffd640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7fa407af30 0x7f7fa407d3f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009b60 tx=0x7f7fb0005f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:48.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.206+0000 7f7fbc831640 1 -- 172.21.15.31:0/1420443350 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f7fa0081ce0 con 0x7f7fb8101650 2024-09-16T09:09:48.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.435+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f7fb810cdd0 con 0x7f7fb8101650 2024-09-16T09:09:48.436 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.435+0000 7f7fbc831640 1 -- 172.21.15.31:0/1420443350 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v38) v1 ==== 107+0+4311 (secure 0 0 0) 0x7f7fa0084e00 con 0x7f7fb8101650 2024-09-16T09:09:48.437 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:48.437 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":32,"btime":"2024-09-16T09:05:16:134233+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":23},{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:05:16.134230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[1],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:48.437 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 32 2024-09-16T09:09:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7fa407af30 msgr2=0x7f7fa407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7fa407af30 0x7f7fa407d3f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0009b60 tx=0x7f7fb0005f90 comp rx=0 tx=0).stop 2024-09-16T09:09:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 msgr2=0x7f7fb819bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:48.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb819bf10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7fa002a9c0 tx=0x7f7fa0032bb0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 shutdown_connections 2024-09-16T09:09:48.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f7fa407af30 0x7f7fa407d3f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7fb8101650 0x7f7fb819bf10 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 --2- 172.21.15.31:0/1420443350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7fb8100d30 0x7f7fb819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:48.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 >> 172.21.15.31:0/1420443350 conn(0x7f7fb80fa730 msgr2=0x7f7fb8102020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:48.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 shutdown_connections 2024-09-16T09:09:48.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:48.438+0000 7f7fbd833640 1 -- 172.21.15.31:0/1420443350 wait complete. 2024-09-16T09:09:49.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:48 smithi031 ceph-mon[91884]: pgmap v364: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:49.041 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:48 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1420443350' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-16T09:09:49.064 DEBUG:tasks.fs:max_mds reduced in epoch 32 2024-09-16T09:09:49.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 33 2024-09-16T09:09:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:48 smithi097 ceph-mon[79892]: pgmap v364: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:49.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:48 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1420443350' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 32, "format": "json"} : dispatch 2024-09-16T09:09:49.274 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:50.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 -- 172.21.15.31:0/1009515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 msgr2=0x7f8d50104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:50.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 --2- 172.21.15.31:0/1009515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d50104dd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009880 tx=0x7f8d4002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:50.955 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 -- 172.21.15.31:0/1009515336 shutdown_connections 2024-09-16T09:09:50.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 --2- 172.21.15.31:0/1009515336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500fe9f0 0x7f8d500fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:50.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 --2- 172.21.15.31:0/1009515336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d50104dd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:50.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 -- 172.21.15.31:0/1009515336 >> 172.21.15.31:0/1009515336 conn(0x7f8d500758a0 msgr2=0x7f8d500fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:50.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 -- 172.21.15.31:0/1009515336 shutdown_connections 2024-09-16T09:09:50.956 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.954+0000 7f8d561f8640 1 -- 172.21.15.31:0/1009515336 wait complete. 2024-09-16T09:09:50.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d561f8640 1 Processor -- start 2024-09-16T09:09:50.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d561f8640 1 -- start start 2024-09-16T09:09:50.957 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d561f8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500fe9f0 0x7f8d5019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d561f8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d5019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d561f8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50195960 con 0x7f8d501049f0 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d561f8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d50195aa0 con 0x7f8d500fe9f0 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d4effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d5019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d4effd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d5019be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:45478/0 (socket says 172.21.15.31:45478) 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d4effd640 1 -- 172.21.15.31:0/2069104592 learned_addr learned my addr 172.21.15.31:0/2069104592 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.955+0000 7f8d4f7fe640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500fe9f0 0x7f8d5019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d4effd640 1 -- 172.21.15.31:0/2069104592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500fe9f0 msgr2=0x7f8d5019b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d4effd640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500fe9f0 0x7f8d5019b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d4effd640 1 -- 172.21.15.31:0/2069104592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d40009560 con 0x7f8d501049f0 2024-09-16T09:09:50.958 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d4effd640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d5019be40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400d6f0 tx=0x7f8d4400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:50.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d4cff9640 1 -- 172.21.15.31:0/2069104592 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d44009a30 con 0x7f8d501049f0 2024-09-16T09:09:50.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d4cff9640 1 -- 172.21.15.31:0/2069104592 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d440054e0 con 0x7f8d501049f0 2024-09-16T09:09:50.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d4cff9640 1 -- 172.21.15.31:0/2069104592 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d44010460 con 0x7f8d501049f0 2024-09-16T09:09:50.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d50195d80 con 0x7f8d501049f0 2024-09-16T09:09:50.959 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.956+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d501962d0 con 0x7f8d501049f0 2024-09-16T09:09:50.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.957+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d50100130 con 0x7f8d501049f0 2024-09-16T09:09:50.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.958+0000 7f8d4cff9640 1 -- 172.21.15.31:0/2069104592 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f8d44009b90 con 0x7f8d501049f0 2024-09-16T09:09:50.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.958+0000 7f8d4cff9640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8d3807af30 0x7f8d3807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:50.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.959+0000 7f8d4cff9640 1 -- 172.21.15.31:0/2069104592 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f8d4409db60 con 0x7f8d501049f0 2024-09-16T09:09:50.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.959+0000 7f8d4f7fe640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8d3807af30 0x7f8d3807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:50.960 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.959+0000 7f8d4f7fe640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8d3807af30 0x7f8d3807d3f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009230 tx=0x7f8d40005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:50.964 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:50.963+0000 7f8d4cff9640 1 -- 172.21.15.31:0/2069104592 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f8d44060c80 con 0x7f8d501049f0 2024-09-16T09:09:51.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:50 smithi031 ceph-mon[91884]: pgmap v365: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:51.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.189+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f8d50196e90 con 0x7f8d501049f0 2024-09-16T09:09:51.190 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.190+0000 7f8d4cff9640 1 -- 172.21.15.31:0/2069104592 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v38) v1 ==== 107+0+4390 (secure 0 0 0) 0x7f8d44063da0 con 0x7f8d501049f0 2024-09-16T09:09:51.191 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:51.191 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":33,"btime":"2024-09-16T09:05:16:143457+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:05:16.143437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34430},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:51.191 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 33 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8d3807af30 msgr2=0x7f8d3807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8d3807af30 0x7f8d3807d3f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d40009230 tx=0x7f8d40005a20 comp rx=0 tx=0).stop 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 msgr2=0x7f8d5019be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d5019be40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8d4400d6f0 tx=0x7f8d4400dbb0 comp rx=0 tx=0).stop 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 shutdown_connections 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8d3807af30 0x7f8d3807d3f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d501049f0 0x7f8d5019be40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 --2- 172.21.15.31:0/2069104592 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8d500fe9f0 0x7f8d5019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 >> 172.21.15.31:0/2069104592 conn(0x7f8d500758a0 msgr2=0x7f8d500fd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 shutdown_connections 2024-09-16T09:09:51.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:51.192+0000 7f8d561f8640 1 -- 172.21.15.31:0/2069104592 wait complete. 2024-09-16T09:09:51.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:50 smithi097 ceph-mon[79892]: pgmap v365: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:51.847 DEBUG:tasks.fs:max_mds reduced in epoch 33 2024-09-16T09:09:51.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 34 2024-09-16T09:09:52.055 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:52.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:51 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2069104592' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-16T09:09:52.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:51 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2069104592' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 33, "format": "json"} : dispatch 2024-09-16T09:09:53.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:52 smithi031 ceph-mon[91884]: pgmap v366: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:53.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:52 smithi097 ceph-mon[79892]: pgmap v366: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:53.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 -- 172.21.15.31:0/1178130960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 msgr2=0x7f13bc104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:53.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 --2- 172.21.15.31:0/1178130960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc104dd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009880 tx=0x7f13ac02a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:53.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 -- 172.21.15.31:0/1178130960 shutdown_connections 2024-09-16T09:09:53.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 --2- 172.21.15.31:0/1178130960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13bc0fe9f0 0x7f13bc0fee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:53.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 --2- 172.21.15.31:0/1178130960 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc104dd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:53.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 -- 172.21.15.31:0/1178130960 >> 172.21.15.31:0/1178130960 conn(0x7f13bc0758a0 msgr2=0x7f13bc0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:53.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 -- 172.21.15.31:0/1178130960 shutdown_connections 2024-09-16T09:09:53.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.717+0000 7f13c2424640 1 -- 172.21.15.31:0/1178130960 wait complete. 2024-09-16T09:09:53.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13c2424640 1 Processor -- start 2024-09-16T09:09:53.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13c2424640 1 -- start start 2024-09-16T09:09:53.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13c2424640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13bc0fe9f0 0x7f13bc105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:53.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13c2424640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc105e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:53.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13c2424640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc1078f0 con 0x7f13bc1049f0 2024-09-16T09:09:53.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13c2424640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13bc1063a0 con 0x7f13bc0fe9f0 2024-09-16T09:09:53.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:53.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc105e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56944/0 (socket says 172.21.15.31:56944) 2024-09-16T09:09:53.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.718+0000 7f13bb7fe640 1 -- 172.21.15.31:0/1709127722 learned_addr learned my addr 172.21.15.31:0/1709127722 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:53.753 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13bbfff640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13bc0fe9f0 0x7f13bc105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:53.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13bb7fe640 1 -- 172.21.15.31:0/1709127722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13bc0fe9f0 msgr2=0x7f13bc105920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:53.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13bb7fe640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13bc0fe9f0 0x7f13bc105920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:53.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13bb7fe640 1 -- 172.21.15.31:0/1709127722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac009560 con 0x7f13bc1049f0 2024-09-16T09:09:53.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13bbfff640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13bc0fe9f0 0x7f13bc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T09:09:53.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13bb7fe640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc105e60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f13b000c8a0 tx=0x7f13b000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:53.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13b97fa640 1 -- 172.21.15.31:0/1709127722 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b0007b80 con 0x7f13bc1049f0 2024-09-16T09:09:53.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13bc106680 con 0x7f13bc1049f0 2024-09-16T09:09:53.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13b97fa640 1 -- 172.21.15.31:0/1709127722 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13b0007ce0 con 0x7f13bc1049f0 2024-09-16T09:09:53.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13b97fa640 1 -- 172.21.15.31:0/1709127722 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b0018810 con 0x7f13bc1049f0 2024-09-16T09:09:53.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.719+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13bc1a1f30 con 0x7f13bc1049f0 2024-09-16T09:09:53.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.720+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13bc100130 con 0x7f13bc1049f0 2024-09-16T09:09:53.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.721+0000 7f13b97fa640 1 -- 172.21.15.31:0/1709127722 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f13b0018a80 con 0x7f13bc1049f0 2024-09-16T09:09:53.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.721+0000 7f13b97fa640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f13a407af30 0x7f13a407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:53.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.722+0000 7f13b97fa640 1 -- 172.21.15.31:0/1709127722 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f13b009e250 con 0x7f13bc1049f0 2024-09-16T09:09:53.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.722+0000 7f13bbfff640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f13a407af30 0x7f13a407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:53.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.722+0000 7f13bbfff640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f13a407af30 0x7f13a407d3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006010 tx=0x7f13ac005e60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:53.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.726+0000 7f13b97fa640 1 -- 172.21.15.31:0/1709127722 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f13b00612c0 con 0x7f13bc1049f0 2024-09-16T09:09:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.949+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f13bc0fee50 con 0x7f13bc1049f0 2024-09-16T09:09:53.951 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.950+0000 7f13b97fa640 1 -- 172.21.15.31:0/1709127722 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v38) v1 ==== 107+0+4394 (secure 0 0 0) 0x7f13b00643e0 con 0x7f13bc1049f0 2024-09-16T09:09:53.951 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:53.951 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":34,"btime":"2024-09-16T09:05:23:752612+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:05:23.292648+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34430},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:53.952 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 34 2024-09-16T09:09:53.953 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f13a407af30 msgr2=0x7f13a407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f13a407af30 0x7f13a407d3f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006010 tx=0x7f13ac005e60 comp rx=0 tx=0).stop 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 msgr2=0x7f13bc105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc105e60 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f13b000c8a0 tx=0x7f13b000cd60 comp rx=0 tx=0).stop 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 shutdown_connections 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f13a407af30 0x7f13a407d3f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13bc1049f0 0x7f13bc105e60 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.952+0000 7f13c2424640 1 --2- 172.21.15.31:0/1709127722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13bc0fe9f0 0x7f13bc105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.953+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 >> 172.21.15.31:0/1709127722 conn(0x7f13bc0758a0 msgr2=0x7f13bc067f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.953+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 shutdown_connections 2024-09-16T09:09:53.954 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:53.953+0000 7f13c2424640 1 -- 172.21.15.31:0/1709127722 wait complete. 2024-09-16T09:09:54.635 DEBUG:tasks.fs:max_mds reduced in epoch 34 2024-09-16T09:09:54.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 35 2024-09-16T09:09:54.853 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:54 smithi031 ceph-mon[91884]: pgmap v367: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:54 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:54.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:54 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/1709127722' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 34, "format": "json"} : dispatch 2024-09-16T09:09:55.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:54 smithi097 ceph-mon[79892]: pgmap v367: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:55.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:54 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:09:55.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:54 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/1709127722' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 34, "format": "json"} : dispatch 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 -- 172.21.15.31:0/1174698811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 msgr2=0x7f47a8100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 --2- 172.21.15.31:0/1174698811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a8100180 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4790009880 tx=0x7f479002a4a0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 -- 172.21.15.31:0/1174698811 shutdown_connections 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 --2- 172.21.15.31:0/1174698811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a8100750 0x7f47a81047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 --2- 172.21.15.31:0/1174698811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a8100180 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 -- 172.21.15.31:0/1174698811 >> 172.21.15.31:0/1174698811 conn(0x7f47a80fb840 msgr2=0x7f47a80fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 -- 172.21.15.31:0/1174698811 shutdown_connections 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.480+0000 7f47aeff8640 1 -- 172.21.15.31:0/1174698811 wait complete. 2024-09-16T09:09:56.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.481+0000 7f47aeff8640 1 Processor -- start 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.481+0000 7f47aeff8640 1 -- start start 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.481+0000 7f47aeff8640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a80727b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.481+0000 7f47aeff8640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a8100750 0x7f47a8072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.481+0000 7f47aeff8640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8073230 con 0x7f47a80ffda0 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.481+0000 7f47aeff8640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8073370 con 0x7f47a8100750 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47acd94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a80727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47acd94640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a80727b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56956/0 (socket says 172.21.15.31:56956) 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47acd94640 1 -- 172.21.15.31:0/2303140891 learned_addr learned my addr 172.21.15.31:0/2303140891 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:56.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47a7fff640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a8100750 0x7f47a8072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47acd94640 1 -- 172.21.15.31:0/2303140891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a8100750 msgr2=0x7f47a8072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47acd94640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a8100750 0x7f47a8072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47acd94640 1 -- 172.21.15.31:0/2303140891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4790009560 con 0x7f47a80ffda0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47acd94640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a80727b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f479002a9c0 tx=0x7f47900055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47a5ffb640 1 -- 172.21.15.31:0/2303140891 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4790004aa0 con 0x7f47a80ffda0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47a5ffb640 1 -- 172.21.15.31:0/2303140891 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4790004c00 con 0x7f47a80ffda0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a80735f0 con 0x7f47a80ffda0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.482+0000 7f47a5ffb640 1 -- 172.21.15.31:0/2303140891 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4790003980 con 0x7f47a80ffda0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.483+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a81a20e0 con 0x7f47a80ffda0 2024-09-16T09:09:56.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.483+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a8101280 con 0x7f47a80ffda0 2024-09-16T09:09:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.484+0000 7f47a5ffb640 1 -- 172.21.15.31:0/2303140891 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f4790003b20 con 0x7f47a80ffda0 2024-09-16T09:09:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.485+0000 7f47a5ffb640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f479407af30 0x7f479407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.485+0000 7f47a5ffb640 1 -- 172.21.15.31:0/2303140891 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f47900befa0 con 0x7f47a80ffda0 2024-09-16T09:09:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.485+0000 7f47a7fff640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f479407af30 0x7f479407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.485+0000 7f47a7fff640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f479407af30 0x7f479407d3f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47a80742b0 tx=0x7f4798009680 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:56.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.489+0000 7f47a5ffb640 1 -- 172.21.15.31:0/2303140891 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f4790082140 con 0x7f47a80ffda0 2024-09-16T09:09:56.713 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.712+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f47a8073ce0 con 0x7f47a80ffda0 2024-09-16T09:09:56.714 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.713+0000 7f47a5ffb640 1 -- 172.21.15.31:0/2303140891 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v38) v1 ==== 107+0+5238 (secure 0 0 0) 0x7f4790085260 con 0x7f47a80ffda0 2024-09-16T09:09:56.714 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:56.715 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":35,"btime":"2024-09-16T09:05:24:758510+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44375,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6827/2995504601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2995504601},{"type":"v1","addr":"172.21.15.97:6827","nonce":2995504601}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:05:23.763260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34430},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-16T09:09:56.715 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 35 2024-09-16T09:09:56.716 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f479407af30 msgr2=0x7f479407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f479407af30 0x7f479407d3f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f47a80742b0 tx=0x7f4798009680 comp rx=0 tx=0).stop 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 msgr2=0x7f47a80727b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a80727b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f479002a9c0 tx=0x7f47900055a0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 shutdown_connections 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f479407af30 0x7f479407d3f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47a8100750 0x7f47a8072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 --2- 172.21.15.31:0/2303140891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47a80ffda0 0x7f47a80727b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.715+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 >> 172.21.15.31:0/2303140891 conn(0x7f47a80fb840 msgr2=0x7f47a80fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.716+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 shutdown_connections 2024-09-16T09:09:56.717 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:56.716+0000 7f47aeff8640 1 -- 172.21.15.31:0/2303140891 wait complete. 2024-09-16T09:09:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:56 smithi031 ceph-mon[91884]: pgmap v368: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:57.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:56 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2303140891' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 35, "format": "json"} : dispatch 2024-09-16T09:09:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:56 smithi097 ceph-mon[79892]: pgmap v368: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:57.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:56 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2303140891' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 35, "format": "json"} : dispatch 2024-09-16T09:09:57.601 DEBUG:tasks.fs:max_mds reduced in epoch 35 2024-09-16T09:09:57.601 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 36 2024-09-16T09:09:57.814 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:09:58.918 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:58 smithi031 ceph-mon[91884]: pgmap v369: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:59.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:58 smithi097 ceph-mon[79892]: pgmap v369: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:09:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 -- 172.21.15.31:0/100837045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e640728e0 msgr2=0x7f0e64072cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 --2- 172.21.15.31:0/100837045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e640728e0 0x7f0e64072cc0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009880 tx=0x7f0e6002a530 comp rx=0 tx=0).stop 2024-09-16T09:09:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 -- 172.21.15.31:0/100837045 shutdown_connections 2024-09-16T09:09:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 --2- 172.21.15.31:0/100837045 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e64073200 0x7f0e6410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 --2- 172.21.15.31:0/100837045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e640728e0 0x7f0e64072cc0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 -- 172.21.15.31:0/100837045 >> 172.21.15.31:0/100837045 conn(0x7f0e640fb860 msgr2=0x7f0e640fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:59.425 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 -- 172.21.15.31:0/100837045 shutdown_connections 2024-09-16T09:09:59.426 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 -- 172.21.15.31:0/100837045 wait complete. 2024-09-16T09:09:59.427 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 Processor -- start 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.424+0000 7f0e6b660640 1 -- start start 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e6b660640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e640728e0 0x7f0e64197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e6b660640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e64073200 0x7f0e64198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e6b660640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e64198900 con 0x7f0e64073200 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e6b660640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6419c670 con 0x7f0e640728e0 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e68bfb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e64073200 0x7f0e64198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e68bfb640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e64073200 0x7f0e64198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56964/0 (socket says 172.21.15.31:56964) 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e68bfb640 1 -- 172.21.15.31:0/2883615568 learned_addr learned my addr 172.21.15.31:0/2883615568 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:09:59.428 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e693fc640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e640728e0 0x7f0e64197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e68bfb640 1 -- 172.21.15.31:0/2883615568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e640728e0 msgr2=0x7f0e64197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e68bfb640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e640728e0 0x7f0e64197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e68bfb640 1 -- 172.21.15.31:0/2883615568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60009560 con 0x7f0e64073200 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e68bfb640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e64073200 0x7f0e64198270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c8f0 tx=0x7f0e5400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.425+0000 7f0e5a7fc640 1 -- 172.21.15.31:0/2883615568 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e54007b80 con 0x7f0e64073200 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.426+0000 7f0e5a7fc640 1 -- 172.21.15.31:0/2883615568 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e540054e0 con 0x7f0e64073200 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.426+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6419c950 con 0x7f0e64073200 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.426+0000 7f0e5a7fc640 1 -- 172.21.15.31:0/2883615568 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5400f450 con 0x7f0e64073200 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.426+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6419cea0 con 0x7f0e64073200 2024-09-16T09:09:59.429 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.427+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e64073f90 con 0x7f0e64073200 2024-09-16T09:09:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.427+0000 7f0e5a7fc640 1 -- 172.21.15.31:0/2883615568 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f0e54004f60 con 0x7f0e64073200 2024-09-16T09:09:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.428+0000 7f0e5a7fc640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0e5007af30 0x7f0e5007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:09:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.428+0000 7f0e5a7fc640 1 -- 172.21.15.31:0/2883615568 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f0e5409dde0 con 0x7f0e64073200 2024-09-16T09:09:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.428+0000 7f0e693fc640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0e5007af30 0x7f0e5007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:09:59.430 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.428+0000 7f0e693fc640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0e5007af30 0x7f0e5007d3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000c00 tx=0x7f0e60005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:09:59.433 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.432+0000 7f0e5a7fc640 1 -- 172.21.15.31:0/2883615568 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f0e54060ed0 con 0x7f0e64073200 2024-09-16T09:09:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.657+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f0e64072cc0 con 0x7f0e64073200 2024-09-16T09:09:59.658 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.657+0000 7f0e5a7fc640 1 -- 172.21.15.31:0/2883615568 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v38) v1 ==== 107+0+5247 (secure 0 0 0) 0x7f0e54063ff0 con 0x7f0e64073200 2024-09-16T09:09:59.659 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:09:59.659 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":36,"btime":"2024-09-16T09:05:25:814181+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44375,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6827/2995504601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2995504601},{"type":"v1","addr":"172.21.15.97:6827","nonce":2995504601}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:05:25.814178+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34430},"failed":[],"damaged":[],"stopped":[1],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":0,"incarnation":33,"state":"up:active","state_seq":17,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34430,"qdb_cluster":[34430]},"id":1}]} 2024-09-16T09:09:59.659 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 36 2024-09-16T09:09:59.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0e5007af30 msgr2=0x7f0e5007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:59.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0e5007af30 0x7f0e5007d3f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0e60000c00 tx=0x7f0e60005a20 comp rx=0 tx=0).stop 2024-09-16T09:09:59.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e64073200 msgr2=0x7f0e64198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:09:59.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e64073200 0x7f0e64198270 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400c8f0 tx=0x7f0e5400cdb0 comp rx=0 tx=0).stop 2024-09-16T09:09:59.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 shutdown_connections 2024-09-16T09:09:59.661 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f0e5007af30 0x7f0e5007d3f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e64073200 0x7f0e64198270 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 --2- 172.21.15.31:0/2883615568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0e640728e0 0x7f0e64197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:09:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 >> 172.21.15.31:0/2883615568 conn(0x7f0e640fb860 msgr2=0x7f0e641042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:09:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 shutdown_connections 2024-09-16T09:09:59.662 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:09:59.660+0000 7f0e6b660640 1 -- 172.21.15.31:0/2883615568 wait complete. 2024-09-16T09:10:00.034 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:09:59 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/2883615568' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 36, "format": "json"} : dispatch 2024-09-16T09:10:00.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:09:59 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/2883615568' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 36, "format": "json"} : dispatch 2024-09-16T09:10:00.386 DEBUG:tasks.fs:max_mds reduced in epoch 36 2024-09-16T09:10:00.386 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 37 2024-09-16T09:10:00.604 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:10:00.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:00 smithi031 ceph-mon[91884]: pgmap v370: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:00.984 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:00 smithi031 ceph-mon[91884]: overall HEALTH_OK 2024-09-16T09:10:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:00 smithi097 ceph-mon[79892]: pgmap v370: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:01.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:00 smithi097 ceph-mon[79892]: overall HEALTH_OK 2024-09-16T09:10:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 -- 172.21.15.31:0/566195626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 msgr2=0x7efe0c101090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 --2- 172.21.15.31:0/566195626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c101090 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009c20 tx=0x7efdfc02aa00 comp rx=0 tx=0).stop 2024-09-16T09:10:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 -- 172.21.15.31:0/566195626 shutdown_connections 2024-09-16T09:10:02.228 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 --2- 172.21.15.31:0/566195626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe0c1015d0 0x7efe0c109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 --2- 172.21.15.31:0/566195626 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c101090 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 -- 172.21.15.31:0/566195626 >> 172.21.15.31:0/566195626 conn(0x7efe0c0758a0 msgr2=0x7efe0c0fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 -- 172.21.15.31:0/566195626 shutdown_connections 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 -- 172.21.15.31:0/566195626 wait complete. 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.218+0000 7efe113b6640 1 Processor -- start 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe113b6640 1 -- start start 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe113b6640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe113b6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe0c1015d0 0x7efe0c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:02.229 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe113b6640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c19c3d0 con 0x7efe0c100cb0 2024-09-16T09:10:02.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe113b6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c195960 con 0x7efe0c1015d0 2024-09-16T09:10:02.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe0affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:02.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe0affd640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c19b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56384/0 (socket says 172.21.15.31:56384) 2024-09-16T09:10:02.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe0affd640 1 -- 172.21.15.31:0/3931342532 learned_addr learned my addr 172.21.15.31:0/3931342532 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:10:02.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe0a7fc640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe0c1015d0 0x7efe0c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:02.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe0affd640 1 -- 172.21.15.31:0/3931342532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe0c1015d0 msgr2=0x7efe0c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:02.230 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe0affd640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe0c1015d0 0x7efe0c19bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:02.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.219+0000 7efe0affd640 1 -- 172.21.15.31:0/3931342532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc009900 con 0x7efe0c100cb0 2024-09-16T09:10:02.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.220+0000 7efe0affd640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c19b870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efdfc006010 tx=0x7efdfc02f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:10:02.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.220+0000 7efdf3fff640 1 -- 172.21.15.31:0/3931342532 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdfc03e070 con 0x7efe0c100cb0 2024-09-16T09:10:02.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.220+0000 7efdf3fff640 1 -- 172.21.15.31:0/3931342532 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdfc02e870 con 0x7efe0c100cb0 2024-09-16T09:10:02.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.220+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c195be0 con 0x7efe0c100cb0 2024-09-16T09:10:02.231 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.220+0000 7efdf3fff640 1 -- 172.21.15.31:0/3931342532 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdfc0047a0 con 0x7efe0c100cb0 2024-09-16T09:10:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.220+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c1960d0 con 0x7efe0c100cb0 2024-09-16T09:10:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.221+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c068510 con 0x7efe0c100cb0 2024-09-16T09:10:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.221+0000 7efdf3fff640 1 -- 172.21.15.31:0/3931342532 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7efdfc00a040 con 0x7efe0c100cb0 2024-09-16T09:10:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.222+0000 7efdf3fff640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efdf407af30 0x7efdf407d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.222+0000 7efdf3fff640 1 -- 172.21.15.31:0/3931342532 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7efdfc0bf2a0 con 0x7efe0c100cb0 2024-09-16T09:10:02.232 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.222+0000 7efe0a7fc640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efdf407af30 0x7efdf407d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.222+0000 7efe0a7fc640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efdf407af30 0x7efdf407d3f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe0c196dd0 tx=0x7efe00005eb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:10:02.233 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.227+0000 7efdf3fff640 1 -- 172.21.15.31:0/3931342532 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7efdfc082390 con 0x7efe0c100cb0 2024-09-16T09:10:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.453+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7efe0c104340 con 0x7efe0c100cb0 2024-09-16T09:10:02.454 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.453+0000 7efdf3fff640 1 -- 172.21.15.31:0/3931342532 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v38) v1 ==== 107+0+5264 (secure 0 0 0) 0x7efdfc0854b0 con 0x7efe0c100cb0 2024-09-16T09:10:02.454 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-16T09:10:02.455 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":37,"btime":"2024-09-16T09:05:31:633189+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44353,"name":"cephfs.smithi097.cagmus","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6825/257407185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":257407185},{"type":"v1","addr":"172.21.15.97:6825","nonce":257407185}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":44375,"name":"cephfs.smithi097.sbwmfq","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6827/2995504601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":2995504601},{"type":"v1","addr":"172.21.15.97:6827","nonce":2995504601}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T08:54:37.289736+0000","modified":"2024-09-16T09:05:31.633158+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi031.lcfutm","rank":0,"incarnation":33,"state":"up:active","state_seq":17,"addr":"172.21.15.31:6827/4118035133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4118035133},{"type":"v1","addr":"172.21.15.31:6827","nonce":4118035133}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34434":{"gid":34434,"name":"cephfs.smithi031.mjhxgi","rank":1,"incarnation":37,"state":"up:starting","state_seq":1,"addr":"172.21.15.31:6829/1583612894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":1583612894},{"type":"v1","addr":"172.21.15.31:6829","nonce":1583612894}]},"join_fscid":1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34430,"qdb_cluster":[34430]},"id":1}]} 2024-09-16T09:10:02.455 INFO:teuthology.orchestra.run.smithi031.stderr:dumped fsmap epoch 37 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efdf407af30 msgr2=0x7efdf407d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efdf407af30 0x7efdf407d3f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe0c196dd0 tx=0x7efe00005eb0 comp rx=0 tx=0).stop 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 msgr2=0x7efe0c19b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c19b870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7efdfc006010 tx=0x7efdfc02f870 comp rx=0 tx=0).stop 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 shutdown_connections 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7efdf407af30 0x7efdf407d3f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efe0c1015d0 0x7efe0c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 --2- 172.21.15.31:0/3931342532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe0c100cb0 0x7efe0c19b870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 >> 172.21.15.31:0/3931342532 conn(0x7efe0c0758a0 msgr2=0x7efe0c101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:10:02.457 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 shutdown_connections 2024-09-16T09:10:02.458 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:02.456+0000 7efe113b6640 1 -- 172.21.15.31:0/3931342532 wait complete. 2024-09-16T09:10:03.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:02 smithi031 ceph-mon[91884]: pgmap v371: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:03.090 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:02 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/3931342532' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 37, "format": "json"} : dispatch 2024-09-16T09:10:03.118 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T09:10:03.130 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T09:10:03.130 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T09:10:03.130 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T09:10:03.152 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-16T09:10:03.152 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T09:10:03.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T09:10:03.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:02 smithi097 ceph-mon[79892]: pgmap v371: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:03.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:02 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/3931342532' entity='client.admin' cmd={"prefix": "fs dump", "epoch": 37, "format": "json"} : dispatch 2024-09-16T09:10:03.468 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:10:04.873 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:04 smithi031 ceph-mon[91884]: pgmap v372: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 -- 172.21.15.31:0/2697423490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 msgr2=0x7fae74101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/2697423490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae74101040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fae700098b0 tx=0x7fae7002a4d0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 -- 172.21.15.31:0/2697423490 shutdown_connections 2024-09-16T09:10:05.121 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/2697423490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74101610 0x7fae74109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/2697423490 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae74101040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 -- 172.21.15.31:0/2697423490 >> 172.21.15.31:0/2697423490 conn(0x7fae740fa730 msgr2=0x7fae740fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 -- 172.21.15.31:0/2697423490 shutdown_connections 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 -- 172.21.15.31:0/2697423490 wait complete. 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.118+0000 7fae7b7dc640 1 Processor -- start 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae7b7dc640 1 -- start start 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae7b7dc640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae7419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae7b7dc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74101610 0x7fae7419bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae7b7dc640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7419c380 con 0x7fae74100c60 2024-09-16T09:10:05.122 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae7b7dc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae74195910 con 0x7fae74101610 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae79578640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae7419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae79578640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae7419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56416/0 (socket says 172.21.15.31:56416) 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae79578640 1 -- 172.21.15.31:0/4227673185 learned_addr learned my addr 172.21.15.31:0/4227673185 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae78d77640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74101610 0x7fae7419bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae79578640 1 -- 172.21.15.31:0/4227673185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74101610 msgr2=0x7fae7419bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae79578640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74101610 0x7fae7419bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae79578640 1 -- 172.21.15.31:0/4227673185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae70009560 con 0x7fae74100c60 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.119+0000 7fae79578640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae7419b820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae70006010 tx=0x7fae700055a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.120+0000 7fae6a7fc640 1 -- 172.21.15.31:0/4227673185 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae70004aa0 con 0x7fae74100c60 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.120+0000 7fae6a7fc640 1 -- 172.21.15.31:0/4227673185 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae70031470 con 0x7fae74100c60 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.120+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae74195b90 con 0x7fae74100c60 2024-09-16T09:10:05.123 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.120+0000 7fae6a7fc640 1 -- 172.21.15.31:0/4227673185 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae70003940 con 0x7fae74100c60 2024-09-16T09:10:05.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.120+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae74196080 con 0x7fae74100c60 2024-09-16T09:10:05.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.121+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae74068d30 con 0x7fae74100c60 2024-09-16T09:10:05.124 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.121+0000 7fae6a7fc640 1 -- 172.21.15.31:0/4227673185 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7fae70031b90 con 0x7fae74100c60 2024-09-16T09:10:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.122+0000 7fae6a7fc640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae6007af30 0x7fae6007d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.122+0000 7fae6a7fc640 1 -- 172.21.15.31:0/4227673185 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7fae700bf130 con 0x7fae74100c60 2024-09-16T09:10:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.122+0000 7fae78d77640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae6007af30 0x7fae6007d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:05.125 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.123+0000 7fae78d77640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae6007af30 0x7fae6007d3f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae64005fd0 tx=0x7fae64005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:10:05.127 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.126+0000 7fae6a7fc640 1 -- 172.21.15.31:0/4227673185 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7fae700821c0 con 0x7fae74100c60 2024-09-16T09:10:05.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:04 smithi097 ceph-mon[79892]: pgmap v372: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.324+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae74101040 con 0x7fae74100c60 2024-09-16T09:10:05.325 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.324+0000 7fae6a7fc640 1 -- 172.21.15.31:0/4227673185 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v99) v1 ==== 81+0+2535 (secure 0 0 0) 0x7fae70004c00 con 0x7fae74100c60 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6827/859148422 2024-09-17T09:05:16.133859+0000 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2307887580 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2307887580 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/662233816 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3400454826 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6829/1804644990 2024-09-17T09:04:42.710262+0000 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3244157830 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.326 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/259397163 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/4251203839 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6829/3476759025 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6828/1804644990 2024-09-17T09:04:42.710262+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/838457214 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/625663167 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/1878583845 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/2994879783 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4102636839 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/202749585 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/489385232 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/2752878219 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2518533508 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:05.327 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2012899681 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3080093455 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2103860772 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/488511267 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2737304770 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3080093455 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/2162767536 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2552418024 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6826/2048602032 2024-09-17T08:54:52.880019+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/1249429801 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2103860772 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6827/2048602032 2024-09-17T08:54:52.880019+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6826/859148422 2024-09-17T09:05:16.133859+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3789167564 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/494755020 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6828/3476759025 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2575391456 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1714541219 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/494755020 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/320751586 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2621367533 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2575391456 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/596031472 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:listed 43 entries 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.326+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae6007af30 msgr2=0x7fae6007d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.326+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae6007af30 0x7fae6007d3f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae64005fd0 tx=0x7fae64005850 comp rx=0 tx=0).stop 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.326+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 msgr2=0x7fae7419b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.326+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae7419b820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fae70006010 tx=0x7fae700055a0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.327+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 shutdown_connections 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.327+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7fae6007af30 0x7fae6007d3f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.327+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fae74101610 0x7fae7419bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.327+0000 7fae7b7dc640 1 --2- 172.21.15.31:0/4227673185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae74100c60 0x7fae7419b820 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:05.330 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.327+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 >> 172.21.15.31:0/4227673185 conn(0x7fae740fa730 msgr2=0x7fae74101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:10:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.327+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 shutdown_connections 2024-09-16T09:10:05.331 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:05.327+0000 7fae7b7dc640 1 -- 172.21.15.31:0/4227673185 wait complete. 2024-09-16T09:10:05.988 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:05 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/4227673185' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T09:10:06.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:05 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/4227673185' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T09:10:06.051 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T09:10:06.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:06.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T09:10:06.081 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T09:10:06.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:06.081 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T09:10:06.146 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T09:10:06.146 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T09:10:06.167 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-09-16T09:10:06.167 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T09:10:06.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:reef shell --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T09:10:06.434 INFO:teuthology.orchestra.run.smithi031.stderr:Inferring config /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/mon.smithi031/config 2024-09-16T09:10:07.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:06 smithi031 ceph-mon[91884]: pgmap v373: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:07.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:06 smithi097 ceph-mon[79892]: pgmap v373: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 -- 172.21.15.31:0/1126507486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0fe880 msgr2=0x7f8a4c0fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/1126507486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0fe880 0x7f8a4c0fec60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8a48009880 tx=0x7f8a4802a4a0 comp rx=0 tx=0).stop 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 -- 172.21.15.31:0/1126507486 shutdown_connections 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/1126507486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0ff230 0x7f8a4c10bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/1126507486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0fe880 0x7f8a4c0fec60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 -- 172.21.15.31:0/1126507486 >> 172.21.15.31:0/1126507486 conn(0x7f8a4c0fa730 msgr2=0x7f8a4c0fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 -- 172.21.15.31:0/1126507486 shutdown_connections 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.997+0000 7f8a53ba2640 1 -- 172.21.15.31:0/1126507486 wait complete. 2024-09-16T09:10:07.999 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a53ba2640 1 Processor -- start 2024-09-16T09:10:08.000 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a53ba2640 1 -- start start 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a53ba2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0fe880 0x7f8a4c19d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a53ba2640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0ff230 0x7f8a4c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a53ba2640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c19e490 con 0x7f8a4c0ff230 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a53ba2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4c197ab0 con 0x7f8a4c0fe880 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a5113d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0ff230 0x7f8a4c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.998+0000 7f8a5113d640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0ff230 0x7f8a4c19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:56420/0 (socket says 172.21.15.31:56420) 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a5113d640 1 -- 172.21.15.31:0/817149633 learned_addr learned my addr 172.21.15.31:0/817149633 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a5193e640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0fe880 0x7f8a4c19d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a5113d640 1 -- 172.21.15.31:0/817149633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0fe880 msgr2=0x7f8a4c19d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:08.001 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a5113d640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0fe880 0x7f8a4c19d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a5113d640 1 -- 172.21.15.31:0/817149633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a48009560 con 0x7f8a4c0ff230 2024-09-16T09:10:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a5113d640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0ff230 0x7f8a4c19df00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c8a0 tx=0x7f8a3c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:10:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a42ffd640 1 -- 172.21.15.31:0/817149633 <== mon.0 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c007b80 con 0x7f8a4c0ff230 2024-09-16T09:10:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a42ffd640 1 -- 172.21.15.31:0/817149633 <== mon.0 v2:172.21.15.31:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a3c007ce0 con 0x7f8a4c0ff230 2024-09-16T09:10:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4c197d90 con 0x7f8a4c0ff230 2024-09-16T09:10:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a42ffd640 1 -- 172.21.15.31:0/817149633 <== mon.0 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c018810 con 0x7f8a4c0ff230 2024-09-16T09:10:08.002 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:07.999+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4c1982e0 con 0x7f8a4c0ff230 2024-09-16T09:10:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.000+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4c068d30 con 0x7f8a4c0ff230 2024-09-16T09:10:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.001+0000 7f8a42ffd640 1 -- 172.21.15.31:0/817149633 <== mon.0 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 39) v1 ==== 103166+0+0 (secure 0 0 0) 0x7f8a3c018a80 con 0x7f8a4c0ff230 2024-09-16T09:10:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.001+0000 7f8a42ffd640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8a3807af30 0x7f8a3807d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T09:10:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.002+0000 7f8a42ffd640 1 -- 172.21.15.31:0/817149633 <== mon.0 v2:172.21.15.31:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6825+0+0 (secure 0 0 0) 0x7f8a3c09e340 con 0x7f8a4c0ff230 2024-09-16T09:10:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.002+0000 7f8a5193e640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8a3807af30 0x7f8a3807d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T09:10:08.003 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.002+0000 7f8a5193e640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8a3807af30 0x7f8a3807d3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000c00 tx=0x7f8a48005a20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T09:10:08.007 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.006+0000 7f8a42ffd640 1 -- 172.21.15.31:0/817149633 <== mon.0 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+196987 (secure 0 0 0) 0x7f8a3c061310 con 0x7f8a4c0ff230 2024-09-16T09:10:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.206+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8a4c0fec60 con 0x7f8a4c0ff230 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.207+0000 7f8a42ffd640 1 -- 172.21.15.31:0/817149633 <== mon.0 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v99) v1 ==== 81+0+2535 (secure 0 0 0) 0x7f8a3c064430 con 0x7f8a4c0ff230 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6827/859148422 2024-09-17T09:05:16.133859+0000 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2307887580 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2307887580 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/662233816 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3400454826 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6829/1804644990 2024-09-17T09:04:42.710262+0000 2024-09-16T09:10:08.208 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3244157830 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/259397163 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/4251203839 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6829/3476759025 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6828/1804644990 2024-09-17T09:04:42.710262+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/838457214 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/625663167 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/1878583845 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/2994879783 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/4102636839 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/202749585 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/489385232 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/2752878219 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2518533508 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:08.209 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2012899681 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/3080093455 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2103860772 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/488511267 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2737304770 2024-09-17T08:59:23.235728+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/3080093455 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/2162767536 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2552418024 2024-09-17T08:49:08.003661+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6826/2048602032 2024-09-17T08:54:52.880019+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:0/1249429801 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2103860772 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6827/2048602032 2024-09-17T08:54:52.880019+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6826/859148422 2024-09-17T09:05:16.133859+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/3789167564 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:08.210 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/494755020 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.97:6828/3476759025 2024-09-17T08:58:28.422583+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6801/2575391456 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/1714541219 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/494755020 2024-09-17T08:48:00.118833+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/320751586 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/2621367533 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:6800/2575391456 2024-09-17T08:57:02.693937+0000 2024-09-16T09:10:08.211 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.15.31:0/596031472 2024-09-17T08:47:27.231716+0000 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:listed 43 entries 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8a3807af30 msgr2=0x7f8a3807d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8a3807af30 0x7f8a3807d3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000c00 tx=0x7f8a48005a20 comp rx=0 tx=0).stop 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0ff230 msgr2=0x7f8a4c19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0ff230 0x7f8a4c19df00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00c8a0 tx=0x7f8a3c00cd60 comp rx=0 tx=0).stop 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 shutdown_connections 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:6800/3442236616,v1:172.21.15.31:6801/3442236616] conn(0x7f8a3807af30 0x7f8a3807d3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a4c0ff230 0x7f8a4c19df00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 --2- 172.21.15.31:0/817149633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a4c0fe880 0x7f8a4c19d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 >> 172.21.15.31:0/817149633 conn(0x7f8a4c0fa730 msgr2=0x7f8a4c0fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T09:10:08.212 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 shutdown_connections 2024-09-16T09:10:08.213 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-16T09:10:08.210+0000 7f8a53ba2640 1 -- 172.21.15.31:0/817149633 wait complete. 2024-09-16T09:10:08.850 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-16T09:10:08.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:08.850 DEBUG:teuthology.orchestra.run.smithi097:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-16T09:10:08.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi097.front.sepia.ceph.com 2024-09-16T09:10:08.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:08.884 DEBUG:teuthology.orchestra.run.smithi097:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-16T09:10:09.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:09.005 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2024-09-16T09:10:09.023 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T09:10:09.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:09.023 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T09:10:09.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:08 smithi031 ceph-mon[91884]: pgmap v374: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:09.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:08 smithi031 ceph-mon[91884]: from='client.? 172.21.15.31:0/817149633' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T09:10:09.087 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:08 smithi031 ceph-mon[91884]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:10:09.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:09.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2024-09-16T09:10:09.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:09.194 DEBUG:teuthology.orchestra.run.smithi097:> ip netns list 2024-09-16T09:10:09.213 INFO:teuthology.orchestra.run.smithi097.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-16T09:10:09.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:09.213 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-16T09:10:09.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:08 smithi097 ceph-mon[79892]: pgmap v374: 65 pgs: 65 active+clean; 201 MiB data, 873 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:10:09.261 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:08 smithi097 ceph-mon[79892]: from='client.? 172.21.15.31:0/817149633' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T09:10:09.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:08 smithi097 ceph-mon[79892]: from='mgr.34104 172.21.15.31:0/3175641505' entity='mgr.smithi031.dcmqpv' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:10:09.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T09:10:09.284 DEBUG:teuthology.orchestra.run.smithi097:> sudo ip link delete ceph-brx 2024-09-16T09:10:09.387 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T09:10:09.398 INFO:tasks.cephadm:Teardown begin 2024-09-16T09:10:09.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:10:09.429 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:10:09.455 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T09:10:09.456 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T09:10:09.485 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T09:10:09.512 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T09:10:09.512 INFO:tasks.cephadm.mon.smithi031:Stopping mon.smithi031... 2024-09-16T09:10:09.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031 2024-09-16T09:10:09.849 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:09 smithi031 systemd[1]: Stopping Ceph mon.smithi031 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:09 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[91880]: 2024-09-16T09:10:09.847+0000 7faf8e68f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi031 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:09 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031[91880]: 2024-09-16T09:10:09.847+0000 7faf8e68f640 -1 mon.smithi031@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T09:10:10.123 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:09 smithi031 podman[150577]: 2024-09-16 09:10:09.878814024 +0000 UTC m=+0.160968682 container died 13ee87bf1609afec3a2588ebde503d047eac9fd7d2a3f8d233d1b5b44066f924 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:10.549 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:10 smithi031 podman[150577]: 2024-09-16 09:10:10.272812954 +0000 UTC m=+0.554967612 container cleanup 13ee87bf1609afec3a2588ebde503d047eac9fd7d2a3f8d233d1b5b44066f924 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T09:10:10.550 INFO:journalctl@ceph.mon.smithi031.smithi031.stdout:Sep 16 09:10:10 smithi031 bash[150577]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi031 2024-09-16T09:10:10.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi031.service' 2024-09-16T09:10:10.767 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:10.767 INFO:tasks.cephadm.mon.smithi031:Stopped mon.smithi031 2024-09-16T09:10:10.767 INFO:tasks.cephadm.mon.smithi097:Stopping mon.smithi097... 2024-09-16T09:10:10.768 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi097 2024-09-16T09:10:11.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:10 smithi097 systemd[1]: Stopping Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:11.328 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:11 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097[79888]: 2024-09-16T09:10:11.072+0000 7fa7cdc3b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi097 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:11.328 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:11 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097[79888]: 2024-09-16T09:10:11.072+0000 7fa7cdc3b640 -1 mon.smithi097@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T09:10:11.633 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:11 smithi097 podman[118627]: 2024-09-16 09:10:11.328293552 +0000 UTC m=+0.369422724 container died ab1d49446e3799e5a12343174d4256f6bbed954bc5f8b86e7538b33d90049c51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T09:10:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:11 smithi097 podman[118627]: 2024-09-16 09:10:11.814772922 +0000 UTC m=+0.855902093 container cleanup ab1d49446e3799e5a12343174d4256f6bbed954bc5f8b86e7538b33d90049c51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-16T09:10:12.012 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:11 smithi097 bash[118627]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097 2024-09-16T09:10:12.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:12 smithi097 podman[118640]: 2024-09-16 09:10:12.091666594 +0000 UTC m=+0.758602549 container remove ab1d49446e3799e5a12343174d4256f6bbed954bc5f8b86e7538b33d90049c51 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-mon-smithi097, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:10:12.262 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:12 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi097.service: Deactivated successfully. 2024-09-16T09:10:12.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:12 smithi097 systemd[1]: Stopped Ceph mon.smithi097 for 1631ad7a-7408-11ef-bceb-c7b262605968. 2024-09-16T09:10:12.263 INFO:journalctl@ceph.mon.smithi097.smithi097.stdout:Sep 16 09:10:12 smithi097 systemd[1]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi097.service: Consumed 6.038s CPU time. 2024-09-16T09:10:12.264 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@mon.smithi097.service' 2024-09-16T09:10:12.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:12.311 INFO:tasks.cephadm.mon.smithi097:Stopped mon.smithi097 2024-09-16T09:10:12.311 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T09:10:12.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.0 2024-09-16T09:10:12.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:12 smithi031 systemd[1]: Stopping Ceph osd.0 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:13.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:12 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[99724]: 2024-09-16T09:10:12.651+0000 7fc91245f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:13.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:12 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[99724]: 2024-09-16T09:10:12.652+0000 7fc91245f640 -1 osd.0 99 *** Got signal Terminated *** 2024-09-16T09:10:13.123 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:12 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0[99724]: 2024-09-16T09:10:12.652+0000 7fc91245f640 -1 osd.0 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:10:18.345 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:18 smithi031 podman[150710]: 2024-09-16 09:10:18.027562001 +0000 UTC m=+5.518983092 container died 8060870d2228a307a6817b56f152ce5bb290fbc5c6b330d4849dbb9cb88d75e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:18.345 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:18 smithi031 podman[150710]: 2024-09-16 09:10:18.344136365 +0000 UTC m=+5.835557455 container cleanup 8060870d2228a307a6817b56f152ce5bb290fbc5c6b330d4849dbb9cb88d75e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:10:18.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:18 smithi031 bash[150710]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0 2024-09-16T09:10:18.623 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:18 smithi031 podman[150725]: 2024-09-16 09:10:18.587330073 +0000 UTC m=+0.555028202 container remove 8060870d2228a307a6817b56f152ce5bb290fbc5c6b330d4849dbb9cb88d75e8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T09:10:19.274 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:19 smithi031 podman[150816]: 2024-09-16 09:10:18.914843963 +0000 UTC m=+0.016679147 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:10:19.275 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:19 smithi031 podman[150816]: 2024-09-16 09:10:19.021615167 +0000 UTC m=+0.123450381 container create c6a93e452e581a3eff683112e2c04839f20666c2a62ba58a16dddb42ab7084a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-16T09:10:19.275 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:19 smithi031 podman[150816]: 2024-09-16 09:10:19.274262127 +0000 UTC m=+0.376097302 container init c6a93e452e581a3eff683112e2c04839f20666c2a62ba58a16dddb42ab7084a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:10:19.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:19 smithi031 podman[150816]: 2024-09-16 09:10:19.278125766 +0000 UTC m=+0.379960942 container start c6a93e452e581a3eff683112e2c04839f20666c2a62ba58a16dddb42ab7084a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T09:10:19.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:19 smithi031 podman[150816]: 2024-09-16 09:10:19.374219506 +0000 UTC m=+0.476054688 container attach c6a93e452e581a3eff683112e2c04839f20666c2a62ba58a16dddb42ab7084a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:10:19.626 INFO:journalctl@ceph.osd.0.smithi031.stdout:Sep 16 09:10:19 smithi031 podman[150816]: 2024-09-16 09:10:19.47122171 +0000 UTC m=+0.573056895 container died c6a93e452e581a3eff683112e2c04839f20666c2a62ba58a16dddb42ab7084a7 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-16T09:10:20.044 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T09:10:20.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:20.087 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T09:10:20.087 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T09:10:20.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.1 2024-09-16T09:10:20.373 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:20 smithi031 systemd[1]: Stopping Ceph osd.1 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:20.873 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:20 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[104919]: 2024-09-16T09:10:20.421+0000 7f177d551640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:20.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:20 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[104919]: 2024-09-16T09:10:20.421+0000 7f177d551640 -1 osd.1 99 *** Got signal Terminated *** 2024-09-16T09:10:20.874 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:20 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1[104919]: 2024-09-16T09:10:20.421+0000 7f177d551640 -1 osd.1 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:10:26.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:25 smithi031 podman[150917]: 2024-09-16 09:10:25.860752323 +0000 UTC m=+5.549108413 container died 539fc5d4e6cf77ae60e41d4b0849c29ddf41adcb7222f7e5e7bde7b04cfdbeb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T09:10:26.441 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:26 smithi031 podman[150917]: 2024-09-16 09:10:26.272817276 +0000 UTC m=+5.961173366 container cleanup 539fc5d4e6cf77ae60e41d4b0849c29ddf41adcb7222f7e5e7bde7b04cfdbeb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T09:10:26.441 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:26 smithi031 bash[150917]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1 2024-09-16T09:10:26.850 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:26 smithi031 podman[150932]: 2024-09-16 09:10:26.440869513 +0000 UTC m=+0.569920271 container remove 539fc5d4e6cf77ae60e41d4b0849c29ddf41adcb7222f7e5e7bde7b04cfdbeb6 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:27.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:26 smithi031 podman[151024]: 2024-09-16 09:10:26.849928902 +0000 UTC m=+0.106816795 container create aa187467fa21fcfc767e5b0f921c2ec519569421649b8ac97d392bef44ef0c08 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-16T09:10:27.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:26 smithi031 podman[151024]: 2024-09-16 09:10:26.760037542 +0000 UTC m=+0.016925434 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:10:27.123 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:27 smithi031 podman[151024]: 2024-09-16 09:10:27.069137487 +0000 UTC m=+0.326025382 container init aa187467fa21fcfc767e5b0f921c2ec519569421649b8ac97d392bef44ef0c08 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T09:10:27.124 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:27 smithi031 podman[151024]: 2024-09-16 09:10:27.072624666 +0000 UTC m=+0.329512567 container start aa187467fa21fcfc767e5b0f921c2ec519569421649b8ac97d392bef44ef0c08 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T09:10:27.598 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:27 smithi031 podman[151024]: 2024-09-16 09:10:27.163915712 +0000 UTC m=+0.420803623 container attach aa187467fa21fcfc767e5b0f921c2ec519569421649b8ac97d392bef44ef0c08 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T09:10:27.598 INFO:journalctl@ceph.osd.1.smithi031.stdout:Sep 16 09:10:27 smithi031 podman[151048]: 2024-09-16 09:10:27.288443776 +0000 UTC m=+0.017085130 container died aa187467fa21fcfc767e5b0f921c2ec519569421649b8ac97d392bef44ef0c08 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux ) 2024-09-16T09:10:27.902 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T09:10:27.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:27.945 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T09:10:27.946 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T09:10:27.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.2 2024-09-16T09:10:28.286 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:28 smithi031 systemd[1]: Stopping Ceph osd.2 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:28.623 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:28 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[110537]: 2024-09-16T09:10:28.284+0000 7fa8f1e6a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:28.623 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:28 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[110537]: 2024-09-16T09:10:28.284+0000 7fa8f1e6a640 -1 osd.2 99 *** Got signal Terminated *** 2024-09-16T09:10:28.623 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:28 smithi031 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2[110537]: 2024-09-16T09:10:28.284+0000 7fa8f1e6a640 -1 osd.2 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:10:33.914 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:33 smithi031 podman[151127]: 2024-09-16 09:10:33.632116728 +0000 UTC m=+5.454940696 container died cd7af82048da4de1c21ebde7f11f05ef398f32080e0d944675f02b1108ecc708 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T09:10:34.373 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:34 smithi031 podman[151127]: 2024-09-16 09:10:34.086424439 +0000 UTC m=+5.909248400 container cleanup cd7af82048da4de1c21ebde7f11f05ef398f32080e0d944675f02b1108ecc708 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T09:10:34.373 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:34 smithi031 bash[151127]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2 2024-09-16T09:10:34.768 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:34 smithi031 podman[151155]: 2024-09-16 09:10:34.404728108 +0000 UTC m=+0.770155487 container remove cd7af82048da4de1c21ebde7f11f05ef398f32080e0d944675f02b1108ecc708 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-16T09:10:35.116 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:34 smithi031 podman[151241]: 2024-09-16 09:10:34.671657774 +0000 UTC m=+0.017119069 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:10:35.117 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:34 smithi031 podman[151241]: 2024-09-16 09:10:34.780617775 +0000 UTC m=+0.126079102 container create 3825beb859c4dd37300c63eeac9dab8530c19dc584c68b2b77e685651b1a0724 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:35.117 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:35 smithi031 podman[151241]: 2024-09-16 09:10:35.007990781 +0000 UTC m=+0.353452067 container init 3825beb859c4dd37300c63eeac9dab8530c19dc584c68b2b77e685651b1a0724 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T09:10:35.117 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:35 smithi031 podman[151241]: 2024-09-16 09:10:35.011472202 +0000 UTC m=+0.356933491 container start 3825beb859c4dd37300c63eeac9dab8530c19dc584c68b2b77e685651b1a0724 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T09:10:35.117 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:35 smithi031 podman[151241]: 2024-09-16 09:10:35.116036279 +0000 UTC m=+0.461497582 container attach 3825beb859c4dd37300c63eeac9dab8530c19dc584c68b2b77e685651b1a0724 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2) 2024-09-16T09:10:35.373 INFO:journalctl@ceph.osd.2.smithi031.stdout:Sep 16 09:10:35 smithi031 podman[151241]: 2024-09-16 09:10:35.20796378 +0000 UTC m=+0.553425078 container died 3825beb859c4dd37300c63eeac9dab8530c19dc584c68b2b77e685651b1a0724 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T09:10:35.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T09:10:35.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:35.950 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T09:10:35.950 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T09:10:35.950 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.3 2024-09-16T09:10:36.262 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:35 smithi097 systemd[1]: Stopping Ceph osd.3 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:36.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:36 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:10:36.276+0000 7f22c967f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:36.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:36 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:10:36.276+0000 7f22c967f640 -1 osd.3 99 *** Got signal Terminated *** 2024-09-16T09:10:36.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:36 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:10:36.276+0000 7f22c967f640 -1 osd.3 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:10:38.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:38 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:38.491+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:39.761 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:39 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:39.507+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:40.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:40 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:40.479+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:40.763 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:40 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:40.627+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:41.264 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:40 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3[88780]: 2024-09-16T09:10:40.961+0000 7f22c5c94640 -1 osd.3 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.053158+0000 front 2024-09-16T09:10:16.053210+0000 (oldest deadline 2024-09-16T09:10:40.753107+0000) 2024-09-16T09:10:41.762 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:41 smithi097 podman[118764]: 2024-09-16 09:10:41.66905464 +0000 UTC m=+5.522900464 container died 9d23936a9f41b9ac67e67f502af668cc92406d70b29dcab364a861ea7eebd3b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2) 2024-09-16T09:10:41.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:41 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:41.452+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:41.763 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:41 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:41.661+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:42.262 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:42 smithi097 podman[118764]: 2024-09-16 09:10:42.102346964 +0000 UTC m=+5.956192787 container cleanup 9d23936a9f41b9ac67e67f502af668cc92406d70b29dcab364a861ea7eebd3b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:10:42.262 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:42 smithi097 bash[118764]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3 2024-09-16T09:10:42.683 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:42 smithi097 podman[118779]: 2024-09-16 09:10:42.320553914 +0000 UTC m=+0.642727397 container remove 9d23936a9f41b9ac67e67f502af668cc92406d70b29dcab364a861ea7eebd3b2 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:42.683 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:42 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:42.485+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:42.937 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:42 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:42.681+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:42.937 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:42 smithi097 podman[118868]: 2024-09-16 09:10:42.664411424 +0000 UTC m=+0.015814423 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:10:42.937 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:42 smithi097 podman[118868]: 2024-09-16 09:10:42.804822681 +0000 UTC m=+0.156225672 container create 754fb1e66dae1342c54935c8c2821b3262b0261c7971d47d54d4bb4bbcc3c6ec (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T09:10:43.262 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:43 smithi097 podman[118868]: 2024-09-16 09:10:43.099808382 +0000 UTC m=+0.451211376 container init 754fb1e66dae1342c54935c8c2821b3262b0261c7971d47d54d4bb4bbcc3c6ec (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-16T09:10:43.262 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:43 smithi097 podman[118868]: 2024-09-16 09:10:43.103084983 +0000 UTC m=+0.454487990 container start 754fb1e66dae1342c54935c8c2821b3262b0261c7971d47d54d4bb4bbcc3c6ec (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:10:43.262 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:43 smithi097 podman[118868]: 2024-09-16 09:10:43.182837757 +0000 UTC m=+0.534240747 container attach 754fb1e66dae1342c54935c8c2821b3262b0261c7971d47d54d4bb4bbcc3c6ec (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T09:10:43.553 INFO:journalctl@ceph.osd.3.smithi097.stdout:Sep 16 09:10:43 smithi097 podman[118868]: 2024-09-16 09:10:43.291080663 +0000 UTC m=+0.642483664 container died 754fb1e66dae1342c54935c8c2821b3262b0261c7971d47d54d4bb4bbcc3c6ec (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-16T09:10:43.554 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:43 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:43.477+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:43.887 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:43 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:43.727+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:43.905 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T09:10:43.945 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:43.946 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T09:10:43.946 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T09:10:43.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.4 2024-09-16T09:10:44.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:44 smithi097 systemd[1]: Stopping Ceph osd.4 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:44.537 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:44 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:44.284+0000 7faf1ccf6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:44.537 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:44 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:44.284+0000 7faf1ccf6640 -1 osd.4 99 *** Got signal Terminated *** 2024-09-16T09:10:44.537 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:44 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:44.284+0000 7faf1ccf6640 -1 osd.4 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:10:44.537 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:44 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:44.491+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:45.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:44 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:44.707+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:46.012 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:45 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:45.537+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:46.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:45 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:45.686+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:46.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:46 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:46.490+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:46.762 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:46 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:46.729+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:48.012 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:47 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:47.515+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:48.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:47 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:47.751+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:48.753 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:48 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:48.468+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:14.985511+0000 front 2024-09-16T09:10:14.985525+0000 (oldest deadline 2024-09-16T09:10:38.485426+0000) 2024-09-16T09:10:48.753 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:48 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4[93798]: 2024-09-16T09:10:48.468+0000 7faf1930b640 -1 osd.4 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:24.286515+0000 front 2024-09-16T09:10:24.286484+0000 (oldest deadline 2024-09-16T09:10:48.386455+0000) 2024-09-16T09:10:49.011 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:48 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:48.751+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:49.992 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:49 smithi097 podman[118982]: 2024-09-16 09:10:49.697209407 +0000 UTC m=+5.516501873 container died 3c82b6b87cd9abfceee7117bb7e55914dd05350e0161ff1824b10bb4f2161880 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-16T09:10:49.992 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:49 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:49.787+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:49.992 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:49 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:49.787+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:50.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:50 smithi097 podman[118982]: 2024-09-16 09:10:50.173430602 +0000 UTC m=+5.992723068 container cleanup 3c82b6b87cd9abfceee7117bb7e55914dd05350e0161ff1824b10bb4f2161880 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:10:50.262 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:50 smithi097 bash[118982]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4 2024-09-16T09:10:50.762 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:50 smithi097 podman[118998]: 2024-09-16 09:10:50.458399749 +0000 UTC m=+0.758757124 container remove 3c82b6b87cd9abfceee7117bb7e55914dd05350e0161ff1824b10bb4f2161880 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T09:10:51.112 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:50 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:50.789+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:51.112 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:50 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:50.789+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:51.112 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:50 smithi097 podman[119084]: 2024-09-16 09:10:50.689737138 +0000 UTC m=+0.016165345 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:10:51.112 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:50 smithi097 podman[119084]: 2024-09-16 09:10:50.809319141 +0000 UTC m=+0.135747335 container create ecd3401c0b4cc4d602087d5f3556115892fefd58fbc5ff1a57f732ea1cf70f03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T09:10:51.112 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:51 smithi097 podman[119084]: 2024-09-16 09:10:51.028203791 +0000 UTC m=+0.354631995 container init ecd3401c0b4cc4d602087d5f3556115892fefd58fbc5ff1a57f732ea1cf70f03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T09:10:51.113 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:51 smithi097 podman[119084]: 2024-09-16 09:10:51.031725731 +0000 UTC m=+0.358153931 container start ecd3401c0b4cc4d602087d5f3556115892fefd58fbc5ff1a57f732ea1cf70f03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=) 2024-09-16T09:10:51.113 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:51 smithi097 podman[119084]: 2024-09-16 09:10:51.111542966 +0000 UTC m=+0.437971175 container attach ecd3401c0b4cc4d602087d5f3556115892fefd58fbc5ff1a57f732ea1cf70f03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:51.512 INFO:journalctl@ceph.osd.4.smithi097.stdout:Sep 16 09:10:51 smithi097 podman[119084]: 2024-09-16 09:10:51.223521893 +0000 UTC m=+0.549950095 container died ecd3401c0b4cc4d602087d5f3556115892fefd58fbc5ff1a57f732ea1cf70f03 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:10:51.808 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:51 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:51.746+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:51.808 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:51 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:51.746+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:51.835 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T09:10:51.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:51.876 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T09:10:51.876 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T09:10:51.877 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.5 2024-09-16T09:10:52.236 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:51 smithi097 systemd[1]: Stopping Ceph osd.5 for 1631ad7a-7408-11ef-bceb-c7b262605968... 2024-09-16T09:10:52.512 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:52 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:52.234+0000 7f076d3b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:10:52.512 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:52 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:52.234+0000 7f076d3b7640 -1 osd.5 99 *** Got signal Terminated *** 2024-09-16T09:10:52.512 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:52 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:52.234+0000 7f076d3b7640 -1 osd.5 99 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:10:53.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:52 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:52.725+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:53.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:52 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:52.725+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:54.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:53 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:53.758+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:54.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:53 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:53.758+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:55.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:54 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:54.792+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:55.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:54 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:54.792+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:56.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:55 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:55.746+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:56.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:55 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:55.746+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:56.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:55 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:55.746+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6822 osd.2 since back 2024-09-16T09:10:28.981651+0000 front 2024-09-16T09:10:28.981667+0000 (oldest deadline 2024-09-16T09:10:54.881537+0000) 2024-09-16T09:10:57.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:56 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:56.707+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6806 osd.0 since back 2024-09-16T09:10:16.780210+0000 front 2024-09-16T09:10:16.780054+0000 (oldest deadline 2024-09-16T09:10:40.180418+0000) 2024-09-16T09:10:57.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:56 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:56.707+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6814 osd.1 since back 2024-09-16T09:10:20.181008+0000 front 2024-09-16T09:10:20.180973+0000 (oldest deadline 2024-09-16T09:10:48.981220+0000) 2024-09-16T09:10:57.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:56 smithi097 ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5[98927]: 2024-09-16T09:10:56.707+0000 7f07659c4640 -1 osd.5 99 heartbeat_check: no reply from 172.21.15.31:6822 osd.2 since back 2024-09-16T09:10:28.981651+0000 front 2024-09-16T09:10:28.981667+0000 (oldest deadline 2024-09-16T09:10:54.881537+0000) 2024-09-16T09:10:57.961 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:57 smithi097 podman[119186]: 2024-09-16 09:10:57.644957496 +0000 UTC m=+5.539909863 container died 857ff6eb4b2a677cc124bfc3eea5783dc6b5bfc19d3cc128c786177f5ae11ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:10:58.262 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:58 smithi097 podman[119186]: 2024-09-16 09:10:58.118615577 +0000 UTC m=+6.013567985 container cleanup 857ff6eb4b2a677cc124bfc3eea5783dc6b5bfc19d3cc128c786177f5ae11ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True) 2024-09-16T09:10:58.262 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:58 smithi097 bash[119186]: ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5 2024-09-16T09:10:58.740 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:58 smithi097 podman[119202]: 2024-09-16 09:10:58.428327092 +0000 UTC m=+0.773002283 container remove 857ff6eb4b2a677cc124bfc3eea5783dc6b5bfc19d3cc128c786177f5ae11ff8 (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:10:59.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:58 smithi097 podman[119289]: 2024-09-16 09:10:58.642754317 +0000 UTC m=+0.016028405 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:10:59.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:58 smithi097 podman[119289]: 2024-09-16 09:10:58.754140422 +0000 UTC m=+0.127414541 container create 0deffd2770fc178bf84780fd827cc03f8908be3084372c964d8db6f13cc1d7fd (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:10:59.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:58 smithi097 podman[119289]: 2024-09-16 09:10:58.973412558 +0000 UTC m=+0.346686633 container init 0deffd2770fc178bf84780fd827cc03f8908be3084372c964d8db6f13cc1d7fd (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-16T09:10:59.012 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:58 smithi097 podman[119289]: 2024-09-16 09:10:58.976670113 +0000 UTC m=+0.349944187 container start 0deffd2770fc178bf84780fd827cc03f8908be3084372c964d8db6f13cc1d7fd (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:59.460 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:59 smithi097 podman[119289]: 2024-09-16 09:10:59.056877968 +0000 UTC m=+0.430152053 container attach 0deffd2770fc178bf84780fd827cc03f8908be3084372c964d8db6f13cc1d7fd (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:10:59.461 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:59 smithi097 podman[119289]: 2024-09-16 09:10:59.1660832 +0000 UTC m=+0.539357274 container died 0deffd2770fc178bf84780fd827cc03f8908be3084372c964d8db6f13cc1d7fd (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T09:10:59.722 INFO:journalctl@ceph.osd.5.smithi097.stdout:Sep 16 09:10:59 smithi097 podman[119313]: 2024-09-16 09:10:59.701967764 +0000 UTC m=+0.529382329 container remove 0deffd2770fc178bf84780fd827cc03f8908be3084372c964d8db6f13cc1d7fd (image=quay.ceph.io/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-1631ad7a-7408-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:10:59.727 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1631ad7a-7408-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T09:10:59.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:10:59.765 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T09:10:59.765 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T09:10:59.929 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T09:11:22.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T09:11:22.624 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T09:11:34.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:11:34.119 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:11:34.147 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T09:11:34.149 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977/remote/smithi031/crash 2024-09-16T09:11:34.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/crash -- . 2024-09-16T09:11:34.187 INFO:teuthology.orchestra.run.smithi031.stderr:tar: /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T09:11:34.187 INFO:teuthology.orchestra.run.smithi031.stderr:tar: Error is not recoverable: exiting now 2024-09-16T09:11:34.189 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977/remote/smithi097/crash 2024-09-16T09:11:34.190 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/crash -- . 2024-09-16T09:11:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T09:11:34.214 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-09-16T09:11:34.215 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T09:11:34.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-16T09:11:34.272 INFO:tasks.cephadm:Compressing logs... 2024-09-16T09:11:34.273 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T09:11:34.318 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T09:11:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi097.log 2024-09-16T09:11:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:: No such file or directory 2024-09-16T09:11:34.342 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T09:11:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T09:11:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi097.log: /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T09:11:34.343 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T09:11:34.348 INFO:teuthology.orchestra.run.smithi031.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-09-16T09:11:34.349 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T09:11:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T09:11:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mon.smithi031.log 2024-09-16T09:11:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T09:11:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mon.smithi031.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi031.mjhxgi.log 2024-09-16T09:11:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T09:11:34.349 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi031.mjhxgi.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T09:11:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T09:11:34.350 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.cephadm.log: /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.2.log: 86.9% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T09:11:34.352 INFO:teuthology.orchestra.run.smithi031.stderr: 91.6% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T09:11:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi097.sbwmfq.log 2024-09-16T09:11:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T09:11:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi097.sbwmfq.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.log 2024-09-16T09:11:34.355 INFO:teuthology.orchestra.run.smithi097.stderr: 93.9% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi097.log.gz 2024-09-16T09:11:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.5.log: 85.9% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T09:11:34.355 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mon.smithi097.log 2024-09-16T09:11:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mgr.smithi097.mpashf.log 2024-09-16T09:11:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mon.smithi097.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi097.cagmus.log 2024-09-16T09:11:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mgr.smithi097.mpashf.log: 88.2% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T09:11:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T09:11:34.356 INFO:teuthology.orchestra.run.smithi097.stderr: 91.7% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T09:11:34.356 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi097.cagmus.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T09:11:34.358 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mgr.smithi031.dcmqpv.log 2024-09-16T09:11:34.358 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.log 2024-09-16T09:11:34.358 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mgr.smithi031.dcmqpv.log: gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi031.lcfutm.log 2024-09-16T09:11:34.359 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.log: 88.1% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T09:11:34.360 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi031.log 2024-09-16T09:11:34.366 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.3.log: 91.1% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T09:11:34.369 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T09:11:34.370 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi031.lcfutm.log: 91.2% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T09:11:34.370 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T09:11:34.373 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi031.log: 93.9% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-client.ceph-exporter.smithi031.log.gz 2024-09-16T09:11:34.394 INFO:teuthology.orchestra.run.smithi097.stderr: 92.7% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi097.cagmus.log.gz 2024-09-16T09:11:34.399 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/cephadm.log: 89.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T09:11:34.400 INFO:teuthology.orchestra.run.smithi097.stderr: 89.0% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mgr.smithi097.mpashf.log.gz 2024-09-16T09:11:34.762 INFO:teuthology.orchestra.run.smithi031.stderr: 88.6% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mgr.smithi031.dcmqpv.log.gz 2024-09-16T09:11:34.766 INFO:teuthology.orchestra.run.smithi097.stderr: 92.5% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mon.smithi097.log.gz 2024-09-16T09:11:35.150 INFO:teuthology.orchestra.run.smithi031.stderr: 90.5% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mon.smithi031.log.gz 2024-09-16T09:11:38.579 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T09:11:39.638 INFO:teuthology.orchestra.run.smithi097.stderr: 94.8% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi097.sbwmfq.log.gz 2024-09-16T09:11:39.781 INFO:teuthology.orchestra.run.smithi031.stderr: 93.5% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T09:11:40.208 INFO:teuthology.orchestra.run.smithi031.stderr: 93.6% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T09:11:40.358 INFO:teuthology.orchestra.run.smithi031.stderr: 93.6% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T09:11:40.675 INFO:teuthology.orchestra.run.smithi097.stderr: 93.6% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T09:11:41.444 INFO:teuthology.orchestra.run.smithi097.stderr: 93.1% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T09:11:41.445 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-09-16T09:11:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m7.114s 2024-09-16T09:11:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m22.658s 2024-09-16T09:11:41.446 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m0.841s 2024-09-16T09:11:41.678 INFO:teuthology.orchestra.run.smithi031.stderr: 94.8% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi031.lcfutm.log.gz 2024-09-16T09:13:09.672 INFO:teuthology.orchestra.run.smithi031.stderr: 92.5% -- replaced with /var/log/ceph/1631ad7a-7408-11ef-bceb-c7b262605968/ceph-mds.cephfs.smithi031.mjhxgi.log.gz 2024-09-16T09:13:09.673 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-16T09:13:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:real 1m35.341s 2024-09-16T09:13:09.673 INFO:teuthology.orchestra.run.smithi031.stderr:user 1m57.744s 2024-09-16T09:13:09.674 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m3.243s 2024-09-16T09:13:09.674 INFO:tasks.cephadm:Archiving logs... 2024-09-16T09:13:09.674 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977/remote/smithi031/log 2024-09-16T09:13:09.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T09:13:26.888 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977/remote/smithi097/log 2024-09-16T09:13:26.889 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T09:13:30.287 INFO:tasks.cephadm:Removing cluster... 2024-09-16T09:13:30.287 DEBUG:teuthology.orchestra.run.smithi031:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 --force 2024-09-16T09:13:30.445 INFO:teuthology.orchestra.run.smithi031.stdout:Deleting cluster with fsid: 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T09:13:30.854 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1631ad7a-7408-11ef-bceb-c7b262605968 --force 2024-09-16T09:13:30.999 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: 1631ad7a-7408-11ef-bceb-c7b262605968 2024-09-16T09:13:31.289 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T09:13:31.290 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T09:13:31.306 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T09:13:31.322 INFO:tasks.cephadm:Teardown complete 2024-09-16T09:13:31.322 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T09:13:31.333 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T09:13:31.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T09:13:31.350 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T09:13:31.439 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T09:13:31.450 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T09:13:31.450 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T09:13:31.453 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T09:13:31.466 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-09-16T09:13:31.467 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-09-16T09:13:31.471 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T09:13:31.471 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-09-16T09:13:31.471 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 -27us[ -25us] +/- 59ms 2024-09-16T09:13:31.471 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 -150us[ -148us] +/- 78ms 2024-09-16T09:13:31.471 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +148us[ +148us] +/- 69ms 2024-09-16T09:13:31.471 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:13:31.472 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T09:13:31.472 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-09-16T09:13:31.472 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 -21us[-5936ns] +/- 59ms 2024-09-16T09:13:31.473 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 29 -139us[ -139us] +/- 78ms 2024-09-16T09:13:31.473 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +149us[ +164us] +/- 69ms 2024-09-16T09:13:31.473 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:13:31.473 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T09:13:31.485 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T09:13:31.486 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T09:13:31.507 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T09:13:31.595 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T09:13:31.669 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T09:13:31.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:13:31.699 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T09:13:31.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:13:31.727 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T09:13:31.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T09:13:31.752 INFO:teuthology.task.internal:Duration was 2009.944614 seconds 2024-09-16T09:13:31.752 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T09:13:31.764 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T09:13:31.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T09:13:31.767 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T09:13:31.806 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T09:13:31.806 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T09:13:32.161 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T09:13:32.161 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-16T09:13:32.161 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T09:13:32.225 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-09-16T09:13:32.225 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T09:13:32.264 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T09:13:32.264 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T09:13:32.269 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T09:13:32.330 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T09:13:32.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T09:13:32.336 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T09:13:33.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T09:13:33.067 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T09:13:33.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T09:13:33.094 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T09:13:33.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T09:13:33.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:13:33.136 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:13:33.157 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-16T09:13:33.185 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-09-16T09:13:33.199 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:13:33.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:13:33.227 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T09:13:33.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:13:33.255 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T09:13:33.269 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T09:13:33.271 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977/remote/smithi031 2024-09-16T09:13:33.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T09:13:33.321 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7906977/remote/smithi097 2024-09-16T09:13:33.322 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T09:13:33.360 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T09:13:33.360 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T09:13:33.362 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T09:13:33.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T09:13:33.429 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T09:13:33.429 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T09:13:33.440 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T09:13:33.440 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T09:13:33.443 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T09:13:33.456 INFO:teuthology.orchestra.run.smithi031.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 09:13 /home/ubuntu/cephtest 2024-09-16T09:13:33.465 INFO:teuthology.orchestra.run.smithi097.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 09:13 /home/ubuntu/cephtest 2024-09-16T09:13:33.466 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T09:13:33.478 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T09:13:33.520 INFO:teuthology.run:Summary data: description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{reef} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2009.9446144104004 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-16T09:13:33.520 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T09:13:33.603 INFO:teuthology.run:pass